flowbber.plugins.sources.json

JSON

This source fetch and parses a local or remote (http, https) json file.

Data collected:

Same as the source file

Dependencies:

pip3 install flowbber[json]

Usage:

[[sources]]
type = "json"
id = "..."

    [sources.config]
    file_uri = "file://{pipeline.dir}/file.json"
    encoding = "utf-8"
    ordered = true
    verify_ssl = true
    extract = true

{
    "sources": [
        {
            "type": "json",
            "id": "...",
            "config": {
                "file_uri": "file://{pipeline.dir}/file.json",
                "encoding": "utf-8",
                "ordered": true,
                "verify_ssl": true,
                "extract": true
            }
        }
    ]
}

file_uri

URI to the JSON file. If no schema is specified, file:// will be used.

Supported schemas:

• file:// (the default): File system path to the JSON file.

  file://path/to/file.json
  

  Or if using Substitutions:

  file://{pipeline.dir}/file.json
  

• http[s]://: URL to download the JSON file.

  https://mydomain.com/archive/file.json
  

• Default: N/A

• Optional: False

• Schema:

  {
      'type': 'string',
      'empty': False,
  }
  

• Secret: False

encoding

Encoding to use to decode the file.

• Default: utf-8

• Optional: True

• Schema:

  {
      'type': 'string',
      'empty': False,
  }
  

• Secret: False

ordered

Parses the file in order and returns it as a py:class:collections.OrderedDict instead of an unordered dictionary.

• Default: False

• Optional: True

• Schema:

  {
      'type': 'boolean'
  }
  

• Secret: False

verify_ssl

Enable or disable SSL verification. This option only applies if the https schema is used.

• Default: True

• Optional: True

• Schema:

  {
      'type': 'boolean'
  }
  

• Secret: False

extract

Extract the JSON file from a Zip archive.

If using extraction, the .zip extension will be automatically appended to the file_uri filename parameter if not present.

It is expected that the file inside the archive matches the file_uri filename without the .zip extension. For example:

extract parameter	file_uri parameter	File loaded	Expected file inside Zip
True	xxx://archive.json	xxx://archive.json.zip	archive.json
True	xxx://archive.json.zip	xxx://archive.json.zip	archive.json

• Default: False

• Optional: True

• Schema:

  {
      'type': 'boolean',
  }
  

• Secret: False

Classes

• JSONSource: Main base class to implement a Source.

class flowbber.plugins.sources.json.JSONSource(index, type_, id_, optional=False, timeout=None, config=None)

Inheritance