Aggregators

Aggregators are plugins that allows to delete, add or modify the data collected by the sources.

The following aggregators plugins are included as part of Flowbber.

Filter

Filter the collected data structure using the provided include and exclude patterns.

For example, consider a collected data that looks as following:

OrderedDict([
    ('my_source', {
        'my_value1': 1000,
        'my_value2': 2000,
        'my_value3': 3000,
        'other_value': 'hello'
    }),
    ('coverage', {
        'files': {
            '__init__.py': {
                'line_rate': 1.0,
                'total_misses': 0,
                'total_statements': 4,
            },
            '__main__.py': {
                'line_rate': 0.5,
                'total_misses': 5,
                'total_statements': 10,
            },
        },
        'total': {
            'line_rate': 0.37275985663082434,
            'total_misses': 350,
            'total_statements': 558,
        },
    }),
])

And you want to remove all the my_value* keys from my_source and all files entries in coverage. If using the configuration:

[[aggregators]]
type = "filter"
id = "filter"

    [aggregators.config]
    include = ["*"]
    exclude = ["my_source.my_value*", "coverage.files"]

The data will be filtered to:

OrderedDict([
    ('my_source', {
        'other_value': 'hello'
    }),
    ('coverage', {
        'total': {
            'line_rate': 0.37275985663082434,
            'total_misses': 350,
            'total_statements': 558,
        },
    }),
])

Dependencies:

pip3 install flowbber[filter]

Usage:

[[aggregators]]
type = "filter"
id = "..."

    [aggregators.config]
    exclude = []
    include = ["*"]
{
    "aggregators": [
        {
            "type": "filter",
            "id": "...",
            "config": {
                "include": ["*"],
                "exclude": []
            }
        }
    ]
}

include

List of patterns of key paths to include.

Matching is performed using Python’s fnmatch.

  • Default: ['*']

  • Optional: True

  • Schema:

    {
        'type': 'list',
        'schema': {
            'type': 'string',
            'empty': False,
        },
    }
    
  • Secret: False

exclude

List of patterns of key paths to exclude.

Matching is performed using Python’s fnmatch.

  • Default: []

  • Optional: True

  • Schema:

    {
        'type': 'list',
        'schema': {
            'type': 'string',
            'empty': False,
        },
    }
    
  • Secret: False