search

metno / pyaro

Interface for reading different airquality measurement databases for intercomparison with transport-model-data.
https://pyaro.readthedocs.io
GNU Lesser General Public License v2.1
0 stars 2 forks source link

Input type for filters #62

Open magnusuMET opened 2 weeks ago

magnusuMET commented 2 weeks ago

To implement a reader it would be great if the type of filters were known. The relevant code is https://github.com/metno/pyaro/blob/85c0d4ec4f78936fe69be3ff42f2e8b1ce385815/src/pyaro/timeseries/Reader.py#L11 As far as I understand it should be filters: Iterable[Filter] | dict[str, Any] | None = None.

In various config files I see filters are given as a dictionary. How can I go from a dictionary to a list of Filters? In the comment for the above filter it is suggested

:param filters: list of filters, or dict of (name, kwargs) for FilterFactory

but FilterFactory does not seem like the right type. FilterCollection might be right, but only stores DataIndexFilter.

heikoklein commented 2 weeks ago

Implementing filters is easiest by using the AutoFilterReaderEngine as described in https://pyaro.readthedocs.io/latest/how-to-add-new-reader.html

https://github.com/metno/pyaro-readers/blob/002af7966793635fcf3ae62509ae64132a2e9e02/src/pyaro_readers/netcdf_rw/Netcdf_RWTimeseries.py#L140 shows how the _get_filters() is then used in a filter to apply a TimeBoundsFilter on the data split on files.