Generator Adapter (Exec function is a python generator which can yield
multiple outputs). The inputs are same as a Regular Adapter
Async Generator Adapter which receives a stream (async generator) for
each wired input. The outputs are same as a Generator Adapter, i.e.
can yield multiple outputs.
Pipeline implements streaming using asynchronous programming and
considers each adapter to be Async Generator. To support Regular and
Generator adapters, it uses a DefaultWrapper Async Generator Adapter.
Modified config files schema to allow root-level pipeline inputs
dictionary, which can be accessed anywhere in the config file using
$$.<inputs dict key>
Config files are now also allowed to have "null" values for adapter
inputs which would result in skipping that particular input.
Added 2 example config files average_daily_all_variables.yml and
average_monthly_precipitation_rate.yml to demonstrate all the new
features. Also introduced DcatRangeStream, DcatVariableStream and
CroppingTransWrapper for GLDAS pipelines.
Adapters can now be of 3 types:
Pipeline implements streaming using asynchronous programming and considers each adapter to be Async Generator. To support Regular and Generator adapters, it uses a DefaultWrapper Async Generator Adapter.
Modified config files schema to allow root-level pipeline inputs dictionary, which can be accessed anywhere in the config file using
$$.<inputs dict key>
Config files are now also allowed to have "null" values for adapter inputs which would result in skipping that particular input.Added 2 example config files average_daily_all_variables.yml and average_monthly_precipitation_rate.yml to demonstrate all the new features. Also introduced DcatRangeStream, DcatVariableStream and CroppingTransWrapper for GLDAS pipelines.