esm-tools / pymorize

A Python based Tool to CMORize NetCDF Data

MIT License

0 stars 1 forks source link

Prefect, Dask, and Cerberus #34

Closed pgierz closed 3 weeks ago

pgierz commented 3 weeks ago

Summary

This PR adds support for Prefect and Dask. Additionally, we include a new API for inputs in Rule and deprecate the gather_inputs function.

There are no tests yet. These will be implemented on a need by need basis once something breaks and we have a reproducible error.

Superceded PRs

28
33

Revelant Commits:

feat: validation for pipeline dictionary
fixes tests
feat: hooks up pipeline validator
feat: rules validator
fixes last tests for cerberus
fix: allowed input_source and input_type is handled by cerberus
proper docs for cerberus
fix: small mistake in gather inputs
fix(gather_inputs.py): deprecates the gather_inputs function
feat: new API for inputs in Rule
doc: fix too long docstring
feat: forgot to add step in Pipeline
feat: preliminary prefect support
test: preliminary prefect support
style: minor fixup to keep line length checker happy
prefect and dask