lisad / phaser

The missing layer for complex data batch integration pipelines
MIT License
4 stars 1 forks source link

Adds support for sources in CLI #107

Closed jeffkole closed 5 months ago

jeffkole commented 5 months ago

There is still an improvement to make to the logic in pipeline that checks which sources need initialization -- namely, the pipeline should check that sources that come from prior phases actually are declared in phases that will be executed before when the source is needed. But that is an optimization that can be left til later.

The other minor annoyance that I skipped figuring out is how to get the CLI help to display the needed sources. I think it may require some subclassing of argparse.ArgumentParser, which is not something I wanted to get into now.

Addresses #88