kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.47k stars 875 forks source link

Default to `raise_errors=True` in `find_pipelines` #3886

Open deepyaman opened 1 month ago

deepyaman commented 1 month ago

Thanks, a bit late to the discussion. I agree we should not change the behavior in a micro release.

In principle I like a default=True, but with concern about discoverability and the ergonomics of this. I imagine this is like a switch that you may need to turn on & off from time to time, going into pipeline_registry.py to do so feel inconsistent with other CLI options etc.

Originally posted by @noklam in https://github.com/kedro-org/kedro/pull/3823#pullrequestreview-2069612447

TODO:

bpmeek commented 1 month ago

Something I would like to see is changing the default entry in pipeline_registry.py to find_pipelines(raise_errors=True) (or False whichever is decided upon), I didn't know until yesterday that this was even a functionality of find_pipelines().

Which may even open up a broader question of "Should Kedro populate default options in the boilerplate to let users know its there?"