Implemented autodiscovery of project pipelines. A pipeline created with kedro pipeline create <pipeline_name> can now be accessed immediately without needing to explicitly register it in src/<package_name>/pipeline_registry.py, either individually by name (e.g. kedro run --pipeline=<pipeline_name>) or as part of the combined default pipeline (e.g. kedro run). By default, the simplified register_pipelines() function in pipeline_registry.py looks like:
def register_pipelines() -> Dict[str, Pipeline]:
"""Register the project's pipelines.
Returns:
A mapping from pipeline names to ``Pipeline`` objects.
"""
pipelines = find_pipelines()
pipelines["__default__"] = sum(pipelines.values())
return pipelines
The Kedro IPython extension should now be loaded with %load_ext kedro.ipython.
The line magic %reload_kedro now accepts keywords arguments, e.g. %reload_kedro --env=prod.
Improved resume pipeline suggestion for SequentialRunner, it will backtrack the closest persisted inputs to resume.
Bug fixes and other changes
Changed default False value for rich logging show_locals, to make sure credentials and other sensitive data isn't shown in logs.
Rich traceback handling is disabled on Databricks so that exceptions now halt execution as expected. This is a workaround for a bug in rich.
When using kedro run -n [some_node], if some_node is missing a namespace the resulting error message will suggest the correct node name.
Updated documentation for rich logging.
Updated Prefect deployment documentation to allow for reruns with saved versioned datasets.
The Kedro IPython extension now surfaces errors when it cannot load a Kedro project.
Relaxed delta-spark upper bound to allow compatibility with Spark 3.1.x and 3.2.x.
Added gdrive to list of cloud protocols, enabling Google Drive paths for datasets.
Added svg logo resource for ipython kernel.
Upcoming deprecations for Kedro 0.19.0
The Kedro IPython extension will no longer be available as %load_ext kedro.extras.extensions.ipython; use %load_ext kedro.ipython instead.
kedro jupyter convert, kedro build-docs, kedro build-reqs and kedro activate-nbstripout will be deprecated.
Implemented autodiscovery of project pipelines. A pipeline created with kedro pipeline create <pipeline_name> can now be accessed immediately without needing to explicitly register it in src/<package_name>/pipeline_registry.py, either individually by name (e.g. kedro run --pipeline=<pipeline_name>) or as part of the combined default pipeline (e.g. kedro run). By default, the simplified register_pipelines() function in pipeline_registry.py looks like:
def register_pipelines() -> Dict[str, Pipeline]:
"""Register the project's pipelines.
Returns:
A mapping from pipeline names to ``Pipeline`` objects.
"""
pipelines = find_pipelines()
pipelines["__default__"] = sum(pipelines.values())
return pipelines
The Kedro IPython extension should now be loaded with %load_ext kedro.ipython.
The line magic %reload_kedro now accepts keywords arguments, e.g. %reload_kedro --env=prod.
Improved resume pipeline suggestion for SequentialRunner, it will backtrack the closest persisted inputs to resume.
Bug fixes and other changes
Changed default False value for rich logging show_locals, to make sure credentials and other sensitive data isn't shown in logs.
Rich traceback handling is disabled on Databricks so that exceptions now halt execution as expected. This is a workaround for a bug in rich.
When using kedro run -n [some_node], if some_node is missing a namespace the resulting error message will suggest the correct node name.
Updated documentation for rich logging.
Updated Prefect deployment documentation to allow for reruns with saved versioned datasets.
The Kedro IPython extension now surfaces errors when it cannot load a Kedro project.
Relaxed delta-spark upper bound to allow compatibility with Spark 3.1.x and 3.2.x.
Added gdrive to list of cloud protocols, enabling Google Drive paths for datasets.
Added svg logo resource for ipython kernel.
Upcoming deprecations for Kedro 0.19.0
The Kedro IPython extension will no longer be available as %load_ext kedro.extras.extensions.ipython; use %load_ext kedro.ipython instead.
kedro jupyter convert, kedro build-docs, kedro build-reqs and kedro activate-nbstripout will be deprecated.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps kedro from 0.18.2 to 0.18.3.
Release notes
Sourced from kedro's releases.
Changelog
Sourced from kedro's changelog.
Commits
4db0ae3
Release/0.18.3 (#1856)a3e1531
Add new line before arguments section of docstring (#1855)123d5bd
Update description forpipeline
arg torun
CLI (#1759)86ada23
Supportfind_pipelines()
for simplified projects (#1821)ab83926
Add svg logo resource for ipython (#1851)98a1f2e
Document pipeline registry and automatic discovery (#1788)1e10537
Add gdrive to CLOUD_PROTOCOLS list (#1708)f68e9e2
Add documentation on how to visualise Matplotlib plots on Kedro-Viz (#1820)5f018ac
Alias Kedro IPython extension to kedro.ipython (#1837)e65655d
Fix class docstring for tracking datasets (#1838)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)