kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.53k stars 879 forks source link

Resolve all path in Kedro #3742

Closed noklam closed 3 months ago

noklam commented 4 months ago

Description

Related:

def _add_src_to_path(source_dir: Path, project_path: Path) -> None: ...

def bootstrap_project(project_path: Path) -> ProjectMetadata: """Run setup required at the beginning of the workflow when running in project mode, and return project metadata. """ project_path = Path(project_path).resolve() # New metadata = _get_project_metadata(project_path) _add_src_to_path(metadata.source_dir, project_path) configure_project(metadata.package_name) return metadata



If you look at the `bootstrap_project`, it expect a path object but in fact sometimes string is passed and it is okay about this. Same go as `_add_src_to_path` that accepts `Path` only but actually taking `str` sometimes.

The fix here is simple, make sure `project_path` is resolved. It is already done in many place, just not consistently (sometimes with `expanduser()` too):
https://github.com/kedro-org/kedro/blob/35ee3eadb1a30557d2c4bf28a96c7369a2e4c4a0/kedro/framework/startup.py#L52-L53

and

https://github.com/kedro-org/kedro/blob/35ee3eadb1a30557d2c4bf28a96c7369a2e4c4a0/kedro/framework/startup.py#L93-L95

- [ ] Todo: add a test
## Developer Certificate of Origin
We need all contributions to comply with the [Developer Certificate of Origin (DCO)](https://developercertificate.org/). All commits must be signed off by including a `Signed-off-by` line in the commit message. [See our wiki for guidance](https://github.com/kedro-org/kedro/wiki/Guidelines-for-contributing-developers/).

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

## Checklist

- [ ] Read the [contributing](https://github.com/kedro-org/kedro/blob/main/CONTRIBUTING.md) guidelines
- [ ] Signed off each commit with a [Developer Certificate of Origin (DCO)](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-the-commit-signoff-policy-for-your-repository)
- [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the [`RELEASE.md`](https://github.com/kedro-org/kedro/blob/main/RELEASE.md) file
- [ ] Added tests to cover my changes
- [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team
noklam commented 4 months ago

3831e82 this seems to work already