kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.47k stars 875 forks source link

kedro new --tools test creates invalid test setup #3966

Open noklam opened 6 days ago

noklam commented 6 days ago

Description

Related: https://github.com/kedro-org/kedro/issues/3954 Slack: https://linen-slack.kedro.org/t/22301026/question-is-there-an-extra-step-required-to-run-pytest-with-#661646b5-a2e7-4b77-a3db-ab585f408793

https://github.com/kedro-org/kedro-starters/blob/main/spaceflights-pandas-viz/%7B%7B%20cookiecutter.repo_name%20%7D%7D/tests/test_run.py#L27

@pytest.fixture
def config_loader():
    return OmegaConfigLoader(conf_source=str(Path.cwd()))

The config loader is using incorrect arguments.

Context

Steps to Reproduce

  1. kedro new --tools test --example yes -n kedro-project
  2. cd kedro-project
  3. pip install -e .
  4. pytest

Try to print out parameters and find that it loads incorrrect config (use with globals. for example)

Expected Result

The default test should have similar behavior as kedro new by default

Potential Solutions:

This solution works but the complication is we introduce these "default" argument which is invisible to Kedro Framework user. Should we do this or instead use KedroSession? IMO KedroSession will be more consistent and it respect all the custom settings built in settings.py.

We may have to update some documentation in the test as well.

Your Environment