search

getindata / kedro-airflow-k8s

Kedro Plugin to support running pipelines on Kubernetes using Airflow.
https://kedro-airflow-k8s.readthedocs.io
Apache License 2.0
29 stars 11 forks source link

Compile step broken with kedro==0.18.0 #106

Open mwiewior opened 3 years ago

mwiewior commented 3 years ago

Steps to reproduce:

  1. install kedro==0.18.0
  2. Run kedro airflow-k8s compile: 
    
bash-3.2$ kedro airflow-k8s -e pipelines compile
Traceback (most recent call last):
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/bin/kedro", line 8, in <module>
sys.exit(main())
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/kedro/framework/cli/cli.py", line 268, in main
cli_collection = KedroCLI(project_path=Path.cwd())
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/kedro/framework/cli/cli.py", line 181, in __init__
self._metadata = bootstrap_project(project_path)
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/kedro/framework/startup.py", line 181, in bootstrap_project
configure_project(metadata.package_name)
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/kedro/framework/project/__init__.py", line 219, in configure_project
settings.configure(settings_module)
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/dynaconf/base.py", line 182, in configure
self._wrapped = Settings(settings_module=settings_module, **kwargs)
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/dynaconf/base.py", line 235, in __init__
self.validators.validate(
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/dynaconf/validator.py", line 417, in validate
validator.validate(self.settings, only=only, exclude=exclude)
File "/Users/mwiewior/job/projects/PKOBP/git/kedro-airflow-k8s/venv/kedro/lib/python3.8/site-packages/dynaconf/validator.py", line 197, in validate
self._validate_items(
TypeError: _validate_items() got an unexpected keyword argument 'only'
3. Downgrading to kedro 0.17.x resolves the issue

kedro --version kedro, version 0.17.5

szczeles commented 3 years ago

@mwiewior I think it's issue of Kedro itself, see: https://github.com/quantumblacklabs/kedro/commit/d36fce90f23a79990ea6470ad2ef9b3f1a4779bf

I observe same on Kedro 0.17.4, upgrading to 0.17.5 helps

michalzelechowski-getindata commented 3 years ago

@mwiewior Did you mean kedro==0.17.0? Title says 0.18.0, but at this very time such version does not exist.

datajoely commented 3 years ago

Hi - checking in from the Kedro team here - we're hard at work on the 0.18.0 expected early 2022. If you have a copy handy we could release that instead :)