getindata / kedro-kubeflow

Kedro Plugin to support running workflows on Kubeflow Pipelines
https://kedro-kubeflow.readthedocs.io
Apache License 2.0
49 stars 21 forks source link

Unable to register project-local `register_config_loader` hook #72

Closed szczeles closed 2 years ago

szczeles commented 3 years ago

The plugin overrides hook called register_config_loader to allow dynamic injection of branch name and commit id. Unfrotunately, this hook is marked as firstresult=True in Kedro spec (see https://github.com/quantumblacklabs/kedro/blob/master/kedro/framework/hooks/specs.py#L332), so plugins system registers plugin's config loader as the only one if the plugin is installed (see: https://pluggy.readthedocs.io/en/stable/#first-result-only). This makes it impossible to use project-local hook -> it is simply never called.

One of the ways to fix the issue is to disable hooks for the plugin using DISABLE_HOOKS_FOR_PLUGINS in settings.py of the project, but this is not elegant solution. Instead, the plugin's related config loader should be used only within kedro kubeflow ... commands context.

Same issue affects kedro-airflow-k8s

szczeles commented 2 years ago

Fixed in 0.6.0 release