getindata / kedro-kubeflow

Kedro Plugin to support running workflows on Kubeflow Pipelines
https://kedro-kubeflow.readthedocs.io
Apache License 2.0
50 stars 21 forks source link

Add support to kedro namespaces for parameters #219

Closed DmitriyLamzin closed 1 year ago

DmitriyLamzin commented 1 year ago

namespaced kedro parameters in general look like:

outer_namespace.inner_namespace.param_name

kubeflow doesn't accept dots in parameter names so during compiling the Error is raised:

ValueError: 'outer_namespace.inner_namespace.param_name' is not a valid parameter name

One of the fixes I see is to merge such parameters in structure so it to became looks like this:

{
  "outer_namespace": {
      "inner_namespace": {
        "param_name": "value",
        ...
      },
      "another_inner_namespace": {
        "param_name": "value"
        ...
      } 
  }
}
marrrcin commented 1 year ago

We will be happy to accept this fix as a PR if you're willing to contribute 🙂