getindata / kedro-kubeflow

Kedro Plugin to support running workflows on Kubeflow Pipelines
https://kedro-kubeflow.readthedocs.io
Apache License 2.0
46 stars 21 forks source link

Invalid resources for kedro nodes with `_` in the name #231

Open szczeles opened 1 year ago

szczeles commented 1 year ago

Pipeline generator sanitizes the node name to be kubeflow-compatible, but when it comes to details selection (like resources, tolerations), it doesn't follow the same logic, it purely checks for op name: https://github.com/getindata/kedro-kubeflow/blob/0.7.4/kedro_kubeflow/generators/utils.py#L177

Therefore, with the Kedro node named some_node, and specific resources for some_node, it uses defaults instead. We should check for uncleaned name in the config file to cover cases like this.