search

machine-learning-exchange / mlx

Machine Learning eXchange (MLX). Data and AI Assets Catalog and Execution Engine
https://ml-exchange.org/
Apache License 2.0
205 stars 54 forks source link

Current limitations for auto-generated pipelines for component #286

Open Tomcli opened 2 years ago

Tomcli commented 2 years ago

Is your feature request related to a problem? Please describe. Here is the list of limitations for the current code generation:

  1. Description is required in MLX, but optional in KFP
  2. MLX only can pass parameters as string type. If KFP component has non-string type, the pipeline will have type errors in the arguments.
  3. Component parameter names need to be sanitized as Python variable. (https://github.com/kubeflow/pipelines/blob/master/components/git/clone/component.yaml)
  4. MLX converted double quote " into single quote ' for all the parameter values. This doesn't work for some components when users need to pass in JSON string with double quote around their keys and values. (e.g. https://github.com/kubeflow/pipelines/blob/master/components/contrib/json/Combine_lists/component.yaml)

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Tomcli commented 2 years ago

I think 3 and 4 should be easier to fix. 1 and 2 is nice to have but could be difficult to cover all the edge cases.

Tomcli commented 2 years ago

/cc @ckadner