kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.53k stars 1.59k forks source link

[sdk] ParallelFor loop argument types not recognized #9777

Closed stijntratsaertit closed 4 months ago

stijntratsaertit commented 1 year ago

Environment

Steps to reproduce

Previously, in KFP SDK v1.8.*, the following worked when the my_component component accepted 2 arguments, the first being int and the second being a str.

@dsl.component
def my_component(first_param: int, second_param: str):
    # do some things
    return

with dsl.ParallelFor(
    [
        {"first_param": 1, "second_param": "hello"},
        {"first_param": 0, "second_param": "world"},
        {"first_param": 1, "second_param": "hello"},
    ],
    parallelism=3,
) as params:
    my_component(first_param=params.first_param, second_param=params.second_param)

When upgrading to KFP SDK v2.0.1, this is now thrown as an error when compiling the pipeline:

ERROR:__main__:Incompatible argument passed to the input 'first_param' of component 'my_component': Argument type 'STRING' is incompatible with the input type 'NUMBER_INTEGER'

Expected result

No error and a correct passage of first_param as an integer parameter to my_component.

Materials and Reference

It could be that I'm misusing the arguments to dsl.ParallelFor as I couldn't find very much more similar examples other than the main docs.

A possible angle to the issue could be regarding this: https://github.com/kubeflow/pipelines/blob/6dfcd533f495bf19019bb09baf5ec0d3775e0f1c/sdk/python/kfp/components/for_loop.py#L46

The type that comes in here is dict, no regex match is found and None is returned. Because of that, an automatic fallback to String is done here: https://github.com/kubeflow/pipelines/blob/6dfcd533f495bf19019bb09baf5ec0d3775e0f1c/sdk/python/kfp/components/for_loop.py#L243-L244


Impacted by this bug? Give it a 👍.

alvarodemig commented 11 months ago

Same error that was commented here: https://github.com/kubeflow/pipelines/issues/9366

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

alvarodemig commented 6 months ago

This is still a bug

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 4 months ago

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.