kubeflow / kfp-tekton

Kubeflow Pipelines on Tekton
https://developer.ibm.com/blogs/kubeflow-pipelines-with-tekton-and-watson/
Apache License 2.0
175 stars 122 forks source link

Pipeline compiled by python SDK 0.3.0 installed by pip failed when running #339

Closed jizg closed 3 years ago

jizg commented 4 years ago

/kind bug

What steps did you take and what happened:

I created a pipeline to test passing file artifacts between tasks. When the pipeline is compiled by SDK installed from pip, the execution failed. But if I install SDK compiled from the lastest master branch, it worked without problem.

What did you expect to happen:

pip installed SDK should work without problem.

Additional information:

pipeline_yaml.zip

pipeline yaml diff, upper is the one compiled by pip SDK.

<       "$(results.output-text.path)"]]}'
---
>       "$(workspaces.repeat-line.path)/repeat-line-output_text"]]}'
84,85c84,90
<     persistentVolumeClaim:
<       claimName: test-pipeline
---
>     volumeClaimTemplate:
>       spec:
>         accessModes:
>         - ReadWriteMany
>         resources:
>           requests:
>             storage: 2Gi

code to generate pipeline:

def test_pipeline():
    @func_to_container_op
    def repeat_line(line: str, output_text_path: OutputPath(str), count: int = 10):
        '''Repeat the line specified number of times'''
        with open(output_text_path, 'w') as writer:
            for i in range(count):
                writer.write(line + '\n')

    # Reading big data
    @func_to_container_op
    def print_text(text_path: InputPath()): # The "text" input is untyped so that any data can be printed
        '''Print text'''
        with open(text_path, 'r') as reader:
            for line in reader:
                print(line, end = '')

    repeat_lines_task = repeat_line(line='Hello', count=500)
    print_text(repeat_lines_task.output) # Don't forget .output !

TektonCompiler().compile(test_pipeline, 'print_repeating_lines_pipeline.yaml')

Environment:

issue-label-bot[bot] commented 4 years ago

Issue-Label Bot is automatically applying the labels:

Label Probability
area/sdk 0.91

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.

issue-label-bot[bot] commented 4 years ago

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

Tomcli commented 4 years ago

yes there's a bug with the big data passing, I fixed few weeks ago and it will be in our 0.4 release.

Tomcli commented 3 years ago

/close

k8s-ci-robot commented 3 years ago

@Tomcli: Closing this issue.

In response to [this](https://github.com/kubeflow/kfp-tekton/issues/339#issuecomment-755747749): >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.