kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.61k stars 1.63k forks source link

[sdk] parameterize PipelineTask's methods: set_cpu_requests, set_env_variables, set_memory_limit, etc #11140

Open haiminh2001 opened 2 months ago

haiminh2001 commented 2 months ago

Environment

Steps to reproduce

@pipeline(name='test_cpu_request'):
  def pipeline(cpu_request: str):

    task.set_cpu_request(cpu_request)

Error:

Traceback (most recent call last):
  File "pipeline.py", line 13, in <module>
    def pipeline(train_image_uri: str,
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_context.py", line 65, in pipeline
    return component_factory.create_graph_component_from_func(
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/component_factory.py", line 673, in create_graph_component_from_func
    return graph_component.GraphComponent(
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/graph_component.py", line 58, in __init__
    pipeline_outputs = pipeline_func(*args_list)
  File "pipeline.py", line 72, in pipeline
    .set_cpu_request(cpu_request)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 56, in wrapper
    return method(self, *args, **kwargs)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 362, in set_cpu_request
    cpu = self._validate_cpu_request_limit(cpu)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 340, in _validate_cpu_request_limit
    if re.match(r'([0-9]*[.])?[0-9]+m?$', cpu) is None:
  File "/miniconda3/envs/dev-3-10/lib/python3.10/re.py", line 190, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or bytes-like object

Expected result

The method set_cpu_requests is expecting a hard-coded string, but I think the cpu requests should be an adjustable param that may vary among runs.


Impacted by this bug? Give it a 👍.

github-actions[bot] commented 4 days ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.