GoogleCloudPlatform / llm-pipeline-examples

Apache License 2.0
107 stars 26 forks source link

NotImplementedError due to NoneType when running pipeline.py #22

Closed anirudh2 closed 1 year ago

anirudh2 commented 1 year ago

Hi there!

I'm running the code with the following pyproject.toml

[tool.poetry]
name = "llm-pipeline-examples"
version = "0.1.0"
description = ""
authors = []
license = "Apache 2.0"
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9"
absl-py = "^1.4.0"
google = "^3.0.0"
kfp = "^1.8.19"
google-cloud-aiplatform = "^1.23.0"

[tool.poetry.group.dev.dependencies]
pyright = "^1.1.301"
pylint = "^2.17.1"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Running pipeline.py results in errors.

$ poetry run python pipeline.py --project=$PROJECT_ID --pipeline_root=gs://$BUCKET_NAME/pipeline_runs/ --config=configs/small1vm1gpu.json
/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/v2/compiler/compiler.py:1290: FutureWarning: APIs imported from the v1 namespace (e.g. kfp.dsl, kfp.components, etc) will not be supported by the v2 compiler since v2.0.0
  warnings.warn(
Traceback (most recent call last):
  File "/home/anirudh/projects/llm-pipeline-examples/pipeline.py", line 402, in <module>
    app.run(main)
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/anirudh/projects/llm-pipeline-examples/pipeline.py", line 345, in main
    compiler.Compiler().compile(
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/v2/compiler/compiler.py", line 1301, in compile
    pipeline_job = self._create_pipeline_v2(
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/v2/compiler/compiler.py", line 1223, in _create_pipeline_v2
    pipeline_func(*args_list)
  File "/home/anirudh/projects/llm-pipeline-examples/pipeline.py", line 303, in my_pipeline
    deploy_op = deploy(
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/components/_dynamic.py", line 53, in Deploy
    return dict_func(locals())  # noqa: F821 TODO
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/components/_components.py", line 389, in create_task_object_from_component_and_pythonic_arguments
    return _create_task_object_from_component_and_arguments(
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/components/_components.py", line 326, in _create_task_object_from_component_and_arguments
    task = _container_task_constructor(
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/dsl/_component_bridge.py", line 319, in _create_container_op_from_component_and_arguments
    _attach_v2_specs(task, component_spec, original_arguments)
  File "/home/anirudh/.cache/pypoetry/virtualenvs/llm-pipeline-examples-uTYd1Pp6-py3.9/lib/python3.9/site-packages/kfp/dsl/_component_bridge.py", line 600, in _attach_v2_specs
    raise NotImplementedError(
NotImplementedError: Input argument supports only the following types: PipelineParam, str, int, float, bool, dict, and list. Got: "None".

I've tried this with various python 3.8, 3.9, and 3.10 without success. It always fails on the same error. Do you know if my dependencies might be incorrect or could something else be causing this issue?

Chris113113 commented 1 year ago

Bug fix in this PR: #24

anirudh2 commented 1 year ago

Perfect. Thanks!