flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.78k stars 659 forks source link

[BUG][Flytekit] `FlyteWorkflowExecution` `inputs` and `outputs` property return `None` #2108

Open mdjong1 opened 2 years ago

mdjong1 commented 2 years ago

Describe the bug

When using FlyteRemote with the fetch_workflow_execution function we can correctly retrieve most of the information related to a workflow execution (i.e. version, closure information). However, when we try to access the inputs and output from here the result is a None, despite knowing/seeing that the execution has inputs & outputs in the Flyte Console.

Expected behavior

I would expect that when I call .inputs or .outputs on a FlyteWorkflowExecution it would Returns the inputs to the execution in the standard python format as dictated by the type engine. (taken from docstring) as opposed to returning a None when inputs and outputs are actually set for the specific workflow execution. For cases when no inputs or outputs are set perhaps an empty dict would make more sense here as well.

Additional context to reproduce

remote = FlyteRemote(flyte_admin_url=_FLYTE_HOST, insecure=False)

execution = remote.fetch_workflow_execution(project=_PROJECT_NAME, domain=_PROJECT_DOMAIN, name=_EXECUTION_NAME)

print(execution.inputs)  # prints None

Screenshots

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

kumare3 commented 2 years ago

cc @wild-endeavor / @pingsutw is this a regression?

wild-endeavor commented 2 years ago

No this isn't a regression this is a feature... and not a good one. Let's think of a way to improve the UX, we'll hold off on this for this release (0.30) and address in 0.31.

wild-endeavor commented 2 years ago

Still working on how to better handle, but merged that PR in the interim. My issue is that this doesn't fully resolve the problem. If the workflow execution hasn't completed, outputs will still be missing, and there's no way to alert the user that they'll need to re-sync the object.

I'll make a v0.30.2 release early next week probably with this.

ggydush-fn commented 2 years ago

@wild-endeavor I believe I'm seeing this issue when fetching workflow executions that have been aborted.

Calling flyte_remote.fetch_workflow_execution(name=execution_name) works fine, but if the execution_name corresponds to an execution that was aborted, you get the following. I can create a separate issue if you'd prefer!

File ~/.virtualenvs/freenome-rpctl-PPAhckSl-py3.9/lib/python3.9/site-packages/flytekit/core/type_engine.py:695, in TypeEngine.literal_map_to_kwargs(cls, ctx, lm, python_types)
    691 """
    692 Given a ``LiteralMap`` (usually an input into a task - intermediate), convert to kwargs for the task
    693 """
    694 if len(lm.literals) != len(python_types):
--> 695     raise ValueError(
    696         f"Received more input values {len(lm.literals)}" f" than allowed by the input spec {len(python_types)}"
    697     )
    698 return {k: TypeEngine.to_python_value(ctx, lm.literals[k], v) for k, v in python_types.items()}

ValueError: Received more input values 0 than allowed by the input spec 3
github-actions[bot] commented 1 year ago

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

github-actions[bot] commented 1 year ago

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏

github-actions[bot] commented 3 months ago

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏