Closed fg91 closed 1 week ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 61.11%. Comparing base (
d04cf66
) to head (0ffe186
). Report is 4 commits behind head on master.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
We noticed that this PR still needs some more work as it still fails with LaunchPlan
outputs used for other nodes. Here is a minimal example:
from flytekit import LaunchPlan, task, workflow
@task
def my_task(inp: int) -> int:
return 42 + inp
@workflow
def inner_workflow(inp: int) -> int:
return my_task(inp=inp)
@workflow
def outer_workflow() -> int:
res = LaunchPlan.get_or_create(inner_workflow, "name_override", default_inputs={"inp": 3})()
return my_task(inp=res)
It fails with:
Compiling tasks...
Compiling workflow: wfs.wf.inner_workflow
Compiling workflow: wfs.wf.outer_workflow
:( Error Compiling workflow: wfs.wf.outer_workflow
Error: Collected Errors: 3
Error 0: Code: ParameterNotBound, Node Id: n1, Description: Parameter not bound [inp].
Error 1: Code: VariableNameNotFound, Node Id: n0, Description: Variable [inp] not found on node [n0].
Error 2: Code: VariableNameNotFound, Node Id: n0, Description: Variable [o0] not found on node [n0].
We tried to understand what is missing/different in flytectl compile
to how the admin handles workflow registration but have not found the root cause yet. Generally wondering if we need to compile the workflows in a certain order and somehow also compile/populate the LaunchPlans as they currently don't have the input and output infos.
@dav009 do you maybe have some time to look at this with us or give us some hints where to start?
Closing in favour of https://github.com/flyteorg/flyte/pull/5463.
Why are the changes needed?
This workflow can be registered with
pyflyte register
and executed in a cluster. However,pyflyte --pkgs <module> package
+flytectl compile
gives the following error:What changes were proposed in this pull request?
The error occurs since
flytectl compile
does not consider all launchplans but only the default launchplan for the currently compiled workflow.How was this patch tested?
Added a unit test based on the example above.
Note to reviewers: We couldn't find whether the source files for the
testdata/*.tgz
should be checked in as well - should they?Check all the applicable boxes