PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.32k stars 1.5k forks source link

`prefect deploy` is executing flow code #14402

Closed zangell44 closed 2 days ago

zangell44 commented 3 days ago

First check

Bug summary

Running prefect deploy appears to execute flow code. I can reproduce on both 3.x and 2.x.

To reproduce:

  1. Copy and paste the linked code into a .py file.
  2. Run prefect deploy and select the flow flow_retries_subflows_entry to deploy
  3. Note the Value error encountered

The same result happens when the flow is saved as a known deployment in prefect.yaml and we try to re-deploy.

Reproduction

from prefect import flow

child_flow_run_count = 0
flow_run_count = 0

@flow
def child_flow(tags=["integration-tests", "expected:failure"]):
    global child_flow_run_count
    child_flow_run_count += 1

    # Fail on the first flow run but not the retry
    if flow_run_count < 2:
        raise ValueError()

    return "hello"

@flow(retries=10)
def flow_retries_subflows_entry():
    global flow_run_count
    flow_run_count += 1

    result = child_flow()

    # It is important that the flow run fails after the child flow run is created
    if flow_run_count < 3:
        raise ValueError()

    return result

Error

11:11:54.273 | INFO    | prefect.engine - View at https://api.stg.prefect.dev/account/9a67b081-4f14-4035-b000-1f715f46231b/workspace/60305128-9439-47ef-b353-f2a8f6a7a8b4/runs/flow-run/f7037938-07b9-449e-afab-3b0fc6a9afef
11:11:54.650 | ERROR   | Flow run 'witty-dachshund' - Encountered exception during execution: 
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py310orion/lib/python3.10/site-packages/prefect/flow_engine.py", line 571, in run_context
    yield self
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py310orion/lib/python3.10/site-packages/prefect/flow_engine.py", line 612, in run_flow_sync
    engine.call_flow_fn()
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py310orion/lib/python3.10/site-packages/prefect/flow_engine.py", line 591, in call_flow_fn
    result = call_with_parameters(self.flow.fn, self.parameters)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py310orion/lib/python3.10/site-packages/prefect/utilities/callables.py", line 208, in call_with_parameters
    return fn(*args, **kwargs)
  File "<ast>", line 14, in child_flow
ValueError
11:11:54.748 | ERROR   | Flow run 'witty-dachshund' - Finished in state Failed('Flow run encountered an exception: ValueError: ')

Versions (prefect version output)

Version:             3.0.0rc9
API version:         0.8.4
Python version:      3.10.4
Git commit:          b75631a7
Built:               Mon, Jun 24, 2024 5:50 PM
OS/Arch:             darwin/arm64
Profile:             stg
Server type:         cloud
Pydantic version:    2.7.1

Additional context

No response

desertaxle commented 2 days ago

Closed by #14403 and #14405