search

PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
16.2k stars 1.58k forks source link

State Changing Issue in the UI for Prefect 3 #15751

Open majo-aqfer opened 4 days ago

majo-aqfer commented 4 days ago

Bug summary

When a task in a flow fails, we attempt to bypass it by changing its state from "failed" to "completed." However, upon re-triggering the flow, the task that we marked as "completed" executes again instead of being skipped. Conversely, when we intentionally change a completed task to "failed" and retry the flow, it also fails to work as expected. We anticipated that the failed task would be skipped, but it gets executed again in Prefect 3.

In contrast, this functionality worked perfectly in Prefect 2.

I suspect this issue may be related to recent changes in the retry mechanism. It appears that for each retry within the flow, a new task instance is created instead of executing the existing task, as the task ID is different each time. This change in the task creation logic seems to be influenced by the dynamic_key, which affects how tasks are identified and executed in Prefect 3.

Version info (prefect version output)

Version:             3.0.5
API version:         0.8.4
Python version:      3.12.5
Git commit:          1d1b92a7
Built:               Tue, Oct 8, 2024 5:55 PM
OS/Arch:             darwin/arm64
Profile:             default
Server type:         cloud
Pydantic version:    2.8.2
Integrations:
  prefect-kubernetes: 0.5.0
  prefect-aws:       0.5.0
  prefect-gcp:       0.6.0
  prefect-github:    0.3.0

Additional context

No response

desertaxle commented 4 days ago

Thanks for the bug report @majo-aqfer! It seems like the result for your task runs is not being persisted and, therefore, can't be used on a flow run retry.

Can you share a simple example script where you're seeing this? It'll really help find where the issue is.