[BUG] Overriding task `name` using `with_overrides(name)` does not display correctly in node view

sherwynds-stripe commented 4 months ago

Describe the bug

Hi Flyte team, we're using with_overrides(name) to change the name of a task and notice:

The graph view correctly shows the new task name
But the node view doesn’t We're on the latest flytekit (1.11.0).

Expected behavior

The node view should show the new task name make_test_dc_renamed, as the graph view correctly does.

Additional context to reproduce

Here is a reproducible code example. The goal is to use with_overrides to rename make_test_dc to make_test_dc_renamed:

@task
def make_test_dc(name: str, path: str, subfield: str, subpath: str) -> OuterDC:
    return OuterDC(name=name, path=path, inner_dc=InnerDC(subfield=subfield, subpath=subpath))

@task
def extract_subpath(dc: OuterDC, simple_list: List[str]) -> str:
    import flytekit

    print(f"flytekit version: {flytekit.__version__}")
    return dc.inner_dc.subpath

@workflow
def wf(
    name: str = "name",
    path: str = "path",
    subfield: str = "subfield",
    subpath: str = "subpath",
) -> str:
    dc = make_test_dc(name=name, path=path, subfield=subfield, subpath=subpath).with_overrides(
        name="make_test_dc_renamed"
    )
    return extract_subpath(dc=dc, simple_list=["a", "b", "c"])

Screenshots

No response

Are you sure this issue hasn't been raised already?

[X] Yes

Have you read the Code of Conduct?

[X] Yes

welcome[bot] commented 4 months ago

Thank you for opening your first issue here! 🛠

dosubot[bot] commented 4 months ago

It looks like the issue you're encountering with with_overrides(name) not reflecting the updated task name in the node view, despite showing correctly in the graph view, might be tied to how Flyte's console and underlying systems handle task names and overrides. The graph view and node view in the Flyte console are likely using different mechanisms or points of reference for displaying task names. While the graph view accurately reflects the override, the node view might not be directly tied to the same naming conventions or update mechanisms.

The with_overrides function is designed to provide node-level overrides at compile time, which includes changes to task names. However, its application and the reflection of these changes in the UI might have limitations, especially in dynamic contexts or when dealing with the console's UI rendering logic. The discrepancy you're seeing could be a result of these limitations or a specific implementation detail of the Flyte console that doesn't account for dynamic name changes in the node view as it does in the graph view.

For a resolution, it might be necessary to delve into the Flyte console's source code or reach out to the Flyte community for insights on whether this behavior is expected or if it's an oversight that needs addressing. This could also be an opportunity to discuss potential enhancements to the Flyte console to ensure consistency between the graph view and node view when displaying overridden task names.

#### Details

**Sources** - [flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go](https://github.com/flyteorg/flyte/blob/master/flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go) - [flyteadmin/pkg/manager/impl/util/single_task_execution.go](https://github.com/flyteorg/flyte/blob/master/flyteadmin/pkg/manager/impl/util/single_task_execution.go) - [flytepropeller/pkg/compiler/test/testdata/branch/compiled/success_6.json](https://github.com/flyteorg/flyte/blob/master/flytepropeller/pkg/compiler/test/testdata/branch/compiled/success_6.json) - [rfc/system/3553-config-override.md](https://github.com/flyteorg/flyte/blob/master/rfc/system/3553-config-override.md)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

eapolinario commented 4 months ago

cc: @ursucarina

architrathore commented 3 weeks ago

Ran into this again, and had some more observations.

It seems that the renaming works in the Nodes view only for @workflow
- Outer workflow can rename (sub)workflows
- Dynamic can rename (sub)workflows
But workflows or dynamic cannot rename tasks or dynamics in the Nodes view
In all of the cases, the graph view is correctly updated ✅

Minimal reproducing code covering all cases:

from datetime import datetime

from flytekit import task, workflow, dynamic
from typing import List

DEFAULT_START_DATE = datetime(2024, 1, 1)

@task
def simple_task(name: str) -> str:
    return f"Hello {name}"

@dynamic
def simple_dynamic(name: str) -> str:
    return simple_task(name=name)

@dynamic
def dynamic_with_task_renaming(name: str) -> str:
    return simple_task(name=name).with_overrides(
        name="simple_task_renamed_from_dynamic"
    )

@workflow
def sub_workflow(name: str) -> str:
    return simple_task(name=name).with_overrides(
        name="simple_task_renamed_from_sub_workflow"
    )

@dynamic
def dynamic_with_sub_workflow_renaming(name: str) -> str:
    return sub_workflow(name=name).with_overrides(
        name="sub_workflow_renamed_from_dynamic"
    )

@workflow
def hello_world_wf(
    name: str = "world",
    start_date: datetime = DEFAULT_START_DATE,
) -> List[str]:
    # ❌ Workflow renames task
    res1 = simple_task(name=name).with_overrides(
        name="simple_task_renamed_from_main_workflow"
    )
    # ❌ Workflow renames dynamic
    res2 = simple_dynamic(name=name).with_overrides(name="simple_dynamic_renamed")

    # ❌ Dynamic renames task
    res3 = dynamic_with_task_renaming(name=name)

    # ✅ Dynamic renames sub workflow
    res4 = dynamic_with_sub_workflow_renaming(name=name)

    # ✅ Workflow renames sub workflow
    res5 = sub_workflow(name=name).with_overrides(name="sub_workflow_renamed")
    return [res1, res2, res3, res4, res5]

Execution: https://serverless.union.ai/org/architrathore/projects/default/domains/development/executions/f5bf474defebd4e99b10/nodes

flyteorg / flyte