flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.43k stars 581 forks source link

[BUG] Cannot pass enums for execution to remote workflows fetched using `remote.fetch_workflow` #3782

Open redartera opened 1 year ago

redartera commented 1 year ago

Describe the bug

Flytekit throws an error when attempting to execute a remote workflow that takes an enum as an input

Error for a workflow that takes input_enum as an enum:

╭───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────╮
│ /Users/reda/Projects/unionai_bugs/enum_remote/replicate.py:52 in <module>                                           │
│                                                                                                                     │
│ ❱ 52 workflow_execution = remote.execute(                                                                           │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1138 in execute         │
│                                                                                                                     │
│ ❱ 1138 │   │   │   return self.execute_remote_task_lp(                                                              │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1227 in                 │
│ execute_remote_task_lp                                                                                              │
│                                                                                                                     │
│ ❱ 1227 │   │   return self._execute(                                                                                │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1002 in _execute        │
│                                                                                                                     │
│ ❱ 1002 │   │   │   │   │   hint = type_hints[k]                                                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'input_enum'

This is similar to https://github.com/flyteorg/flyte/issues/2278 though the error message is different

Expected behavior

Workflow should execute

Additional context to reproduce

Here is a script to reproduce the bug against the sandbox

import typing
from enum import Enum

import flytekit
from flytekit.remote import FlyteRemote
from flytekit.configuration import (
    Config,
    ImageConfig,
    SerializationSettings,
    FastSerializationSettings
)

class MyEnum(Enum):
    FOO = "FOO"
    BAR = "BAR"

@flytekit.task
def get_enum_value(input_enum: MyEnum) -> str:
    return input_enum.value

@flytekit.workflow
def wf(input_enum: MyEnum) -> MyEnum:
    return get_enum_value(input_enum=input_enum)

_PROJECT = "flytesnacks"
_DOMAIN = "development"
_VERSION = "abc"

remote = FlyteRemote(config=Config.for_sandbox())

img = ImageConfig.auto_default_image()
fast_serialization_settings = FastSerializationSettings(enabled=True, destination_dir="/root")
serialization_settings = SerializationSettings(
    image_config=img,
    project=_PROJECT,
    domain=_DOMAIN,
    version=_VERSION,
    fast_serialization_settings=fast_serialization_settings
)

remote.register_workflow(wf, serialization_settings=serialization_settings)

remote_workflow = remote.fetch_workflow(
    project=_PROJECT,
    domain=_DOMAIN,
    version=_VERSION,
    name="wf"
)

workflow_execution = remote.execute(
    remote_workflow,
    inputs={"input_enum": MyEnum.FOO},
    project=_PROJECT,
    domain=_DOMAIN,
    wait=True
)

Screenshots

No response

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

github-actions[bot] commented 6 months ago

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏

mhoeger commented 1 month ago

For anyone else - a workaround is to pass in:

inputs = {"input_enum": Literal(scalar=Scalar(primitive=Primitive(string_value=MyEnum.FOO.value)))}