[BUG] Boolean values within pydantic base model being casted to scalar value

jiwidi commented 5 months ago

Describe the bug

Trying to pass a pydantic model between task but its converting bool variables to int. Is there a way to make flyte work with pydantic models?

Expected behavior

For bool types to stay as bool types

Additional context to reproduce

I have this main.py file containing my workflow:

from flytekit import task, workflow
from pydantic import BaseModel

class Config(BaseModel):
    bool_var: bool

@task
def create_config() -> Config:
    return Config(bool_var=True)

@task
def check_bool(x: bool) -> bool:
    return x

@workflow
def workflow() -> bool:
    config = create_config()
    return check_bool(x=config['bool_var'])

When I run this with the command

pyflyte run issue.py workflow

I get the error:

TypeTransformerFailedError: Failed to convert inputs of task 'issue.check_bool':
  Error converting input 'x' at position 0:
  Cannot convert literal <FlyteLiteral(Literal) scalar { blob { metadata { type { format: "PythonPickle" } } uri:
"/var/folders/zr/72m6cw8x66b4g3q_s8dmy5c00000gn/T/flyte-bgc4g0zt/raw/d8a7be6139519884010995bc2aa7ddd9/cd165a40ea1442ca1acb6fbd4b3700f4" } }> to <class 'bool'>

The bool property in the Config object is converted to a scalar.

Screenshots

No response

Are you sure this issue hasn't been raised already?

[X] Yes

Have you read the Code of Conduct?

[X] Yes

runllm[bot] commented 5 months ago

Flyte currently has issues with handling Pydantic models, particularly with type conversions. Boolean values within Pydantic models may be cast to scalar values due to serialization issues. This is a known problem and has been discussed in the community.

Would you like more detail on potential workarounds or ongoing efforts to address this issue?

Sources:

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

novahow commented 4 months ago

Hi, @jiwidi

I guess this might relate to how flyte handles complex data types. Currently as a workaround, you can try the following:

from flytekit import task, workflow
from pydantic import BaseModel

class Config(BaseModel):
    bool_var: bool

@task
def create_config() -> Config:
    return Config(bool_var=True)

@task
def check_bool(x: bool) -> bool:
    return x

@task
def bool_parser(x: Config) -> bool:
    return x.bool_var

@workflow
def wf() -> bool:
    config = create_config()
    return check_bool(x=bool_parser(x=config))

kumare3 commented 2 months ago

fixed by - https://github.com/flyteorg/flyte/issues/5318

flyteorg / flyte