flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.17k stars 550 forks source link

[BUG] Missing inputs when using datetime.date type hint. #5491

Closed dansola closed 2 days ago

dansola commented 1 week ago

Describe the bug

When using datetime.date as a type hint in a task, flyte appears to lose the input file:

FileNotFoundError: Failed to convert inputs of task 'wf.hello_date':
  [Errno 2] No such file or directory: '/var/folders/yv/l9q99njs4x35fp3x2_g8gykh0000gn/T/flyte-3c_0m7fn/raw/4f3290e595dcb84915aa481e45b92074/1f3c185c4a74647a397e5ca43ddcb5b3'

Expected behavior

Flyte should recognize the datetime.date object the same way it recognizes the datetime object.

Additional context to reproduce

Good example:

@task
def hello_date(date: datetime):
    print(f"Today is {str(date)}")

@workflow
def wf(date: datetime):
    hello_date(date=date)

pyflyte run --remote -p daniel date_bug/wf.py wf --date "2024-01-03"

This has no errors and works as expected.

Bad example:

@task
def hello_date(date: datetime.date):
    print(f"Today is {str(date)}")

@workflow
def wf(date: datetime.date):
    hello_date(date=date)

pyflyte run --remote -p daniel date_bug/wf.py wf --date "2024-01-03"

This results in the error:

FileNotFoundError: Failed to convert inputs of task 'wf.hello_date':
  [Errno 2] No such file or directory: '/var/folders/yv/l9q99njs4x35fp3x2_g8gykh0000gn/T/flyte-3c_0m7fn/raw/4f3290e595dcb84915aa481e45b92074/1f3c185c4a74647a397e5ca43ddcb5b3

Screenshots

No response

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

runllm[bot] commented 1 week ago

Flyte currently does not support the datetime.date object for passing to or from tasks. According to this GitHub issue, the Flyte team is aware of the limitation and has discussed adding a type transformer to support datetime.date. As a workaround, you can use datetime.datetime or convert the date to a string.

Would you like more details on how to implement this workaround?

Sources:


This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

mao3267 commented 3 days ago

@dansola Have you solved the problem? It seems to work on flytekit 1.12.3. Are you using an older version?

dansola commented 2 days ago

@mao3267 You're right. This was happening on 1.12.0. I'll close this, thanks!