Prefect v2 to v3 transition, idempotency key generation not consistent

mattijsdp commented 1 month ago

Bug summary

I recently upgraded to Prefect v3. I have some runs which run daily and Prefect creates scheduled runs such that there are always 3 in the future. With the transition, these daily runs accidentally ran twice for three days and then ran once again as expected.

I believe this is because the idempotency key generation is every so slightly different e.g.: scheduled d6c42edb-55da-4dc0-9a62-b9cb693695e4 2024-09-22T01:00:00+00:00 vs. scheduled d6c42edb-55da-4dc0-9a62-b9cb693695e4 2024-09-22 01:00:00+00:00 Notice the space after the date instead of 'T'. It is quite hard for me to reproduce this unfortunately but I thought it would be good to share.

Version info (`prefect version` output)

The output post upgrade:

Version:             3.0.2
API version:         0.8.4
Python version:      3.11.9
Git commit:          c846de02
Built:               Fri, Sep 13, 2024 10:48 AM
OS/Arch:             linux/x86_64
Profile:             ephemeral
Server type:         server
Pydantic version:    2.9.1



### Additional context

_No response_

zzstoatzz commented 1 month ago

hi @mattijsdp - thank you for the issue! we will look into this

for later, here's where this is coming from

and it appears the difference is that we are now using the default pendulum datetime -> str render, as opposed to iso8601

In [1]: from pendulum import DateTime

In [2]: f"{DateTime.now()}"
Out[2]: '2024-09-24 14:53:26.419160-05:00'

In [3]: f"{DateTime.now().to_iso8601_string()}"
Out[3]: '2024-09-24T14:53:30.409176-05:00'

aaazzam commented 1 month ago

:old-man-shakes-fist-at-timezone-rendering:

PrefectHQ / prefect