flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.76k stars 657 forks source link

[BUG] Relaunch workflow converts large numbers in array of structs to objects. #5424

Open njubatyrov opened 5 months ago

njubatyrov commented 5 months ago

Describe the bug

When relaunching a workflow in FlyteConsole, large numbers within an array of structs are incorrectly converted into objects. For example, 1716759253199713 will become { "low": 695446369, "high": 399714, "unsigned": false }. This issue specifically occurs when the value of a key within the struct of an array is a large number, such as a Unix timestamp in microseconds.

The root cause appears to be in the parseJson.ts of flyteconsole repo. This file parses JSON and returns a Long type for large numbers. However, when this Long type is included when pressing "Relaunch" button, it is not correctly serialised and thus potentially causing unexpected behaviour.

Expected behavior

When relaunching a workflow, the input data should be accurately populated in the text area prior to launch.

Additional context to reproduce

  1. Create a workflow where the input arguments are a collection of objects, and one of the key/value pairs is an integer that overflows JavaScript's Number type.
  2. Once the job is complete, click "Relaunch". You will observe that the value has been converted into an object.

Screenshots

Screenshot of an input, observe the value of bar key: input-data

Screen of an input when pressing "Relaunch", observe the value turned into object: relaunch-data

No response

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

welcome[bot] commented 5 months ago

Thank you for opening your first issue here! 🛠

runllm[bot] commented 5 months ago

The issue you described seems to be related to how FlyteConsole handles large numbers during JSON parsing and serialization. Specifically, the Long type used for large numbers is not correctly serialized when relaunching workflows, leading to unexpected behavior.

Would you like more details on how to address this issue, including potential code snippets and further

Sources:


This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

eapolinario commented 5 months ago

cc: @jsonporter , can you take a look? This seems to be particular to the browser javascript, but maybe we can catch this error earlier?

kumare3 commented 5 months ago

@eapolinario / @jsonporter this is actually a struct and a json type. @njubatyrov If you use Flyte native float or int it does work right as seen below. Screenshot 2024-06-05 at 11 39 14 AM

The problem is possibly with json schemas in the return

njubatyrov commented 5 months ago

@kumare3

The issue only happens when you have an input as an array of dataclass objects which is converted to struct.

kumare3 commented 2 months ago

@njubatyrov we are working on an complete JSON overhaul in flyte - follow the issue - https://github.com/flyteorg/flyte/issues/5318