contribsys / faktory

Language-agnostic persistent background job server
https://contribsys.com/faktory/
Other
5.76k stars 229 forks source link

Number gives floating point number due to JSON marshal/unmarshal #380

Closed ghilesmeddour closed 3 years ago

ghilesmeddour commented 3 years ago

Hello,

The Go json package assumes float64 for numbers, this alters the content of Args and Custom for a Job when it comes to large integers.

Example:

C: PUSH {"jid": "718576994e934703bf414d6dc9677e3a", "jobtype": "foo", "args": [899999999999999990, 999999999999999990], "queue": "default", "reserve_for": 1800, "at": "", "retry": 25, "backtrace": 5, "custom": null}

S: +OK
C: FETCH default

S: $263
{"jid":"718576994e934703bf414d6dc9677e3a","queue":"default","jobtype":"foo","args":[900000000000000000,1000000000000000000],"created_at":"2021-09-24T08:30:19.630407267Z","enqueued_at":"2021-09-24T08:30:19.630787813Z","reserve_for":1800,"retry":25,"backtrace":5}

Thanks

mperham commented 3 years ago

What do you expect me to do for this? What is the fix?

jbielick commented 3 years ago

Could Args be json.RawMessage? Does the server ever really read them at all?

ghilesmeddour commented 3 years ago

I wondered if this was the desired behaviour. Shouldn't the Faktory Server keep the content as is, since it is not supposed to care about the content of Args, but just to serve them when the job is fetched?

mperham commented 3 years ago

One fundamental assumption that Faktory makes is that your arguments serialize to native JSON and JavaScript numbers are limited to 53 bits. If you want to send a number that is larger, it is typical to convert it to a string first.

mperham commented 3 years ago

Could Args be json.RawMessage? Does the server ever really read them at all?

There are a few times where faktory does look at the arguments, for example with the uniqueness feature in Faktory Enterprise. 

ghilesmeddour commented 3 years ago

Thanks @mperham, I understand that this assumption is made. I think I can close this.