ugorji / go

idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go]
MIT License
1.85k stars 295 forks source link

Coercion from Float to Int isn't allowed #358

Closed BrianGenisio closed 3 years ago

BrianGenisio commented 3 years ago

I see that you support Int=>Float coercion. But you don't support Float=>Int coercion. I get that there could be data loss, but there are times when this is valuable. Would you consider this as a decoder option?

More info on my use case: I have some JSON data that I deserialize as generic (map[string]interface{}) and then encode using MsgPack. Then I use that MsgPack data do decode it into a concrete type. The json.Unmarshal() default behavior will turn all Number types into float64. So if the MsgPack encoded data gets a float, but the type I'm decoding into is an int, the decoder fails. I can likely create a custom decoder to fix this, but I really want the receiver type that gets decoded from the MsgPack to have the final say:

JSON {"foo": 1}

Unmarshalled JSON data into map[string]interface{}

{
  "foo": 1.0
}

Decode into struct fails

type Thing struct {
  Foo int `json:"foo"`
}
BrianGenisio commented 3 years ago

Was thinking about this more and the JSON decoder also fails if you try to decode a float into an int. The problem is that the generic form is always a float. So perhaps the decoder option is to allow coercion only if the float is really an int?

ugorji commented 3 years ago

Yes, I will look into it over the weekend.

It's an excellent idea to have both forms supported i.e. allow decoding into a destination number from a number in the stream (so long as the destination can accomodate the stream number e.g. if decoding into a uint64, then it must be a positive integer or a positive floating point number with no fractional part, etc).

Should have something within a few days.

ugorji commented 3 years ago

And the support will be consistent for all formats across the board.

BrianGenisio commented 3 years ago

Another thing to consider... not sure how coupled you want to be... but you can read JSON like this:

jsonEncoder := json.NewDecoder(bytes.NewReader(jsonData))
jsonEncoder.UseNumber()

In that case, the type of the data is not Float64, but it is a json.Number which is a string that can be cast as Float64() or Int64(). I'm currently using that to "fix-up" the JSON data by checking for "." which feels a bit safer than floatValue == float64(int64(floatValue)) for detecting whether this is a Float or a Number.

Just riffing a bit.

ugorji commented 3 years ago

I've never liked the idea of UseNumber. I think a more natural handling with proper error detection is preferred.