danielgtaylor / python-betterproto

Clean, modern, Python 3.6+ code generator & library for Protobuf 3 and async gRPC
MIT License
1.5k stars 210 forks source link

Very small Durations are formatted using scientific notation when converted to dict #447

Open elwinar opened 1 year ago

elwinar commented 1 year ago

I have a field in a message that is a very small timedelta (initialized with timedelta(microseconds=11)). When converting this to JSON using the to_dict method, I end-up with the value represented like 1.1e-050s.

The reason is this line https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/__init__.py#L1535, the float is converted to a string using str, which defaults to scientific notation for small values. Then, the padding part is adding 0s until there is 3/6/9 chars after the ., which in this case result in the value being incorrectly shifted 48 positions to the right :joy: .

Switching the line to "{:f}".format(delta.total_seconds()).split(".") should solve the issue by preventing scientific notation. This seems to be the original intention, and as a bonus it makes interoperability with other parsers easier. For example, I discovered with because the program consuming said message once converted in JSON is written in Go, and the time.ParseDuration doesn't understand scientific notation and is considering the e as an unit (which it doesn't know about, thus fail.)

Do you want me to submit a PR? I've had a look around the test suite and couldn't find the correct place to add a testcase for this, but I'll be glad to add it too if pointed in the right direction.

Gobot1234 commented 1 year ago

Thank you for the big report, submitting a PR to change the line to f"{delta.total_seconds():f}".split(".") (an f string would be preferred here) would be a great help. A test case could go in test_features.py test_to_dict_datetime_values

elwinar commented 1 year ago

Here you go https://github.com/danielgtaylor/python-betterproto/pull/448. I've had to trim 0s first before padding to keep the same alignment properties because the format seems to have a default precision of 6.

Gobot1234 commented 1 year ago

Awesome thank you