search

ericlagergren / decimal

A high-performance, arbitrary-precision, floating-point decimal library.
https://godoc.org/github.com/ericlagergren/decimal
BSD 3-Clause "New" or "Revised" License
531 stars 62 forks source link

format: fix `f` precision when exponent > 0 #148

Closed kjgorman closed 4 years ago

kjgorman commented 4 years ago

👋Hi @ericlagergren — we've noticed a small edge case in the f format flag. It looks like this will be also fixed in https://github.com/ericlagergren/decimal/pull/144 but I figured I would report it anyway and propose a small change for the existing version if that one is still WIP.

Previously we would take only precision without also considering the exponent, meaning you would have scenarios like 200 being formatted with %.2f as just 200 as x.exp would be 2, and x.Precision() would be 1.

ericlagergren commented 4 years ago

hi, just letting you know I did see this issue.

I really appreciate the PR and I plan on merging it in.