Math: Round trailing nines up in `@convert`

progval / Limnoria

A robust, full-featured, and user/programmer-friendly Python IRC bot, with many existing plugins.

https://docs.limnoria.net/

Other

622 stars 174 forks source link

Math: Round trailing nines up in `@convert` #1573

Open progval opened 6 months ago

jlu5 commented 5 months ago

I'm honestly surprised that this code is so complicated. Would anyone actually complain if we used a fixed (or configurable?) max precision like 2 or 3 decimal places?

progval commented 5 months ago

I would.

2 is definitely too little (8/9 = 1.125) so I wouldn't feel comfortable with 3 either.

The complexity is because it tries to fix physics calculous rules, ie. output should have as many significant digits as the input