search

sharkdp / insect

High precision scientific calculator with support for physical units
https://numbat.dev/
MIT License
3.17k stars 125 forks source link

Insect giving incorrect result! #403

Closed charlescochran closed 8 months ago

charlescochran commented 8 months ago

image

Insect seems to be incorrectly reporting this simple division, whereas Ubuntu's built-in calculator tool gets it right.

This is Insect terminal version 5.9.0 (installed with npm), but the web version seems to be doing it as well.

Note this behavior:

image

Perhaps it is simply rounding the result for display? But this is alarming nonetheless!

sharkdp commented 8 months ago

This is just rounding when displaying the numbers. The precision is not lost. But I agree that this is unfortunate. Please see https://github.com/sharkdp/numbat/issues/179, https://github.com/sharkdp/numbat/issues/219 and https://github.com/sharkdp/insect/issues/54 for more details.