search

sharkdp / insect

High precision scientific calculator with support for physical units
https://numbat.dev/
MIT License
3.17k stars 126 forks source link

Insect giving incorrect result! #403

Closed charlescochran closed 10 months ago

charlescochran commented 10 months ago

image

Insect seems to be incorrectly reporting this simple division, whereas Ubuntu's built-in calculator tool gets it right.

This is Insect terminal version 5.9.0 (installed with npm), but the web version seems to be doing it as well.

Note this behavior:

image

Perhaps it is simply rounding the result for display? But this is alarming nonetheless!

sharkdp commented 10 months ago

This is just rounding when displaying the numbers. The precision is not lost. But I agree that this is unfortunate. Please see https://github.com/sharkdp/numbat/issues/179, https://github.com/sharkdp/numbat/issues/219 and https://github.com/sharkdp/insect/issues/54 for more details.