search

cogentcore / core

A free and open source framework for building powerful, fast, and elegant 2D and 3D apps that run on macOS, Windows, Linux, iOS, Android, and the Web with a single pure Go codebase, allowing you to Code Once, Run Everywhere.
http://cogentcore.org/core
BSD 3-Clause "New" or "Revised" License
1.33k stars 71 forks source link

spinner rounding is failing badly #819

Closed rcoreilly closed 5 months ago

rcoreilly commented 5 months ago

I tried to enter 99534 into spinner to attach to a process, and it converted it to 99533.99 which rounded down to ..32

and I'm seeing line 1192.9 in my debug list

rcoreilly commented 5 months ago

This is the culprit. reverting to horrible-but-functional sprintf-based version that works reliably. there was a reason I didn't go this route before.

sp.Value = mat32.Truncate(sp.Value, sp.Prec)
kkoreilly commented 5 months ago

I added a todo about making mat32.Truncate faster, but it probably isn't that much of a problem and thus it doesn't seem worth doing now.