murrayo / yape

yet another pButtons extractor
MIT License
11 stars 7 forks source link

It is possible to set a fix number of digits for decimal precision for y axis? #10

Closed casep closed 6 years ago

casep commented 6 years ago

I am processing a pButtons file and I get this extra precision for the Y axis (0.075000000000000001 instead of 0.075) It is possible to set a fix number of digits for decimal precision for y axis? Y-axis extra precision

murrayo commented 6 years ago

yeah, I am sure, this has annoyed me in the past as well. Next time I am in the code I will see if there is a generic way. the trick is to differentiate when you need say precision of 3 for something like ms latency, which is the exception, most of the time it integers.

kazamatzuri commented 6 years ago

https://github.com/murrayo/yape/commit/5a705b63725d83900aab88babc6e72735355b2d6#diff-2209a57b778bd1af21cd425e688f46eb fixes it. I'll look into making this configurable too. (sorry for the messy diff, i also ran a code formatter...)