search

avsa242 / mlx90621-spin

Parallax P8X32A/Propeller, P2X8C4M64P/Propeller 2 driver object for the Melexis mlx90621 thermal sensor array
MIT License
0 stars 0 forks source link

Pre-calculated offsets are imprecise #2

Closed avsa242 closed 4 years ago

avsa242 commented 6 years ago

...and possibly inaccurate. PTAT calculation, as well as probably other pre-calculated values, are currently calculated using integer only math because SPIN has no floating point support, so things like offsets are imprecise, and possibly inaccurate. For reference, the example calculations in the Melexis datasheet have 8 decimal places!!!

Using fixed-point math would normally be an option, by scaling everything up by some static factor, but some numbers involved end up overflowing even the largest storage size (long). Kt1 is a good example of this, when using a scaling factor of 1000. 100 is within bounds, but offers no precision when used on, for example, Kt2.

I'm reluctant to integrate something like Float32 into the driver/object, as it just seems too much of a burden (here's wishing SPIN had better support for fractional numbers).

avsa242 commented 4 years ago

Closed - very old issue that's no longer valid - no correction at all is currently done to any sensor readings.