search

sparkfun / SparkFun_Indoor_Air_Quality_Sensor-ENS160_Arduino_Library

Other
2 stars 3 forks source link

Reduce loss of precision in compensated temperature and humidity #3

Closed dordnung closed 1 year ago

dordnung commented 1 year ago

When using this library i noticed a big difference between written and read compensations. So for example a temperature of 22.15°C and humidity of 52.70% will get to 21.85°C and 52%. The problem here is that the corresponding float values are converted into an integer before multiplication by 64 or 512 and thus the loss of precision is higher. I have adjusted this accordingly with this pull request

edspark commented 1 year ago

Makes a lot of sense. It's one of those, "Did I really do that?" moments. Thank you very much for the pull request.