Calibration / correction of sensor readings

zerni commented 4 years ago

Is there any standardised way of calibrating the sensors provided with the Enviro+ or can a simple linear correction be applied in python to some of these?

The following readings seem to be off (I don't have comparables for the others):

Temperature: Around 10C to high (enviro board is attached to a RPi4 with a heatsink and at 90deg angle, so it's unlikely this is due to the heat from the CPU)
Humidity: Is reading around 15-20% while other sensors (incl Luftdaten) suggest 46-52%
Pressure: Off by 6 hPa compared to Luftdaten (accuracy is pretty important here)

How are other's handling this issue?

roscoe81 commented 4 years ago

I had a similar problem and you’ll find some discussions on the topic here: https://forums.pimoroni.com/t/enviro-readings-unrealiable/12754/56 You’ll see that some people have chosen to use a separate sensor but I didn’t go down that path. I found that it was possible to correct the problem by separating the Enviro+ and Raspberry Pi boards and connecting them via a ribbon cable. Even then, I had to undertake regression analysis and apply cubic polynomial compensation to the temperature readings and quadratic compensation to the humidity readings. I also found it necessary to vary the polynomial factors if there are even minor changes to the enclosure that I’m using. With the two versions of my enclosure, the polynomial correction factors are:

No weather protection cover in place Cubic polynomial temp comp coefficients comp_temp_cub_a = -0.0001 comp_temp_cub_b = 0.0037 comp_temp_cub_c = 1.00568 comp_temp_cub_d = -6.78291 Quadratic polynomial hum comp coefficients comp_hum_quad_a = -0.0032 comp_hum_quad_b = 1.6931 comp_hum_quad_c = 0.9391

Weather protection cover in place Cubic polynomial temp comp coefficients comp_temp_cub_a = -0.00028 comp_temp_cub_b = 0.01370 comp_temp_cub_c = 1.07037 comp_temp_cub_d = -12.35321 Quadratic polynomial hum comp coefficients comp_hum_quad_a = -0.0098 comp_hum_quad_b = 2.0705 comp_hum_quad_c = -1.2795 The air pressure reading also needed altitude compensation and that’s built into my code too.

I now get temperature readings within +-0.5 degrees C and relative humidity readings with +-5% and my resulting code is here, if that helps: https://github.com/roscoe81/enviro-monitor/blob/master/Northcliff_AQI_Monitor_Gen.py

My enclosure is here: https://github.com/roscoe81/enviro-monitor/tree/master/3DP_Files and https://github.com/roscoe81/enviro-monitor/issues/2

My regression analysis code for the compensation is here: https://github.com/roscoe81/enviro-monitor/blob/master/Regression_Analysis/Northcliff_Enviro_Monitor_Regression_Analyser.py

A final thing to note is that my regression analysis drove me to conclude that I could not arrive at any reasonably effective temperature and humidity compensation without separating the two boards.

roscoe81 commented 4 years ago

Sorry, but I forgot to add the air pressure compensation. I used the standard formula for altitude and temperature compensation: comp_factor = math.pow(1 - (0.0065 altitude/(temp + 0.0065 alt + 273.15)), -5.257)

I found that temperature only had a relatively minor impact on air pressure but altitude has a big impact and is probably the cause of your issue.

David-Hari commented 4 years ago

So there is no standard formula for calculating these compensated values?

I don't have another humidity sensor to compare against, so it's going to be hard for me to find the right coefficients to plug into roscoe's formula above.

Looking at the code for the BME280 sensor, I can see that it attempts to do some sort of compensation already. So why isn't that working?

David-Hari commented 4 years ago

On second thought, I realise that the compensation from the BME280 Python code does not take the CPU temperature into account. I guess my real question is why it does not, given that it has such a big impact. Now, it would be impossible for the code to know if the sensor is close to the CPU or separated by (for example) a 10cm ribbon cable. But it would be nice if there were just one coefficient that the user could plug in to get at least semi-accurate results.

roscoe81 commented 4 years ago

Yes, I found that it's the heat generated by the Raspberry Pi that caused the issue. There is some CPU-based temperature compensation for the BME280 built into https://github.com/pimoroni/enviroplus-python/blob/master/examples/luftdaten.py . I tried using that compensation method and I found that it couldn't give me anywhere near an adequate level of accuracy. Without the ribbon cable, I found that the CPU temperature overwhelmed the BME280 readings so much, that no compensation could counter it. Even with the ribbon cable, I found that the CPU temperature impacted the relationship between the BME280's readings and the actual temperature/humidity in a non-linear manner.

David-Hari commented 4 years ago

I don't see any compensation applied to humidity in that example. Which means that it could be pushing data to Luftdaten that is totally inaccurate.

I am seeing readings of only between 13% and 19% humidity when I expect it to be much higher, It's been raining the past few days.

roscoe81 commented 4 years ago

That’s true @David-Hari. If the temperature is incorrect, the relative humidity will be also incorrect. In the early versions of my code, I used an algorithm that calculated the dewpoint using the incorrect temperature and humidity, did the temperature compensation and then used the new temperature with the to dewpoint to determine the correct relative humidity. That worked OK but I then found that quadratic humidity compensation gave an equally effective result. I have spent many hours on this issue and unless I’m missing something, I don’t think that there is any chance of having accurate temperature and humidity readings without a lot of physical (separating the Enviro+ and Raspberry Pi) and coding (compensation) changes.

David-Hari commented 4 years ago

It would be good if the compensate_humidity function allowed us to pass in the correct temperature (or the CPU temp) and use that in it's calculations, instead of relying purely on the temperature_fine variable that it calculates itself.

I don't have enough understanding of the numbers to make such adjustments myself.

nophead commented 4 years ago

I wonder if the board could be coupled to an ESP12F instead as that would give far less heating.

The gas sensor actually has three heaters in it and resistive droppers, so that probably raises the temperature of the PCB as well.

I have a fan blowing on mine and that makes the temperature correct without compensation but I still think the humidity reading is too low.

David-Hari commented 4 years ago

pimoroni / enviroplus-python

Calibration / correction of sensor readings #67