Error in humidity calculation?

[wnelis]

I think that the formula to calculate the calibrated humidity does not match with the c code in data-sheet of the BME280, chapter 8.1, page 49. If I am correct, var5 is multiplied one time too often into var6.

[uscbutterworth]

I don't think this is an issue - the BME280 driver has the same math:

Adafruit_CircuitPython_BME280 driver (print commands removed):

var1 = float(self._t_fine) - 76800.0
var2 = (self._humidity_calib[3] * 64.0 + (self._humidity_calib[4] / 16384.0) * var1)
var3 = adc - var2
var4 = self._humidity_calib[1] / 65536.0
var5 = (1.0 + (self._humidity_calib[2] / 67108864.0) * var1)
var6 = 1.0 + (self._humidity_calib[5] / 67108864.0) * var1 * var5
var6 = var3 * var4 * (var5 * var6)
humidity = var6 * (1.0 - self._humidity_calib[0] * var6 / 524288.0)

Bosch Driver here (line 965 and on): bme280.c

var1 = ((double)calib_data->t_fine) - 76800.0;
var2 = (((double)calib_data->dig_H4) * 64.0 + (((double)calib_data->dig_H5) / 16384.0) * var1);
var3 = uncomp_data->humidity - var2;
var4 = ((double)calib_data->dig_H2) / 65536.0;
var5 = (1.0 + (((double)calib_data->dig_H3) / 67108864.0) * var1);
var6 = 1.0 + (((double)calib_data->dig_H6) / 67108864.0) * var1 * var5;
var6 = var3 * var4 * (var5 * var6);
humidity = var6 * (1.0 - ((double)calib_data->dig_H1) * var6 / 524288.0);

[wnelis]

It seems that the data-sheet of the BME280 contains another calibration formula. Using the data-sheet version 1.3 as of 2016.05.03, page 49, the calibration formula is:

var_H = (((double)t_fine) – 76800.0);
var_H = (adc_H – (((double)dig_H4) * 64.0 + ((double)dig_H5) / 16384.0 * var_H)) *
(((double)dig_H2) / 65536.0 * (1.0 + ((double)dig_H6) / 67108864.0 * var_H *
(1.0 + ((double)dig_H3) / 67108864.0 * var_H)));
var_H = var_H * (1.0 – ((double)dig_H1) * var_H / 524288.0);

Reworking the data-sheet formula to something looking like the Adafruit driver results in:

var1 = t_fine – 76800.0;
var2 = dig_H4 * 64.0 + dig_H5 / 16384.0 * var1;
var3 = adc_H - var2;
var4 = dig_H2 / 65536.0;
var5 = 1.0 + dig_H3 / 67108864.0 * var1;
var6 = 1.0 + dig_H6 / 67108864.0 * var1 * var5;
var6 = var3 * var4 * var6;
var_H = var6 * (1.0 – dig_H1 * var6 / 524288.0); 

The definition of each variable is in the two sets of formulas the same, except for the second definition of var6. Now the question is which of the two sets of formulas is the correct /best one.

In the mean time, I have changed my BME280 driver to contain both calibration formulas. At a temperature of about 21 [C] and a relative humidity of about 70%, the difference between the two results is less than 0.0001 %. Thus var5 seems to be very close to 1.

(Btw, the complexity of these calibration formulas is mind boggling.)