BME680 calculation of air quality percentage

andreas-bulling commented 5 years ago

Hi,

would it be possible to directly integrate the calculation of air quality into Tasmota so that it can be published over MQTT?

It seems the HA bme680 sensor platform supports basic air quality calculation using gas resistance and humidity measurements to calculate a percentage based air quality measurement: https://www.home-assistant.io/components/sensor.bme680/ But it seems it requires a sensor directly connected through I2C.

A similar calculation I found here: https://github.com/robmarkcole/bme680-mqtt/blob/master/bme680-air-quality-RC.py

Jason2866 commented 5 years ago

On which document (source?) is this calculation based?

andrethomas commented 5 years ago

Should be possible to add a calibration command but I do not yet have this sensor so cannot yet experiment with it. Not sure if @arendst or @ascillato has one.

henfri commented 5 years ago

Hello,

the air-quality in percent is only possible with the propietary Bosch Library.

Greetings, Hendrik

andreas-bulling commented 5 years ago

What about the above formula/calculations? Do you deem them worth implementing?

ascillato2 commented 5 years ago

The calculations are super easy. Adding the label enhancement to add this. Thanks for sharing :+1:

henfri commented 5 years ago

Hello,

the calculations are indeed super simple. Too simple? Also I don't like that the humidity is included in the AirQuality value. What humidity is the best humidity? It would be worth comparing the result of these simple equations to those of the Bosch Library. Furthermore, it should be possible to come up with a regression of what the bosch-lib does.

Greetings, Hendrik

ascillato2 commented 5 years ago

@henfri

If you have a ready to use formula to implement this, please share it.

henfri commented 5 years ago

Hello,

unfortunately, I don't.

Greetings, Hendrik

ascillato commented 5 years ago

Ok. No problem, we will add the formula proposed by @andreas-bulling Thanks

andreas-bulling commented 5 years ago

Actually two formulas to choose from potentially...

mike2nl commented 5 years ago

@ascillato2 @Jason2866 @andrethomas 04:55. To bring up some light in it. Point1. To get the right Air Quality you have to use a special function provided in the BME680. See my next posting to henfri.

Point2. Please read the document for software design which can you get from the chip designer.

Point 3. I was busy with some test, but in the last 2 month there is so many work with different sensors about changings in libs or chip dsign. Yes chip design because some version are retired. So i got a lot of software desgin docs in PDF after i asked nicely and gave my company name for it. Nothing with esp but in the PLC world very well known. These special software designer docs are mostly not public. There are some but only with half of the information.

Point 4. After the first read of code i would change things but all the work has to be done. When the TSl2561 issue is done, the issue with 30% messaure difference between the TSL and the BH1750 must be done. Then norrmally the VEML6075 UVa and UVb, but i could first look into the BME680.

about the first comment: It seems the HA bme680 sensor platform supports basic air quality calculation using gas resistance and humidity measurements to calculate a percentage based air quality measurement:

That is the half work only and it has to be done more. I had done very qucik and dirty things based on these document and when i qompare these new values against laboratory messaurement i come very close to to good values. But yeah i have 2 hands only ;-)

mike2nl commented 5 years ago

@henfri You are someone who understand my concerns. The BOSCH library is the only one which handle it right in the moment. That lib handles the chip calculation for it. And when i look on the software designer doc i see that back. I was busy with implementing it without any use of a library. That i learned from arendst/Theo a while ago because he told me so. Staars and me have the same meaning of that to get more stable things in it and we are busy togehter.

andreas-bulling commented 5 years ago

Reimplementing the BOSCH algorithm would be even nicer, of course, but I think at the moment any calculation would be better than none. It would at least allow for relative comparison of values over time. And I am not into that business but I wonder if there is a single, widely agreed on definition of air quality anyway. That is, there might not be a right or wrong in terms of how an air quality metric is calculated.

mike2nl commented 5 years ago

@andreas-bulling you are right in the case for the meantime. I discussed it with @ascillato2 today in the early morning and we will look into it. But we all have a day job and possible ascillato2 can add the simple formula first until i'm ready to PR the new driver. And there definitions about air quality. As an Example that image here. It is based on the use of the 100% calculation. It comes from official sources here in the Netherlands. have to search back the link. Jesus to many links in my browser. I will get this straight too and i think we need a wiki page for it. Possibile i have to hire some people, hehe. air quality

andreas-bulling commented 5 years ago

@mike2nl Thanks a lot - will be great to see this implemented (Dutch calculation is fine, I honestly think few people will care where the formula comes from). I might be wrong but from skimming the code in the two links at the top they might also use the 100% calculation.

henfri commented 5 years ago

@andreas-bulling It is right that noone will care where the calculation comes from. But as far as I see, the above mentioned calculation is flawed. 1) it is referenced to the average of the first 50 readings. So if you have bad air when starting the sensor, you will get different readings, compared to a situation where you start the sensor when having good air quality 2) The Humidity will influence the dependancy resistance=f(humidity) but whether the relation is right, we don't know. Bosch has a lab and come up certainly with a better resistance=f(humidity) equation than what is implemented there.

Greetings, Hendrik

Jason2866 commented 5 years ago

I dont want to see a bling bling value which has nothing to do with reality This was my initial question. On which source is this calculating based? This question is still not answered. So it is nothing with background @mike2nl, @andrethomas and @ascillato i am against implementing it. It suggests something and isnt validated on a reliable source

andrethomas commented 5 years ago

I agree if the formula is not derived from manufacturer data then there is no point in implementing it... it will just be another random number generator like the SHT11 sensors.

This is a non-starter for me because even if you could derive a valid "air quality" value from what the data provided by the BME680 sensors the question is what does 48% air quality mean from one person to another, or from one organism to another - If you thought it would mean different things then you are right - so the conclusion is that it would depend on use case.

Temperature and humidity for me is only about comfort level. Co2 measurement I view from a health perspective insofar that the scientific standard is well defined for various use cases

My definition is limited to the qualia of a room based on what I can measure at the moment but to try and derive a percentage from this would be irrelevant.

Is it too hot or cold?
Is it too humid or too dry?
Is there enough ventilation to keep Co2 levels at acceptable levels?

But the above is not the only way to decide how you would represent it as a % so the best that can be done is to try and define it as qualia (quality of human experience) which will differ from person to person with specific reference to temperature and humidity.

If air quality can be defined as a fixed desired outcome things would be different.

ascillato2 commented 5 years ago

After all this discussion It is shown that this value will be just a gross interpretation. So, I think would be better that anyone that wants that, make the calculation he/she thinks fits better for their use case in their home automation software and let Tasmota just send the actual readings of the sensor.

Adding the label hold over for future reference.

Thanks everyone for sharing theirs ideas. Very interesting discussion.

523p commented 5 years ago

My approach:

1) Connect and burn in. 2) Measure some time - with fresh air, with bad air. Observer your own MIN and MAX resistance values. For me: 50-250 kOhm (BAD-GOOD). 3) So, the range is 200 kOhm (0% ... 100% air quality) 4) Calculate % (measured resistance / range * 100) 5) Done

But, I am not very happy with this sensor. For 15 minutes after start, it is useless (you have to ignore measured resistance). But you have to measure. And resistance is affected by number and speed of measurements... grrrr. Crap bosh sensor with bad docs and closed source.

andreas-bulling commented 5 years ago

@523p Thanks for the information - this is valuable. I just got this sensor and I am now wondering if you have any suggestion for a better one that addresses the limitations you mentioned?

andrethomas commented 5 years ago

The MICS-4514 looks interesting, but I am not sure what to make of the information in its datasheet yet. You would still have to add temperature, humidity and Co2 sensors to the mix to make it useful I guess, but interesting nevertheless.

mike2nl commented 4 years ago

@andrethomas @Jason2866 @henfri @andreas-bulling @ascillato2 Hi all,

sorry for this long post but it is needed.

I picked it all up again because all postings here, include mine, missing important things. First the ambient tempoerature is set to 25 deg.C. That's not the best way to do so because the ambient goes into calculation. So but there are a lot of registers with many information i found to handle that. Next is the calulation of the IAQ. The humidty is needed because some gasses lay down on the thin film which is used to messaure the gas resistior (BOSCH application documentation). So therefore are registers too which give us some more information from the factory calibration. Done a lot of research again for all that.

The gas resistor value is depending on heating the gas sensor, the time of heating and current values from temperature, ambient, humi and some regsiters. The heating process push up the temperture messaured values. Around 0.7 deg.C. The humidity too, around up to 2%. Tested the heating time down to 40ms in a range of 30..150ms in steps of 10ms. How shorter the heating time is how better the values for temp and humi. But the gas resistor values goes down too. Default it is a max of 500K (500.000 ohm). All in the BOSCH BME680 has moved values around ~7% at all but that can be compensated.

Next is the value for sea-level. No one has seen that or read the driver code we have? The default sea-level value is 500 meter. For me in my place i have -5 meter, yes minus. So that is missing as a #define too. Tested it and it gives me then the real values in my hight. I have some very good calibrated things to control it.

Yesterday i talked about all that to @Jason2866 and showed him some of my findings. And he is righjt better do nothing then bling bling things. I'm on it but it needs time to solve certain things. And i have to write code away from tasmota first....to test things to get it 100% clear. Everyone who is willing todo test with me together is very welcome after i have something to test. @Jason2866, kick me again, haha.

So there are some points where i working on right now. The sensor works correct only with the BOSCH library but it is possible to get some of the functions out of the original. E.g. the IAQ calculation. It looks easy yes, but it really complex. Ask the books of physics and chemical things so i have done the last few weeks. BOSCH's libray is working based on these things i had read to understand it really, really good.

arendst commented 4 years ago

Next is the value for sea-level. No one has seen that or read the driver code we have? The default sea-level value is 500 meter. For me in my place i have -5 meter, yes minus. So that is missing as a #define too. Tested it and it gives me then the real values in my hight. I have some very good calibrated things to control it.

@mike2nl as this could easily be fixed I scanned the library as currently used by Tasmota but failed to find any relation to sea-level, offset or 500. Can you provide more info where you found the 500 meter pressure offset information?

illusdolphin commented 4 years ago

@arendst Is there any issue adding this proprietary Bosch Library? Looks like it's available for ESP8266. I know it has a different license model, so it might be possible to add this as additional option for custom build so user will be responsible for license impact. Existing custom implementations are hard to trust - too simple or unstable by design:

Low-high * 0.75 + 0.25 diff(40, hum%) - looks too simple (but stable and predictable). Looks like nobody really known about real min/max and is it linear resistance/IAQ.
log version (found one in the internet) - gives very strange results
Auto calculate low/high - for sure gives floating result Maybe adding this as option can give some statistics for dependency between Bosch IAQ and input (just 4 parameters, maybe pressure is out of scope), maybe we can build something approximation then. Thanks!

Jason2866 commented 4 years ago

@illusdolphin the bme680 driver does use the Bosch library and no simple air quality calculation is added in Tasmota. Just no one has found the time to implement it like it is suggested from Bosch A PR is highly welcome for review

illusdolphin commented 4 years ago

Maybe I'm wrong, but here https://www.bosch-sensortec.com/software-tools/software/bsec/ we can download this magic software BSEC ( https://www.bosch-sensortec.com/media/boschsensortec/downloads/bsec/bsec_1-4-7-4_generic_release.zip is a direct link, this software is mentioned inside datasheet). Inside zip: 1) bsec_iot_example.ino (and .c and .h) 2) inside "\BSEC_1.4.7.4_Generic_Release\algo\lite_version\bin\esp8266" libalgobsec.a (compiled lib?) + bsec_interface.h I'm fine with JS/c#, but quite new in pure c IoT. Just looks like those sources should be easy to add if you know how to do this :) Maybe somebody can at least say if this .a file can be compiles with tasmota?

illusdolphin commented 4 years ago

Also found https://github.com/arendst/Tasmota/issues/2969 with issues and memo about linker https://github.com/BoschSensortec/BSEC-Arduino-library#esp8266---modify-the-linker-script-header . Solution will take min 17k of ROM + 1Kb of RAM that is acceptable for custom build (like define #bme680_bosch_bsec ).

Jason2866 commented 4 years ago

To many changes in build chain. This will make a lot of troubles. Tasmota supports two IDE. Arduino and PlatformIO. For both Platforms a solution has to be found to patch the files. Highly unlikly that this will be done for a small feature in ONE sensor. I did the latest changes in PlatformIO setup for Tasmota. I will vote against for a such a approach

ruimarinho commented 4 years ago

Me and @mcspr have been working together during the last month to add support for BME680 using BSEC on espurna. The experience is completely seamless, requiring zero modifications on the user side - it just works.

I am an early Tasmota user as well and use it in many of my ESP8266 projects (and getting even more excited about the Zigbee bridge support for Sonoff) so sometime in the future I'd like to bring the same level of support here, time permitting.

I hope it serves as inspiration in case others want to jumpstart that work. Our approach to licensing was:

To include Bosch's own Arduino library as part of a user-downloaded dependency (the package being BSD-3-Clause licensed), with all copyright notices carried over.
To require an opt-in flag to enable support for the sensor, mentioning on every document that enabling the flag is confirmation of agreement of the Bosch's license.
To not distribute a pre-compiled binary with support for this sensor, instead requiring users to proactively seek to enable the correct flag.

This is truly a fantastic sensor which deserved a special attention to detail.

henfri commented 4 years ago

Great Work. Thanks for it! Would bei really cool to see it Here as well

Jason2866 commented 4 years ago

Since most Tasmota users are using pre built binaries (a great benefit from Tasmota) this will probably not the way of a wide useage for this driver. It would recommend to ask @arendst if he will accept in general a driver that needs accepting a extra special license. I dont think he will be fine with. The ugly build chain patching is still necessary?

arendst commented 4 years ago

My first reaction would be if Bosch wants to sell these sensors AND have users using it then they shouldn't use such a restricting license.

As said before many users use pre-compiled binaries and I won't add any interface to satisfy any license.

In short, they have the wrong license for Tasmota. Pity Bosch.

derFrickler commented 4 years ago

Maybe it would be a option to add support for the GY-MCU680V1 Board, which contains the BME680 and an onboard processor doing the Bosch BSEC stuff. It outputs IAQ and other values via serial interface.

Check this project for an example on how to use it: https://www.hackster.io/xxlukas84/bme680-gy-mcu680v1-indoor-air-quality-meter-901d02

Jason2866 commented 4 years ago

@derFrickler not directly related to the issue which is closed anyway. Since this BME680 version is completely different. Please open a new feature request.

ams2121 commented 2 years ago

I know this is closed, just for reference for others who find this thread. Here is a PDF from a UK indoor air quality group which appears to provide basic guidance with information on which standards the guidance came from.

https://www.betterindoors.com/wp-content/uploads/2019/09/IAQ_Rating_Index.pdf

arendst / Tasmota

BME680 calculation of air quality percentage #5024