letscontrolit / ESPEasy

Easy MultiSensor device based on ESP8266/ESP32
http://www.espeasy.com
Other
3.24k stars 2.2k forks source link

MH-Z19 should disable ABC mode (Auto Base Calibration) upon init #466

Closed TD-er closed 6 years ago

TD-er commented 7 years ago

The MH-Z19 sensor has ABC enabled from factory, or at least two of my sensors now have their baseline messed up because the ABC mode was not disabled.

The automatic base calibration will set the lowest measured internal RAW value in the last 24h as new reference point at 400 ppm. This means the base line will make a big jump when being used in a not well ventilated room for more than 24h. And when opening a door or window after such a recalibration, the measured value will hardly change and stay at 400ppm for hours.

This can be seen in the graphs of two of my sensors (N.B. Domoticz was not recording between 6am and 9am this morning, when one sensor did its recalibration)

2017-08-06 20_17_42-domoticz 2017-08-06 20_18_18-domoticz

So in my opinion, the ABC mode should be disabled at init of the sensor.

A nice feature would be to trigger a recalibration when the sensor is clearly off with its calibration. Perhaps when the measured value is 400ppm for longer than a few samples (not taking initialisation into account) or when the measured value is limiting itself to a lowest value for a number of samples (e.g. minimum value of something like 475 ppm for hours without peaking inbetween) In the 1st example, the actual value measured is lower than 400ppm which the sensor will not output. This is very likely to happen when ABC was performed in a not well ventilated room.

Another nice feature would be to add a command to show the lowest RAW value and the set zero level as calibration. This could be JSON-formatted output, or even in the webinterface (not sure where/how).

I am running ESP easy v2.0.0-dev11 and the sensors used were never used for longer than a few minutes at a time and never written anything to them before being used with ESP easy. So internal sensor settings were as factory default as can be.

In order to reproduce the behavior, one has to run the sensor for more than a day continuously, without restarts. To reset the sensor, the sensor has to be placed outside (in the dark, no IR radiation) for a while and apply the URL http:///control?cmd=mhzcalibratezero without anyone being near the sensor producing CO2.

infernix commented 7 years ago

The default is correct (ABC left on) because the assumption that the user is intelligent and does correct calibration is a wrong one.

However, a setting could be added to the sensor that allows ABC to be disabled upon poweron.

As for applying calibration based on PPM values == 400 for X time, I have tried that and it does not yield any accurate results because it depends on an accurate outdoor-air calibration after power-on, which is an assumption one can almost never make.

infernix commented 7 years ago

And FWIW, if you open your windows daily to ventilate the room with the sensor, the results are reasonably accurate, but for an absolutely accurate sensor you will need to look at other options :)

TD-er commented 7 years ago

I don't think I understand your explanation. I do get the point that the standard user will not perform the calibration correct and at the right moment. However, I think the problems caused by the ABC feature rendering the measurements useless, will occur much more often than a new baseline calibration is needed.

You could detect the amount of ventilation by looking at the RAW value. If that stays the same for a while and is near the lowest RAW value seen in the last couple of weeks (perhaps the last few resets), you can perform the re-zero process automatically and ignoring it when no good ventilation is detected. It still leaves the question how often the reset of the baseline is needed and I don't think it is needed that often. Leaving it on as a default setting, most users will experience a useless measurement. For example the 1200ppm value around 20h in the top chart, that was in an empty room, which was without CO2 producing humans/cats for almost 8 hours and the window open.

Such behavior will also make it impossible to run a demand based ventilation scheme.

infernix commented 7 years ago

We cannot assume that the user will calibrate manually. If we do assume that, and we disable ABC by default, and the user powers on the sensor in a room with PPM levels of, say, 1000 - the sensor will show 400 for any real PPM between 400 and ~1000. Forever. That's not an acceptable default setting IMHO.

You're essentially asking to re-implement a smarter ABC but I do not think there is any ABC smart enough to handle PPM fluctuations. If by RAW value you mean the U value - how this value is composed is entirely unknown and I have not found any meaningful way to use it as an indicator to when calibration is to be triggered. Same for the PPM value; sometimes PPM drops < 400 and therefore using an if PPM is 400 for X time then do_calibrate did not yield good results in my testing.

ABC enabled sensor

If you look at the above graph, you see where each autocalibration happens. Most ABC events happen when CO2 PPM is already reported at a low point (~400), but even in the cases where ABC happens on a higher PPM count, it does not directly affect the current CO2 PPM reported value. So the ABC isn't doing as bad a job as you are outlining here.

All things considered, I suggest adding a feature to disable ABC at sensor powerup but leaving ABC enabled by default (which means ESPEasy does nothing with the sensor by default, unless an ABC Disable option is selected).

TD-er commented 7 years ago

Adding an ABC disabled option would already be a nice work around. At least it is the best of both worlds I guess, since there may well be good use cases for the ABC or else the manufacturer would not have chosen it.

I think my misunderstanding of your previous answer is about when to disable the ABC. Is it correct to assume to disable it as soon as possible after the sensor is restarted? Or are you saying it is always setting the initial zero-level at boot? I thought it only ran a baseline calibration after so many hours of operation.

And indeed, there are a lot of unknowns about the undocumented features of this sensor, which may affect a "more intelligent self-made ABC", but that was more of a separate feature-request.

infernix commented 7 years ago

Is it correct to assume to disable it as soon as possible after the sensor is restarted? Or are you saying it is always setting the initial zero-level at boot?

I am assuming the sensor boot procedure (when U == 15000) does an initial calibration. I think we wouldn't be able to post anything untl after sensor completes bootup anyway, but haven't tested. If it does, then probably not a good idea to mess with it during bootup stage.

By the way, if you want to disable ABC at poweron and test your 400ppm self calibration, enable the rules engine and add this:

On System#Boot do
        mhzabcdisable // you need to test this, not sure if this works directly. may need to prefix with a delay,2000
endon

On MHZ19#PPM=400 do
      timerset,1,300 // Run Rules#Timer=1 in 300 seconds
endon

On Rules#Timer=1 do
     if [MHZ19#PPM]=400 // If still 400
      timerset,2,300 // Run Rules#Timer=2 in 300 seconds
     endif
endon

On Rules#Timer=2 do
     if [MHZ19#PPM]=400 // If still 400
      mhzcalibratezero // calibreate. also untested, not sure if callable directly
     endif
endon

Entirely untested, probably needs fixing :)

TD-er commented 7 years ago

I added the ABC disable setting, but apparently I cannot push the commit. (and thus create a pull request) So just renamed the file to .ino.txt to add it here. (linked at end of this comment)

In short, the default setting is the current behavior. When changing the setting, it will be applied as soon as there is a stable reading.

The setting will be updated on the sensor when:

External commands to set the ABC enable/disable will not affect the stored device settings.

_P049_MHZ19.ino.txt

infernix commented 7 years ago

@TD-er fork this repo to your github account, create branch, add commit to that branch, create PR in github

TD-er commented 7 years ago

@infernix That did the trick :)

Grovkillen commented 6 years ago

This is now merged (#472) and issue can be closed.

psy0rz commented 6 years ago

thanks @Grovkillen

dusjagr commented 6 years ago

great to find this thread! I am currently trying to use the MH-Z19 sensor for building low-cost CO2 soil respiration chambers to be used in agriculutre and urban farming. measuring the emission of CO2 over a short period of time from soil in a small box, which gives an estimate of the microbial activity.

for me it's kinda crucial to be sure the sensor does not re-calibrate or does anything else i don't know of during the course of several experiments.

can someone help me on "how" to disable the ABC, and also on how to check if it's disabled?

also i might want to measure co2 values below 400ppm, as due to a plant cover the chamber might be slowly getting depleted from co2 from the photosynthesis.

i am not using an ESP, but working with a teensy.

some notes here: https://forum.hackteria.org/t/soil-respiration-co2-chamber-measurements/

TD-er commented 6 years ago

If you're using these sensors, please use the "B" version. image The bottom one is the "B" version, which is missing some labels for 2 pins.

Then the ABC disable option is working for sure and the sensor returns an acknowledgement when you set the ABC disabled. You have to make sure to set it again after every reboot of the sensor, so take notice of the sensor data at all time. At reboot, the sensor is outputting the max. ppm value for a few samples. Or you should send the ABC disabled command within intervals less than 24h apart.

The B version can report values < 400 ppm, but I am not sure if the sensor can actually measure below 400 ppm. The A version will never output values < 400 ppm.

The B version also does some filtering based on the stability of the measurements. For the A version you have to either ignore unstable values (S-value != 64, the B version always outputs 0 as S-value) or apply some filtering, which you can see in the sourcecode of the plugin in ESPeasy.

Grovkillen commented 6 years ago

I think the MH-Z19 is hard coded to minimum ppm of 400. They are designed for offices which have a daily purge during nights and weekends. Thus the 24h ABC.

dusjagr commented 6 years ago

hmmm.... hard coded, means it's still hackable for my own use. i just want to make sure i am disabling the ABC, calibrate it myself while outdoors, and then do a series of measurements.

i am sending this comand: byte setABC[9] = {0xFF,0x01,0x79,0x00,0x00,0x00,0x00,0x00,0x86};

but how to read if ABC is disabled?

and i can only set the ABC disabled after the boot of the sensor is complete? so i gotta chech that u thingy until it's not 15000 anymore?

TD-er commented 6 years ago

The datasheet doesn't mention a reply to this command: image

It is best to send the command after the boot is complete and thus indeed check the given values. And just to make sure you're not missing a restart of the sensor, disable it again within 24h.

dusjagr commented 6 years ago

thx. i am trying to detect when the boot is complete by reading all 9 bytes. but i only get this: FF 86 1 8B 3E 0 0 0 0 -> 395 ppm 22 T Celsius

somehow the "s" and "u" values are not being returned....

send command is: byte cmd[9] = {0xFF,0x01,0x86,0x00,0x00,0x00,0x00,0x00,0x79};

i am using "SoftwareSerial mySerial(9, 10); // RX, TX" on my teensy.

TD-er commented 6 years ago

The 'S' and 'U' are not present on the "B" version of the sensor. So congratulations, you have the correct version :)

dio5 commented 5 years ago

Hi,

Sorry to hijack this thread, but can anyone explain what I should think of this? (I have the B version)

When I first started reading values, the first day I got values around 1000-1100ppm indoors constantly, which dropped to around 900 when I left the window open for a while.

The second day it jumped to around 600, presumably because of calibration, where it has been at since then (3-4 days ago), dropping to around 400 at night and going up to 700 when I'm near it (on my desk) and don't ventilate.

But now I don't know what values I should trust, were the initial readings wrong because it had not calibrated yet? Or are these calibrated values just wrong?

Again, sorry to ask it here, but it seemed like the only place where it was discussed.

TD-er commented 5 years ago

You can also ask it on the Forum, or in another issue. But I will read any issue that's being updated, so this should work too ;)

You have to make sure it is used in a room which is ventilated every now and then and depending on the version of the sensor it should be in 400 ppm concentration every 24h of operation. Other issues which may affect the accuracy:

If all the above applies, you can trust the values which are at 400 ppm every day and not suddenly reach that point, but get lower in a curve.

image

Just as an example I took a part of one of the CO2 sensors samples of last week. As you can see, it takes quite a while in a closed room to get to 400 ppm again when there are no people/animals present anymore. (and also that I get to bed quite late sometimes)

HannHank commented 3 years ago

Hey,

is there a easy way to check if the ABC got disabled? I am using a MHZ14a and i am not sure if this command {0xFF,0x01,0x86,0x00,0x00,0x00,0x00,0x00,0x79};will also work for this one. Maybe it will, because the command to read the CO2 value is equally to the MH-Z19, but it would be great if there is a command to check this.

And if I understood it correctly, I have to send this command on every boot up, because the sensor will no save the setting?

I am also thinking about an ABC with a longer cycle, maybe up to one week. Is there a way to send the sensor the new base value?

thx.