[FEATURE]: CO2 CCS811 support?

[bggangel]

I am thinking about adding a CO2 sensor to my environment. I saw that you already have support for a COZIR sensor, but that is a little pricy. I also saw support for a K30 sensor, but I am not able to find any sensor (in the US) for that. I did find a lot of CCS811 CO2 sensors on amazon, for example from KEYESTUDIO for $15.

Is that sensor already supported or is there a way to already read it or add support for it?

[theyosh]

Hi, I created a new branch where you could test this sensor. In order to get the right branch, remove the existing installation to a temp dir. And clone again my software, but then with the command: git clone --recursive https://github.com/theyosh/TerrariumPI.git and then enter the folder TerrariumPI and run git checkout CCS811 and restart it.

This should add the new sensor for testing. As I saw it is using I2C, you have to enter the I2C address without '0x' in front of it. https://github.com/theyosh/TerrariumPI/wiki/Hardware#i2c-bus

[bggangel]

perfect. I will order the sensor and will give testing feedback once received and hooked up!

[bggangel]

So I received my sensor and tried out the code. There were some problems with the code. I have 0% experience with Python scripting code or the way the pi os works, but I do know .NET and some C code, so I tried to fix the problems. I was able to get the correct reading out of the sensor in TerriPi. I have attached the changed file (not sure how to commit those changes in git, but if you prefer that and give me the instructions, I will do that). Potentially I did something wrong or against style, but this gives a reading so maybe you can use that as a guide.

terrariumI2CSensor.zip

That leaves 2 remaining problems, potentially related. When starting up, there is the following message: /home/pi/TerrariumPI/CCS811_RPi.py:43: ResourceWarning: unclosed file <_io.FileIO name='/dev/i2c-1' mode='rb' closefd=True> CCS811_fr= io.open("/dev/i2c-"+str(twi), "rb", buffering=0) ResourceWarning: Enable tracemalloc to get the object allocation traceback /home/pi/TerrariumPI/CCS811_RPi.py:44: ResourceWarning: unclosed file <_io.FileIO name='/dev/i2c-1' mode='wb' closefd=True> CCS811_fw= io.open("/dev/i2c-"+str(twi), "wb", buffering=0) ResourceWarning: Enable tracemalloc to get the object allocation traceback

When restarting, by pressing Ctrl-C in screen -r 2020-10-10 20:03:49,406 - WARNING - terrariumI2CSensor - Error closing css811 sensor 'CO2 seonsor'. Error message: [Errno 9] Bad file descriptor

I am not sure how to tackle those two problems, and if they need to be tackled. If you need me to generate more or other debug info, let me know. So far, so good.

[theyosh]

Thanks for the zip file with code changes. It is always helping, so I have made some changes in my code. If you revert you changes back and do a git pull, you should get the new code.

I have only changed the measurement value to once every 10 sec '0b100000' as I only take measurements every 30 seconds.

The remaining errors of unclosed files is due to the poor programming in the copied CCS811_RPi.py file. It is using global variables to read the sensor... this is bad practise, but it will take more time to make it myself. For now, just ignore it.

Also the close error, is not nice, but not that bad.

But, there is another library available, but that means that I also have to change my code again. The most examples are made for Python2 (still)

[bggangel]

I will update this afternoon. I did experiment with different modes. The sensor seems to return a lot of Invalid readings (which apparently is a known issue in combination with a pi) and 0b100000 returned a little more false values. However, I did run it with your mode (my code) for a while this morning and that seems to work well enough also. It is not that the CO2 values are very volatile anyway. And perhaps the sensor has burned in better now. (which also is a thing with these sensors). So I will update you this afternoon.

In regards to the other library: I am doing all this on a separate test system, without any time pressure (it's just an extra), so if you want me to test the other library, after we confirmed the workings of the current branch, I can test that for you. Or I could even try to change/update the code to use the new lib as long as you point me to that new lib. FYI: The sensor that I ended up buying way the Adafruit CCS881. A little more expensive, but hopefully better quality. I did see there is a AdaFruit CCS881lib... is that the other lib you mean?

[bggangel]

So I tested with the reverted and updated code from git. I had experimented with the modes before and it seemed that the 1-second mode, was the mode that produced the least amount of false or erroneous readings. I had tried the 10-second mode and it quadrupled the failed readings. I thought it had gotten better when I changed my code to the 10-second mode, but after using the 10-second mode in TerriPi, the readings are very bad again. Switching the code back to the 1-second mode makes the readings improve significantly instantly.

Of course, this could be a faulty sensor on my side, but I would advise using the 1-second mode as it seems a lot better.

I did do some experimenting with a different lib. Like I said before, I have the AdaFruit version. So I experimented with separate programs.
I am not sure if it is because it is a AdaFruit on a AdaFruit sensor, but the attached zip has a test.py in it (and the required libs) and this produces correct readings about 90% of the times, which is by far the best results I have seen before. I tried to integrate this code into the TerriPi code but was not successful.

Maybe the attached code helps you out.

Ada.zip

[theyosh]

Sorry for the late reply. I will look into this this weekend

[theyosh]

Ok, I made a rewrite. Not so happy with Adafruit. They have a lot of hardcoded port numbers in their code. But if this works, it is a solution.

So, I guess you have already installed the libraries with pip. If not: sudo pip3 install adafruit-circuitpython-ccs811

Then do a git pull and restart. I hope it will work then.

I have changed the code, because your example did a check every 10 sec. But I get the feeling that this will also add 10 sec wait before sensor is ready. And that is not handy. So I set it to 1 sec measurements. As I am not able to keep the sensor open all the time.

Let's hope this works.

[theyosh]

Any updates? Did you had time to test this?

[bggangel]

Sorry, not this week. It has been a crazy week in the US but all is well now. Will probable look at it this week or weekend

On Sun, Nov 8, 2020, 12:13 TheYOSH notifications@github.com wrote:

Any updates? Did you had time to test this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/theyosh/TerrariumPI/issues/462#issuecomment-723646045, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ2KX26TCMRNYW2ME7C3JTSO3NTZANCNFSM4SB5RCGA .

[theyosh]

Hi, any updates? I would like to make a release, and would be nice to know that this is working

[bggangel]

Sorry, this got put on a back burner. I will pick this up soon again.

On Wed, Dec 9, 2020 at 2:16 PM TheYOSH notifications@github.com wrote:

Hi, any updates? I would like to make a release, and would be nice to know that this is working

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/theyosh/TerrariumPI/issues/462#issuecomment-742022478, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ2KX6ZLOZSOWRWBTYZRLLST7LITANCNFSM4SB5RCGA .

-- Bernardus Gerben Gangel

[bggangel]

So I made a copy of the previous code, did the update (and did the sudo pip3 install, just to be sure), and rebooted. But now, the sensor just returns "None". From "terrariumpi.log" : Measured co2 value Noneppm from css811 sensor 'CO2 seonsor' is outside valid range 1.00ppm - 10000.00ppm in 0.30608 seconds.

So I put the previous code back, rebooted, and the sensor returns values again.

I then figured, I'd start with a clean folder, did a fresh pull from the info in this thread, git checkout CCS811, copied my .cfg files, and restart it. The results are still the same. Nothing gets read. I then rebooted and verified that the test.py that I send before works (stop TerrariumPI, run test.py, get valid readings, start TerrariumPI). The values are still wrong in TerrariumPI.

Even though I did not change my settings, just letting you know that my address in the settings is 5a.

I didn't have time to do any more research, but let me know what info you need from me for further analyses.

[escomputers]

Any update? I got same issue

[theyosh]

It is hard to debug without the hardware. So for both, I updated the ccs811 branch with more debug. If you can both run it on this branch, and make some screenshots from the debug screen session, that would help a lot. https://github.com/theyosh/TerrariumPI/wiki/FAQ#how-to-debug-terrariumpi

[escomputers]

thanks, later today I will test it out. Just two question please 1) when adding a i2c sensor like this CCS811 what kind of hardware address I should use?Custom script or else? 2) I also tried to add it as remote sensor connected to a remote raspberry (e.g. http://1.1.1.1:8000/co2.json) and tried to use TerrariumPI as webserver, putting co2.json into folder /TerrariumPI/static/extern/. JSON file contains just this line {"co2": 409} and into sensor setting I put http://127.0.0.1:8090/co2.json. In both cases I got no success. Log says "None ppm, value outside valid range" As last option I could send to you a brand new CCS811 I bought by mistake as duplicate (obviously for free ;)

[theyosh]

1. Address does not matter. But I suggest to enter the I2C number according to: https://github.com/theyosh/TerrariumPI/wiki/Hardware#i2c-bus . The Adafruit drivers are fixed on the address of the sensor. So any address will do to make the form submit.

2. You have missed the part after '#' https://github.com/theyosh/TerrariumPI/wiki/Remote-data#url-format . So your url will be http://127.0.0.1:8090/co2.json#co2 <- It needs to know witch value you want.

[escomputers]

Sorry, I wrote a wrong word in my first question on last post: when adding a i2c sensor like this CCS811 what kind of hardware address I should use?Custom script or else? I meant what kind of hardware should I set?Now is set as "custom script" and in address field I set i2c address according to wiki

[theyosh]

Ah, you are not on the right branch. Look at https://github.com/theyosh/TerrariumPI/issues/462#issuecomment-703525028

Here is described how to get extra branches and switch to them. After switching to the new branch, restart and clear browser cache. You should have a new hardware type called ccs811

[escomputers]

It seems to be a clock speed error. Adafruit suggests slowing down i2c clock speed setting, by inserting dtparam=i2c_baudrate=10000 into /boot/config.txt file CCS811 works fine, but after adding it to TerrariumPI it always gives this error:

  File "/root/terrariumI2CSensor.py", line 6, in <module>
    ccs811 = adafruit_ccs811.CCS811(i2c)
  File "/usr/local/lib/python3.7/dist-packages/adafruit_ccs811.py", line 121, in __init__
    "Device returned a error! Try removing and reapplying power to "
RuntimeError: Device returned a error! Try removing and reapplying power to the device and running the code again.

I obviously already tried to disconnect and reconnect power to the sensor without success. It's not the first time that someone has issues with this sensor, read this topic from adafruit forum: . The guy solved by replacing it with a bosch sensor. Below my TerrariumPI settings, hope this helps Here adafruit library code that throws the error:

    def __init__(self, i2c_bus, address=0x5A):
        self.i2c_device = I2CDevice(i2c_bus, address)

        # check that the HW id is correct
        if self.hw_id != _HW_ID_CODE:
            raise RuntimeError(
                "Device ID returned is not correct! Please check your wiring."
            )
        # try to start the app
        buf = bytearray(1)
        buf[0] = 0xF4
        with self.i2c_device as i2c:
            i2c.write(buf, end=1)
        time.sleep(0.1)

        # make sure there are no errors and we have entered application mode
        if self.error:
            raise RuntimeError(
                "Device returned a error! Try removing and reapplying power to "
                "the device and running the code again."
            )
        if not self.fw_mode:
            raise RuntimeError(
                "Device did not enter application mode! If you got here, there may "
                "be a problem with the firmware on your sensor."
            )

        self.interrupt_enabled = False

[escomputers]

I resetted the CCS811 by connecting RST pin to raspi GND pin and in a first moment, error disappered when running external code. When I added it again to terrariumpi error comes alive. I also noted sensor first values within initial seconds are always equal to 0. After 5-6 seconds it starts giving 400ppm values. Below terrariumpi log for debug.

[theyosh]

Hmm.. this is getting difficult. An issue could be that the Adafruit library is not 100% thread safe. Because I use Gevent, which will patch certain part of the python code.

A solution could be, that this sensor will only work using an external script. So when you have your own script, that reads the value and print it to screen, you can then use that as a script sensor. If that works, maybe that is easier to do.

Else, I have done some debugging remote. If you are handy enough to create port forwarding for ssh to your pi, and you trust me enough, you could sent me the ssh login to: github@theyosh.nl

[escomputers]

No problem, I sent you an email containing my pi ssh connection details, I am glad to help improving this wonderful software. As you said, it works well as remote HTTP sensor, I created a simple python3 script which pull the data from the CCS811 every 15 seconds (timing could be set to any value, I put 15sec because I read TerrariumPI pulls data every 30sec) and then dumping data to co2.json file into /TerrariumPI/static/extern folder and setting a every minute cronjob. Below co2.py and some screenshots. I hope I could help. (p.s. sorry if python scripting is not good enough, I am a rookie)

#!/usr/bin/python3

import time
import board
import busio
import adafruit_ccs811
import json

i2c = busio.I2C(board.SCL, board.SDA)
ccs811 = adafruit_ccs811.CCS811(i2c)

# Wait for the sensor to be ready
while not ccs811.data_ready:
    pass

t_end = time.time() + 15
while time.time() < t_end:
    data = {
        #"alarm_min": 20.0, 
        #"address": "26C04A8F01000042", 
        #"id": "f745feb874df5673e157ef8c69e586b4", 
        #"limit_max": 45.0, 
        #"alarm_max": 30.0, 
        #"name": "Top", 
        #"alarm": false, 
        #"hardwaretype": "owfs", 
        "co2": ccs811.eco2 
        #"limit_min": 10.0, 
        #"type": "temperature"
    }
with open('/TerrariumPI/static/extern/co2.json', 'w') as f:
        json.dump(data, f)

[escomputers]

After some testing I noted sensor gets stuck when polling data from it too quckly. So I extended collecting data period to 30 seconds then cronjob every 1/30 minute to avoid this problem. It seems to work well as remote sensor

[bggangel]

Not sure where we are in the debug process, but I ran it with the new debugging software: First got:

---

2020-12-20 16:19:39,913 - WARNING - terrariumSensor - Measured co2 value Noneppm from css811 sensor 'CO2 seonsor' is outside valid range 1.00ppm - 10000.00ppm in 0.30654 seconds. Loaded I2C <busio.I2C object at 0xa32400f0> terrariumCCS811Sensor Ex: Device returned a error! Try removing and reapplying power to the device and running the code again. Returning data None

---

So I did remove the power and got a lot of

---

Sensor not ready...... Sensor not ready...... Sensor not ready......

---

After adding power, I got the same messages as above (Device returned a error! Try removing and reapplying power to the device and running the code again. ).

One more note: I seem to remember that the sensor/software worked before we switched to the adata fruit lib. I don't remember if there were down sides to that version because it was before I had to take a break on this, but I think it worked good. I still have a copy of that enviroment and just restored and ran it, and right away: 2020-12-20 16:30:19,337 - INFO - terrariumSensor - Updated css811 sensor 'CO2 seonsor' co2 from 0.00ppm to 400.00ppm in 0.47016 seconds 2020-12-20 16:30:19,454 - INFO - terrariumEngine - Update done in 0.61344 seconds. Waiting for 29.38656 seconds for next update 2020-12-20 16:30:33,884 - INFO - terrariumEnvironment - Update done in 0.00144 seconds. Waiting for 14.99856 seconds for next update 2020-12-20 16:30:48,832 - INFO - terrariumAudio - Update done in 0.00001 seconds. Waiting for 29.99999 seconds for next update 2020-12-20 16:30:48,883 - INFO - terrariumEngine - Webcam update(s) done in 0.00001 seconds. Waiting for 29.99999 seconds for next update 2020-12-20 16:30:48,886 - INFO - terrariumEnvironment - Update done in 0.00131 seconds. Waiting for 14.99869 seconds for next update 2020-12-20 16:30:49,352 - INFO - terrariumSensor - Updated css811 sensor 'CO2 seonsor' co2 from 400.00ppm to 400.00ppm in 0.47029 seconds 2020-12-20 16:30:49,467 - INFO - terrariumEngine - Update done in 0.59907 seconds. Waiting for 29.40093 seconds for next update

So that worked and has been working untill we switched to adat fruit, probably because we thought they had better software/drivers.... but maybe not ? I would be fine running that previous version (unless there are downsides)... If you need a copy of certain files fron that working enviroment, tell me which onces you want and I can zip/send them...

[theyosh]

Ok, after some testing, I get the feeling that this is a very slow sensor. It takes some time to get values using the Adafruit libraries. Also, it always starts at 400, which is suspicious, as this is a clean room value. And after a while it goes up. When you then restart, it starts al over again at the value of 400. So, I am not sure, but that does not sound right.

@bggangel Could you sent me the terrariumI2CSensor.py file? You have a version whithout the Adafruit libraries?

[escomputers]

adafruit said it always starts from 400ppm as minimum value. It is so slow that I have to increase cronjob time for python script to every 4 minutes, if you shorten this you will always get the sensor 'freezed'. Now I am using it as normal remote sensor and it seems to work well after 48hours.

[theyosh]

@escomputers Hmm, I have disabled that cronjob, so you have old data ;) Sorry

[escomputers]

yes I saw it, when you commented out it was every 3 minutes and it does not work well, sensor get stuck. Last night I edited it again and now works fine

[bggangel]

regarding terrariumI2CSensor.py -> Attached

regarding slow start: The first 10 readings ussually are out of range, so at least it is a slow starter. Once it sends readings, it should be ok

regarding 400: Yes, this is the minimum it reports. But it does register values. When breathing/blowing on the sensor values read like: 2020-12-21 18:33:07,739 - INFO - terrariumSensor - Updated css811 sensor 'CO2 seonsor' co2 from 402.00ppm to 549.00ppm in 0.47051 seconds

regarding slow readings: I have attached a script test.py and the the output of the script. When ignoring the readings in the start (this is normal) and the readings that are out of the range (there are a lot but that is also normal), the other readings are inline with me breating on it and then not anymore. So with a pause of 2 seconds, it updates. Not sure if that is considered fast or not, but that is it. test2.py was run in /home/pi/TerrariumPI

terriPi.zip

[theyosh]

Ok, after some debugging and testing, I think I got it working. We are dropping the Adafruit library, as it give me error that the sensor is not connected well, and needs a reset. While my other code will just read it :+1:

So, for @escomputers your version is up to date and should work. For @bggangel please do a git pull and restart. Then you can add the sensor to the system.

Please take care: The first reading will always be zero. It needs 10 secs for the first measurement, and we are not waiting on that. This happens after a restart, adding a new sensor, or updating an existing sensor.

[bggangel]

Just updated and it has been reading correctly for the last 15 minutes.

I will close the issue and thank you for all the hard work you are doing on this. Glad we got it working!

Hartelijk bedankt en beste wensen voor 2021!

[theyosh]

Euh, you are still on the special branch. So I just merged it to the master branch. You should switch back, else you will miss updates.

[escomputers]

is it possibile to switch back to master without recloning the repo?

[theyosh]

Yes, just run git checkout master and then git pull. Then you should restart TerrariumPI and it should run on the master branch

[escomputers]

Ok thanks I don't know why but after you logged out this afternoon it started giving crazy values. Like 8000ppm 1200ppm 600ppm I dont'think that co2 quantity could be into a 5 square meter closed room (even in few hours) I can open again vpn or ssh port if you want. In my opinion it should be run as remote sensor

[theyosh]

Hmm,

you could try to restart it. But if the values keep rising, I am afraid that the only solution is through a remote sensor.

Because, I use gevent https://pypi.org/project/gevent/, and a lot of threading. So if the sensor library is not thread safe, it could happen that we get strange errors. I cannot do much about that.

So, try a reboot/restart. If that does solve it, I am afraid that I can't do much more. Then go back to your remote solution.

But, totally different, I think using a different CO2 sensor. I use https://thingpulse.com/product/mh-z19-co2-sensor/. And I do not have issues. Also, you have a DHTXX which is known not to be stable. Use a BME280 instead. That works better.

@bggangel Do you also strange values??

[escomputers]

ok I've just ordered those temp and co2 sensor you suggested

[bggangel]

my values are stable as a rock (with variances so we know it is now a constant or a minimum)

here is a link of my sensor https://www.amazon.com/gp/product/B076PR2WKW/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

[escomputers]

my values are stable as a rock (with variances so we know it is now a constant or a minimum)

here is a link of my sensor https://www.amazon.com/gp/product/B076PR2WKW/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Maybe you should test it at leat 48hours before saying it works like a rock

[wolli1234]

Hello,

I use the BME280 for temperature and humidity and the CCS811 for CO2. When humidity and temperature are known by the CCS811, it is able to refine it's tVOC and CO2 readings.

Is this taken into account in the current code?

`def setCompensation(self, temperature, humidity): temperature = round(temperature,2) humidity = round(humidity,2) print('Setting compensation to {} C and {} %'.format(temperature, humidity)) hum1 = int(humidity//0.5) hum2 = int(humidity512-hum1256)

            temperature = temperature+25
            temp1 = int(temperature//0.5)
            temp2 = int(temperature*512-temp1*256)

            s = [CSS811_ENV_DATA,hum1,hum2,temp1,temp2,0x00]
            s2 = bytearray( s )
            CCS811_fw.write( s2 )

            return`

Or is this something else?

Greetings, Wolli