search

kizniche / Mycodo

An environmental monitoring and regulation system
http://kylegabriel.com/projects/
GNU General Public License v3.0
2.96k stars 494 forks source link

MH-Z16-uart sensor command string fails "unicode strings are not supported" v5.5 #359

Closed Theoi-Meteoroi closed 6 years ago

Theoi-Meteoroi commented 6 years ago

Mycodo Issue Report:

Problem Description

MH-Z16 uart sensor command string contains unicode characters making the sensor framework fail.

Errors

2017-12-25 17:07:43,143 - mycodo.daemon - INFO - Mycodo daemon v5.5.0 starting 2017-12-25 17:07:44,016 - mycodo.daemon - INFO - Anonymous statistics enabled 2017-12-25 17:07:44,086 - mycodo.daemon - INFO - Starting rpyc server 2017-12-25 17:07:44,823 - mycodo.output - INFO - Output controller activated in 238.9 ms 2017-12-25 17:07:45,325 - mycodo.daemon - INFO - All activated Timer controllers started 2017-12-25 17:07:45,327 - mycodo.daemon - INFO - All activated Input controllers started 2017-12-25 17:07:45,327 - mycodo.daemon - INFO - All activated Math controllers started 2017-12-25 17:07:45,328 - mycodo.daemon - INFO - All activated PID controllers started 2017-12-25 17:07:45,329 - mycodo.daemon - INFO - All activated LCD controllers started 2017-12-25 17:07:47,169 - mycodo.conditional - INFO - Conditional settings refreshed 2017-12-25 17:07:47,170 - mycodo.conditional - INFO - Conditional controller activated in 1338.9 ms 2017-12-25 17:07:47,171 - mycodo.daemon - INFO - Mycodo daemon v5.5.0 started in 4.005 seconds 2017-12-25 17:07:47,178 - mycodo.daemon - INFO - 38.72 MB RAM in use 2017-12-25 17:09:43,384 - mycodo.daemon - INFO - Regenerating stats file 2017-12-25 17:19:45,317 - mycodo.input_1 - INFO - Activated in 121.8 ms 2017-12-25 17:19:46,320 - mycodo.inputs.mh_z16.devttyAMA0 - ERROR - MHZ16Sensor raised an exception when taking a re ading: unicode strings are not supported, please encode to bytes: 'ÿ\x01\x86\x00\x00\x00\x00\x00y' 2017-12-25 17:20:01,353 - mycodo.inputs.mh_z16.devttyAMA0 - ERROR - MHZ16Sensor raised an exception when taking a re ading: unicode strings are not supported, please encode to bytes: 'ÿ\x01\x86\x00\x00\x00\x00\x00y' 2017-12-25 17:20:09,873 - mycodo.input_1 - INFO - Deactivated in 36.2 ms

Steps to Reproduce the issue:

How can this issue be reproduced?

  1. Added the MH-Z16_uart sensor to /dev/ttyAMA0
  2. Configure as sensor input
  3. Activate sensor

Additional Notes

Is there anything that should be added to make it easier to address this issue?

Theoi-Meteoroi commented 6 years ago

MH-Z19 UART CO2 sensor also has this problem. I've tried a hack to fix the command, but something else is broken with readings.

2017-12-25 18:12:03,816 - mycodo.inputs.mhz19.devttyUSB0 - ERROR - MHZ19Sensor raised an exception when taking a reading: unicode strings are not supported, please encode to bytes: 'ÿ\x01\x86\x00\x00\x00\x00\x00y' 2017-12-25 18:12:18,847 - mycodo.inputs.mhz19.devttyUSB0 - ERROR - MHZ19Sensor raised an exception when taking a reading: unicode strings are not supported, please encode to bytes: 'ÿ\x01\x86\x00\x00\x00\x00\x00y'

Changed line - inputs/mh_z19.py

90 self.ser.write(("\xff\x01\x86\x00\x00\x00\x00\x00\x79".encode()))

This fixed the unicode error for both MH-Z16 and MH-Z19 however no readings:

2017-12-25 18:14:47,755 - mycodo.input_4 - ERROR - StopIteration raised. Possibly could not read input. Ensure it's connected properly and detected.

kizniche commented 6 years ago

Change the mh_z19.py around Line 90 to look like this, then activate the sensor and copy the output that goes to the log, please:

            self.ser.write("\xff\x01\x86\x00\x00\x00\x00\x00\x79".encode())
            time.sleep(.01)
            resp = self.ser.read(9)
            self.logger.info("TEST: {}".format(resp))

Let's see what the sensor is actually returning, if anything, then see if we can interpret it.

kizniche commented 6 years ago

Also, just to be sure, have you enabled Serial with raspi-config?

Theoi-Meteoroi commented 6 years ago

I'm monitoring the ttyAMA0 with a Saleae analyzer, so yes it is enabled. I checked with a USB MH-Z19 (pretty much the same code) and it fails in the app the same way.

When I look at what gets sent, looks like some values aren't correct. I've written C drivers for this sensor so I'm familiar with the quirks. At some point we need to add a calibration function for these sensors. I've heard that they self-calibrate unless they are power-cycled every 24 hours, which is not really a good idea if you have high constant levels. I'll have to verify that rumor with the manufacturer on that point when I contact them for another issue with some probes I obtained directly from Winsen.

I'll figure this defect out and suggest a code change.

kizniche commented 6 years ago

Thanks for the help. I don't have either of these sensors, and you appear more knowledgeable about them.

In the meantime, if you need the sensors to work with Mycodo, you can always use the "Linux Command" input to execute a script that returns the measurement value. Set the measurement to "co2" for it to integrate with the default CO2 graph y-axis.

Theoi-Meteoroi commented 6 years ago

We seem to be writing 0xc3 0xbf 0x01 0xc2 0x86 0x0 0x0 0x0 0x0 0x0 0x79

Not sure why. Some gets properly encoded. Strings are a PITA.

I'm really into sensors of most all types. Once I started using these CO2 sensors for more practical purposes I researched them extensively. I contacted the company to get the MH-Z19 sensors since they are quite a bit cheaper than the MH-Z16. I also am working on some particle detectors that I would like very much to add to Mycodo. They detect PPM 1.0, 2.5 and 10. I would imaging some folks could use them to detect spawn. I've been using Mycodo to monitor air quality and after some fires here thought PPM monitoring could be useful for humans as well.

pm me an address and I'll send some sensors. I usually obtain at least two, if any.

kizniche commented 6 years ago

Strange. I've been reading about serial and haven't come across incorrectly sent information.

Just out of curiosity, does this produce the same result?

self.ser.write(bytearray([0xff, 0x01, 0x86, 0x00, 0x00, 0x00, 0x00, 0x00, 0x79]))
Theoi-Meteoroi commented 6 years ago

M'kay - that yielded a correct command string. So we got a response from the sensor on serial but the driver says:

2017-12-25 20:55:02,134 - mycodo.inputs.mh_z16.devttyAMA0 - INFO - TEST: b'\xff\x86\x03\x16D\x00\x00\x00\x1d' 2017-12-25 20:55:02,135 - mycodo.inputs.mh_z16.devttyAMA0 - ERROR - MHZ16Sensor raised an exception when taking a reading: a bytes-like object is required, not 'int'

This is part of what I looked forward to and feared from Py3. Its gonna be porting types.

kizniche commented 6 years ago

Progress! I'm off to sleep, so I can't help any more tonight. I'll return to this tomorrow if I get some free time from work and my holiday guests.

Theoi-Meteoroi commented 6 years ago

Have a good holiday!

So I applied this fix to MH-Z19 and it progressed similarly - to a type error that made more sense to me: 2017-12-25 21:08:02,752 - mycodo.inputs.mhz19.devttyUSB0 - ERROR - MHZ19Sensor raised an exception when taking a reading: ord() expected string of length 1, but int found

kizniche commented 6 years ago

Thanks. You as well. I'm supposed to have all this week off, but bad shipping organization meant I wasn't able to get some fungal cultures delivered to my lab by Thursday last week and I've been trying to locate and ensure they're taken care of (refrigerated). Unfortunately, it appears they were sitting in a parked delivery truck for the past 4 days.

Theoi-Meteoroi commented 6 years ago

Uhg. That would be a unique farming story for many. We live in a far more tenuous food web than many humans assume. I hope there was at least some salvage for your troubles.