search

DigitalShoestringSolutions / AirParticleMonitoring

GNU General Public License v3.0
0 stars 1 forks source link

I2C failure of particle sensor #14

Open tobyaharris opened 3 weeks ago

tobyaharris commented 3 weeks ago

As first mentioned in #13 - a grove air particle sensor suffered an I2C error at 06:49 this morning and I would like to know why.

Logs at 0649 ``` 2024-10-31T06:48:55.510087+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.5, 'mc_10p0': 4.5, 'ambient_rh': 47.37, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:48:55.509125+00:00'} 2024-10-31T06:48:56.486315+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:RAW Values:Mass Concentration PM1.0: 4.1 µg/m^3 2024-10-31T06:48:56.486830+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM2.5: 4.4 µg/m^3 2024-10-31T06:48:56.487096+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM4.0: 4.5 µg/m^3 2024-10-31T06:48:56.487326+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM10.0: 4.5 µg/m^3 2024-10-31T06:48:56.488299+00:00 raspberrypi docker-particle-dc[1230]: Ambient Humidity: 47.38 %RH 2024-10-31T06:48:56.488925+00:00 raspberrypi docker-particle-dc[1230]: Ambient Temperature: 20.50 °C 2024-10-31T06:48:56.489167+00:00 raspberrypi docker-particle-dc[1230]: VOC Index: 69.0 2024-10-31T06:48:56.489394+00:00 raspberrypi docker-particle-dc[1230]: NOx Index: 1.0 2024-10-31T06:48:56.509697+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:Device Status:0x00000000 [OK] 2024-10-31T06:48:56.510516+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.5, 'mc_10p0': 4.5, 'ambient_rh': 47.38, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:48:56.509233+00:00'} 2024-10-31T06:48:57.485400+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:RAW Values:Mass Concentration PM1.0: 4.1 µg/m^3 2024-10-31T06:48:57.485865+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM2.5: 4.4 µg/m^3 2024-10-31T06:48:57.486095+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM4.0: 4.4 µg/m^3 2024-10-31T06:48:57.486339+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM10.0: 4.5 µg/m^3 2024-10-31T06:48:57.486523+00:00 raspberrypi docker-particle-dc[1230]: Ambient Humidity: 47.38 %RH 2024-10-31T06:48:57.486720+00:00 raspberrypi docker-particle-dc[1230]: Ambient Temperature: 20.50 °C 2024-10-31T06:48:57.486911+00:00 raspberrypi docker-particle-dc[1230]: VOC Index: 69.0 2024-10-31T06:48:57.487090+00:00 raspberrypi docker-particle-dc[1230]: NOx Index: 1.0 2024-10-31T06:48:57.509661+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:Device Status:0x00000000 [OK] 2024-10-31T06:48:57.510285+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.38, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:48:57.507987+00:00'} 2024-10-31T06:48:58.487838+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:RAW Values:Mass Concentration PM1.0: 4.1 µg/m^3 2024-10-31T06:48:58.488443+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM2.5: 4.4 µg/m^3 2024-10-31T06:48:58.488908+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM4.0: 4.4 µg/m^3 2024-10-31T06:48:58.489128+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM10.0: 4.5 µg/m^3 2024-10-31T06:48:58.489357+00:00 raspberrypi docker-particle-dc[1230]: Ambient Humidity: 47.38 %RH 2024-10-31T06:48:58.489744+00:00 raspberrypi docker-particle-dc[1230]: Ambient Temperature: 20.50 °C 2024-10-31T06:48:58.490212+00:00 raspberrypi docker-particle-dc[1230]: VOC Index: 69.0 2024-10-31T06:48:58.490563+00:00 raspberrypi docker-particle-dc[1230]: NOx Index: 1.0 2024-10-31T06:48:58.512301+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:Device Status:0x00000000 [OK] 2024-10-31T06:48:58.512847+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.38, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:48:58.510785+00:00'} 2024-10-31T06:48:59.486689+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:RAW Values:Mass Concentration PM1.0: 4.1 µg/m^3 2024-10-31T06:48:59.487159+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM2.5: 4.4 µg/m^3 2024-10-31T06:48:59.487403+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM4.0: 4.4 µg/m^3 2024-10-31T06:48:59.487603+00:00 raspberrypi docker-particle-dc[1230]: Mass Concentration PM10.0: 4.5 µg/m^3 2024-10-31T06:48:59.487958+00:00 raspberrypi docker-particle-dc[1230]: Ambient Humidity: 47.37 %RH 2024-10-31T06:48:59.488240+00:00 raspberrypi docker-particle-dc[1230]: Ambient Temperature: 20.50 °C 2024-10-31T06:48:59.488576+00:00 raspberrypi docker-particle-dc[1230]: VOC Index: 69.0 2024-10-31T06:48:59.488927+00:00 raspberrypi docker-particle-dc[1230]: NOx Index: 1.0 2024-10-31T06:48:59.510015+00:00 raspberrypi docker-particle-dc[1230]: INFO:main:Device Status:0x00000000 [OK] 2024-10-31T06:48:59.510853+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.37, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:48:59.509145+00:00'} 2024-10-31T06:49:00.480646+00:00 raspberrypi docker-particle-dc[1230]: ERROR:main.measure:Sampling lead to exceptionI2C transceive failed: [Errno 5] Input/output error \ 2024-10-31T06:49:00.481144+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.37, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:49:00.479209+00:00'} 2024-10-31T06:49:01.457550+00:00 raspberrypi docker-particle-dc[1230]: ERROR:main.measure:Sampling lead to exceptionI2C transceive failed: [Errno 5] Input/output error \ 2024-10-31T06:49:01.458172+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.37, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:49:01.456774+00:00'} 2024-10-31T06:49:02.457363+00:00 raspberrypi docker-particle-dc[1230]: ERROR:main.measure:Sampling lead to exceptionI2C transceive failed: [Errno 5] Input/output error \ 2024-10-31T06:49:02.458247+00:00 raspberrypi docker-particle-dc[1230]: INFO:main.measure:dispatch to of {'mc_1p0': 4.1, 'mc_2p5': 4.4, 'mc_4p0': 4.4, 'mc_10p0': 4.5, 'ambient_rh': 47.37, 'ambient_t': 20.5, 'voc_index': 69.0, 'nox_index': 1.0, 'machine': 'Machine_1', 'timestamp': '2024-10-31T06:49:02.456838+00:00'} 2024-10-31T06:49:03.458249+00:00 raspberrypi docker-particle-dc[1230]: ERROR:main.measure:Sampling lead to exceptionI2C transceive failed: [Errno 5] Input/output error \ ```

The only detail these logs give is 

ERROR:main.measure:Sampling lead to exceptionI2C transceive failed: [Errno 5] Input/output error \

which is not much to go on.

According to i2cdetect, the sensor is no longer visible at 0x69 as the datasheet would expect: image

The pi's clock seems to have been correct all morning and I doubt anyone was here before 7am. There is no sign of damage nor loose plugs.

Stopping the solution and rebooting does not make the sensor reappear in i2cdetect.

At least the logs have got more interesting:

Logs after reboot ``` 2024-10-31T12:18:44.027966+00:00 raspberrypi docker-particle-dc[1222]: INFO:main:config:{'constants': {'machine': 'Machine_1'}, 'adc': {'adc_module': 'sen5x'}, 'sampling': {'sample_interval': 1}, 'computing': {'hardware': 'Pi4'}, 'mqtt': {'broker': 'mqtt.docker.local', 'port': 1883, 'base_topic_template': 'airparticle_monitoring/{{machine}}', 'reconnect': {'initial': 5, 'backoff': 2, 'limit': 60}}} 2024-10-31T12:18:44.140502+00:00 raspberrypi docker-particle-dc[1222]: INFO:main.measure:started 2024-10-31T12:18:44.209544+00:00 raspberrypi docker-particle-dc[1222]: INFO:main.wrapper:connecting to mqtt.docker.local:1883 2024-10-31T12:18:44.483393+00:00 raspberrypi docker-particle-dc[1222]: INFO:main.wrapper:Connected! 2024-10-31T12:18:44.553045+00:00 raspberrypi docker-particle-dc[1222]: INFO:main:init sen5x 2024-10-31T12:18:44.573886+00:00 raspberrypi docker-particle-dc[1222]: Process AirParticleMeasureBuildingBlock-1: 2024-10-31T12:18:44.673804+00:00 raspberrypi docker-particle-dc[1222]: Traceback (most recent call last): 2024-10-31T12:18:44.700584+00:00 raspberrypi docker-particle-dc[1222]: File "/usr/local/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap 2024-10-31T12:18:44.700833+00:00 raspberrypi docker-particle-dc[1222]: self.run() 2024-10-31T12:18:44.700951+00:00 raspberrypi docker-particle-dc[1222]: ~~~~~~~~^^ 2024-10-31T12:18:44.701256+00:00 raspberrypi docker-particle-dc[1222]: File "/app/measure.py", line 88, in run 2024-10-31T12:18:44.701466+00:00 raspberrypi docker-particle-dc[1222]: adc = adc_module.ADC(self.config) 2024-10-31T12:18:44.701575+00:00 raspberrypi docker-particle-dc[1222]: File "/app/adc/sen5x.py", line 16, in __init__ 2024-10-31T12:18:44.701664+00:00 raspberrypi docker-particle-dc[1222]: logger.info(f"Sensor Version:{self.device.get_version()}") 2024-10-31T12:18:44.701755+00:00 raspberrypi docker-particle-dc[1222]: ~~~~~~~~~~~~~~~~~~~~~~~^^ 2024-10-31T12:18:44.701865+00:00 raspberrypi docker-particle-dc[1222]: File "/usr/local/lib/python3.13/site-packages/sensirion_i2c_sen5x/device.py", line 94, in get_version 2024-10-31T12:18:44.701949+00:00 raspberrypi docker-particle-dc[1222]: return self.execute(Sen5xI2cCmdGetVersion()) 2024-10-31T12:18:44.702048+00:00 raspberrypi docker-particle-dc[1222]: ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-10-31T12:18:44.702141+00:00 raspberrypi docker-particle-dc[1222]: File "/usr/local/lib/python3.13/site-packages/sensirion_i2c_driver/device.py", line 61, in execute 2024-10-31T12:18:44.702249+00:00 raspberrypi docker-particle-dc[1222]: return self._connection.execute(self._slave_address, command) 2024-10-31T12:18:44.702350+00:00 raspberrypi docker-particle-dc[1222]: ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-10-31T12:18:44.702447+00:00 raspberrypi docker-particle-dc[1222]: File "/usr/local/lib/python3.13/site-packages/sensirion_i2c_driver/connection.py", line 116, in execute 2024-10-31T12:18:44.702564+00:00 raspberrypi docker-particle-dc[1222]: return self._interpret_response(command, response) 2024-10-31T12:18:44.702645+00:00 raspberrypi docker-particle-dc[1222]: ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ 2024-10-31T12:18:44.702749+00:00 raspberrypi docker-particle-dc[1222]: File "/usr/local/lib/python3.13/site-packages/sensirion_i2c_driver/connection.py", line 205, in _interpret_response 2024-10-31T12:18:44.702838+00:00 raspberrypi docker-particle-dc[1222]: raise response 2024-10-31T12:18:44.702924+00:00 raspberrypi docker-particle-dc[1222]: sensirion_i2c_driver.errors.I2cTransceiveError: I2C transceive failed: [Errno 5] Input/output error ```

Disconnecting and reconnecting the sensor does and the solution runs happily. But what went wrong?

Is there a conflict between sensirion_i2c_driver (no smbus!) and i22cdetect that causes them to break each other?

Does the pi have any i2c logs for further interrogation?

Greg-Hawkridge commented 3 weeks ago

see #13