josdo / smesh

3 stars 2 forks source link

Logging incomplete packets #3

Open danineamati opened 4 weeks ago

danineamati commented 4 weeks ago

When a sensor (e.g., the BME) does not send a complete packet, the logger in read_aqi.py does not log any of the packet to CSV. Instead, we should log the incomplete information and note the missing values as nan.

The relevant lines are below where metrics may be missing keys. The same holds for powerMetrics and deviceMetrics. https://github.com/josdo/smesh/blob/dccdf23f8f1b03efeb273a56db9a9385c982125c/snode/scripts/read_aqi.py#L42-L55

Example in the field of streamed packets that are not logged to file due to missing iaq data in the BME.

From node: 0x336544fc
Packet from !336544fc at 2024-10-06 13:59:12.246944
Device Metrics
{'batteryLevel': 99, 'voltage': 4.182, 'channelUtilization': 7.5833335, 'airUtilTx': 0.21463889, 'uptimeSeconds': 166}
Received Packet: {'from': 1129905132, 'to': 4294967295, 'decoded': {'portnum': 'TELEMETRY_APP', 'payload': b'\ro\x00\x00\x00\x1a\x14\r\x8c\xee\x15B\x15g}\x83A\x1dO\x91yD%~\xc6\x8cD', 'telemetry': {'time': 111, 'environmentMetrics': {'temperature': 37.482956, 'relativeHumidity': 16.436232, 'barometricPressure': 998.27045, 'gasResistance': 1126.2029}, 'raw': time: 111
environment_metrics {
  temperature: 37.4829559
  relative_humidity: 16.4362316
  barometric_pressure: 998.270447
  gas_resistance: 1126.20288
}
}}, 'id': 1567542689, 'rxSnr': 6.5, 'hopLimit': 6, 'rxRssi': -50, 'hopStart': 6, 'raw': from: 1129905132
to: 4294967295
decoded {
  portnum: TELEMETRY_APP
  payload: "\ro\000\000\000\032\024\r\214\356\025B\025g}\203A\035O\221yD%~\306\214D"
}
id: 1567542689
rx_snr: 6.5
hop_limit: 6
rx_rssi: -50
hop_start: 6
, 'fromId': '!4358fbec', 'toId': '^all'}

From node: 0x4358fbec
Packet from !4358fbec at 2024-10-06 13:59:19.477577
BME688
{'temperature': 37.482956, 'relativeHumidity': 16.436232, 'barometricPressure': 998.27045, 'gasResistance': 1126.2029}

and at a different location:

From node: 0x336544fc
Packet from !336544fc at 2024-10-06 14:21:48.006976
Device Metrics
{'batteryLevel': 101, 'voltage': 4.218, 'channelUtilization': 7.57, 'airUtilTx': 0.13002777, 'uptimeSeconds': 106}
Received Packet: {'from': 1129905132, 'to': 4294967295, 'decoded': {'portnum': 'TELEMETRY_APP', 'payload': b'\r#\x01\x00\x00\x1a\x14\r6\xc9\x0eB\x15\xa7\xaa\x8eA\x1d\xb08}D%\x90|\x89D', 'telemetry': {'time': 291, 'environmentMetrics': {'temperature': 35.696495, 'relativeHumidity': 17.833326, 'barometricPressure': 1012.88574, 'gasResistance': 1099.8926}, 'raw': time: 291
environment_metrics {
  temperature: 35.6964951
  relative_humidity: 17.8333263
  barometric_pressure: 1012.88574
  gas_resistance: 1099.89258
}
}}, 'id': 2023253506, 'rxSnr': 6.0, 'hopLimit': 6, 'rxRssi': -34, 'hopStart': 6, 'raw': from: 1129905132
to: 4294967295
decoded {
  portnum: TELEMETRY_APP
  payload: "\r#\001\000\000\032\024\r6\311\016B\025\247\252\216A\035\2608}D%\220|\211D"
}
id: 2023253506
rx_snr: 6
hop_limit: 6
rx_rssi: -34
hop_start: 6
, 'fromId': '!4358fbec', 'toId': '^all'}

From node: 0x4358fbec
Packet from !4358fbec at 2024-10-06 14:22:00.797666
BME688
{'temperature': 35.696495, 'relativeHumidity': 17.833326, 'barometricPressure': 1012.88574, 'gasResistance': 1099.8926}
danineamati commented 2 weeks ago

The dev_logging branch should address most of this issue. The remaining issue relates to including the headers when writing a new file.