Parsing fails on new Sagecom T210D

[cloudcosmonaut]

I just got a new meter, as I upgraded my 1x40A to 3x25A.

But when I connect it to my Home-Assistent, I no longer receive any data.

When trying to figure out why, I noticed the logs are getting filled with the same message every time the p1 message is received.

2021-01-08 19:31:14 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SerialTransport._read_ready() Traceback (most recent call last): File "/usr/local/lib/python3.8/asyncio/events.py", line 81, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.8/site-packages/serial_asyncio/__init__.py", line 119, in _read_ready self._protocol.data_received(data) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 96, in data_received self.handle_telegram(telegram) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 111, in handle_telegram parsed_telegram = self.telegram_parser.parse(telegram) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 59, in parse telegram[signature] = parser.parse(match.group(0)) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 174, in parse return MBusObject(self._parse(line)) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 153, in _parse return self._parse_values(values) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 138, in _parse_values return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 138, in return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 284, in parse value = self.coerce_type(value) if value is not None else value File "/usr/local/lib/python3.8/site-packages/dsmr_parser/value_types.py", line 7, in timestamp naive_datetime = datetime.datetime.strptime(value[:-1], '%y%m%d%H%M%S') File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "/usr/local/lib/python3.8/_strptime.py", line 352, in _strptime raise ValueError("unconverted data remains: %s" % ValueError: unconverted data remains: 525

Hopefully you have an idea what caused this, and where I can look further.

Thanks in advance!

[cloudcosmonaut]

This is the data I get back:

/Ene5\T210-D ESMR 5.0

1-3:0.2.8(50)
0-0:1.0.0(210109092640W)
0-0:96.1.1(4530303438303030303438353534333230)
1-0:1.8.1(000009.129*kWh)
1-0:1.8.2(000017.943*kWh)
1-0:2.8.1(000000.087*kWh)
1-0:2.8.2(000000.000*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(00.255*kW)
1-0:2.7.0(00.363*kW)
0-0:96.7.21(00007)
0-0:96.7.9(00005)
1-0:99.97.0(1)(0-0:96.7.19)(201013190420S)(0000000198*s)
1-0:32.32.0(00002)
1-0:52.32.0(00002)
1-0:72.32.0(00002)
1-0:32.36.0(00000)
1-0:52.36.0(00000)
1-0:72.36.0(00000)
0-0:96.13.0()
1-0:32.7.0(229.0*V)
1-0:52.7.0(231.0*V)
1-0:72.7.0(230.0*V)
1-0:31.7.0(001*A)
1-0:51.7.0(000*A)
1-0:71.7.0(001*A)
1-0:21.7.0(00.169*kW)
1-0:41.7.0(00.085*kW)
1-0:61.7.0(00.000*kW)
1-0:22.7.0(00.000*kW)
1-0:42.7.0(00.000*kW)
1-0:62.7.0(00.363*kW)
0-1:24.1.0(003)
0-1:96.1.0(0000000000000000000000000000000000)
0-1:24.2.1(632525252525W)(00000.000)
!288A

[cloudcosmonaut]

I see the gas meter is not returning values. Is it possible to ignore these values when they're invalid?

[lowdef]

Did you already find the offending line? With your telegram, I can reproduce the error here.

[lowdef]

Offending line is 0-1:24.2.1(632525252525W)(00000.000)

It is parsed with: obis.HOURLY_GAS_METER_READING: MBusParser( ValueParser(timestamp), ValueParser(Decimal) )

This is the same problem as in issue https://github.com/ndokter/dsmr_parser/issues/51 and has to do with a wrong setup of the MBus on your side.

0-1:24.1.0(003) ==> Device Type Gasmeter 0-1:96.1.0(0000000000000000000000000000000000) ==> Equipment Identifier Gas 0-1:24.2.1(632525252525W)(00000.000) ==> Should be something like (200426223001S)(00246.138*m3)

The parser expects a timestamp and a gas meter reading. It finds someting else, therfore can not parse.

Possible solutions:

1. If possible fix the erroneous Mbus setup. That would result in correct P1 telegrams.
2. Filter the offending entries from the telegram, before presenting it to the parser.

As a structural solution for making the parser more robust, I see the following solution:

• Do not throw parsing exceptions for single offending entries, but instead ignore the offending telegram line and try to parse the rest as if nothing happened.

[cloudcosmonaut]

Yeah, I noticed the invalid data also (albeit, the date is valid, but it shouldn't be, as you can't state something happened with a future data).

I think, the parsing should be more resilient. But I don't have clue how I could do that in this project, as I don't have experience in Python. I tried to run the unit tests, so I could add a failing test case that I could fix. But I don't understand how the tests can be run.

[dennissiemensma]

One possible way to fix this, as commented above by lowdef, is to catch the datetime parsing error. I'm not sure whether it's the best way to solve the problem, but it seems to work fine for me for a while: value_types.py

However, do note that I still needed to check for the empty value to clear the device info as well: code.

Note that the check/clear code is outside the scope of dsmr_parser's code though. It might require any application utilizing the parser to check it, similar to above, depending on whether a None value affects it at all.

[lowdef]

I provided a pull request https://github.com/ndokter/dsmr_parser/pull/74 That should fix the Issue. The offending line will not be part of the telegram object.

[cloudcosmonaut]

Thanks, that would solve my problem 👌🏻