search

ndokter / dsmr_parser

Library to parse Dutch Smart Meter Requirements (DSMR) telegrams.
MIT License
112 stars 63 forks source link

Hourly Gas Consumption is not available / Ignore line with signature \d-\d:24\.2\.1.+?\r\n, because parsing failed. #120

Closed eelcohn closed 1 year ago

eelcohn commented 1 year ago

In Home Assistant the sensor.hourly_gas_consumption always shows unavailable. I also regulary get the following warnings:

Logger: dsmr_parser.parsers
Source: /usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py:61
First occurred: 16:57:10 (165 occurrences)
Last logged: 16:59:54

ignore line with signature \d-\d:24\.2\.1.+?\r\n, because parsing failed.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 59, in parse
    telegram[signature] = parser.parse(match.group(0))
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 177, in parse
    return MBusObject(self._parse(line))
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 156, in _parse
    return self._parse_values(values)
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 141, in _parse_values
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 141, in <listcomp>
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 285, in parse
    value = self.coerce_type(value) if value is not None else value
  File "/usr/local/lib/python3.10/site-packages/dsmr_parser/value_types.py", line 7, in timestamp
    naive_datetime = datetime.datetime.strptime(value[:-1], '%y%m%d%H%M%S')
  File "/usr/local/lib/python3.10/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.10/_strptime.py", line 352, in _strptime
    raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: 525

These errors started when my smart gas meter was installed. Before that the integration worked without any problems. The 24.2.1 telegram seems to be the hourly gas consumption value.

My smart meter is a Sagemcom XS210 ESMR5, and my gas meter is a Itron G4RF1eWL.

dupondje commented 1 year ago

I think it might be useful to share the data you receive from the p1 port. The full dsmr message.

eelcohn commented 1 year ago

I think it might be useful to share the data you receive from the p1 port. The full dsmr message.

How do I do that?

dupondje commented 1 year ago

I think it might be useful to share the data you receive from the p1 port. The full dsmr message.

How do I do that?

You can enable debug on the integration in HA from the UI, and then download the logs. The DSMR data will be in the log.

hultenvp commented 1 year ago

Chipping in here, got the same issue. When changing to 3 phase my smart meter got replaced with a Sagemcom T211. Gas meter did not get replaced. Since then I stopped receiving gas data (see screenshot)

image

I haven't looked much into the protocol, but it looks like the telegram does only contain zeroes for gas consumption. All log entries are reporting 0-1:24.2.1(632525252525S)(00000.000). Still I'd expect the DSMR component doesn't trip over it and returns the reported value.

2023-01-27 09:41:31.288 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: /Ene5\T211 ESMR 5.0
1-3:0.2.8(50)
0-0:1.0.0(220817134512S)
0-0:96.1.1(4530303632303030303130303331343232)
1-0:1.8.1(000026.413*kWh)
1-0:1.8.2(000000.000*kWh)
1-0:2.8.1(000001.164*kWh)
1-0:2.8.2(000000.000*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(00.383*kW)
1-0:2.7.0(00.000*kW)
0-0:96.7.21(00005)
0-0:96.7.9(00003)
1-0:99.97.0(0)(0-0:96.7.19)
1-0:32.32.0(00001)
1-0:52.32.0(00001)
1-0:72.32.0(00001)
1-0:32.36.0(00000)
1-0:52.36.0(00000)
1-0:72.36.0(00000)
0-0:96.13.0()
1-0:32.7.0
2023-01-27 09:41:31.331 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: (230.0*V)
1-0:52.7.0(231.0*V)
1-0:72.7.0(228.0*V)
1-0:31.7.0(000*A)
1-0:51.7.0(000*A)
1-0:71.7.0(002*A)
1-0:21.7.0(00.013*kW)
1-0:41.7.0(00.040*kW)
1-0:61.7.0(00.329*kW)
1-0:22.7.0(00.000*kW)
1-0:42.7.0(00.000*kW)
1-0:62.7.0(00.000*kW)
0-1:24.1.0(003)
0-1:96.1.0(0000000000000000000000000000000000)
0-1:24.2.1(632525252525S)(00000.000)
!37DF
2023-01-27 09:41:31.331 DEBUG (MainThread) [dsmr_parser.clients.protocol] got telegram: /Ene5\T211 ESMR 5.0
1-3:0.2.8(50)
0-0:1.0.0(220817134512S)
0-0:96.1.1(4530303632303030303130303331343232)
1-0:1.8.1(000026.413*kWh)
1-0:1.8.2(000000.000*kWh)
1-0:2.8.1(000001.164*kWh)
1-0:2.8.2(000000.000*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(00.383*kW)
1-0:2.7.0(00.000*kW)
0-0:96.7.21(00005)
0-0:96.7.9(00003)
1-0:99.97.0(0)(0-0:96.7.19)
1-0:32.32.0(00001)
1-0:52.32.0(00001)
1-0:72.32.0(00001)
1-0:32.36.0(00000)
1-0:52.36.0(00000)
1-0:72.36.0(00000)
0-0:96.13.0()
1-0:32.7.0(230.0*V)
1-0:52.7.0(231.0*V)
1-0:72.7.0(228.0*V)
1-0:31.7.0(000*A)
1-0:51.7.0(000*A)
1-0:71.7.0(002*A)
1-0:21.7.0(00.013*kW)
1-0:41.7.0(00.040*kW)
1-0:61.7.0(00.329*kW)
1-0:22.7.0(00.000*kW)
1-0:42.7.0(00.000*kW)
1-0:62.7.0(00.000*kW)
0-1:24.1.0(003)
0-1:96.1.0(0000000000000000000000000000000000)
0-1:24.2.1(632525252525S)(00000.000)
!37DF
2023-01-27 09:41:31.337 ERROR (MainThread) [dsmr_parser.parsers] ignore line with signature \d-\d:24\.2\.1.+?\r\n, because parsing failed.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 59, in parse
telegram[signature] = parser.parse(match.group(0))
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 177, in parse
return MBusObject(self._parse(line))
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 156, in _parse
return self._parse_values(values)
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 141, in _parse_values
return [self.value_formats[i].parse(value)
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 141, in <listcomp>
return [self.value_formats[i].parse(value)
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/parsers.py", line 285, in parse
value = self.coerce_type(value) if value is not None else value
File "/usr/local/lib/python3.10/site-packages/dsmr_parser/value_types.py", line 7, in timestamp
naive_datetime = datetime.datetime.strptime(value[:-1], '%y%m%d%H%M%S')
File "/usr/local/lib/python3.10/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/usr/local/lib/python3.10/_strptime.py", line 352, in _strptime
raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: 525
eelcohn commented 1 year ago

Here's my log file:

2023-01-27 11:20:56.246 DEBUG (MainThread) [dsmr_parser.clients.protocol] connected
2023-01-27 11:20:56.278 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: \00

2023-01-27 11:20:56.407 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: /Ene5\XS210 ESMR 5.0

1-3:0.2.8(50)
0-0:1.0.0(230127112102W)
0-0:96.1.1(4530303437303030303130303331343138)
1-0:1.8.1(005513.321*kWh)
1-0:1.8.2(004237.979*kWh)
1-0:2.8.1(000775.722*kWh)
1-0:2.8.2(001693.364*kWh)
0-0:96.14.0(0002)
1-0:1.7.0(00.000*kW)
1-0:2.7.0(00.019*kW)
0-0:96.7.21(00664)
0-0:96.7.9(00003)
1-0:99.97.0(1)(0-0:96.7.19)(190509140157S)(0000000258*s)
1-0:32.32.0(00003)
1-0:32.36.0(00000)
0-0:96.13.0()
1-0:32.7.0(229.0*V)
1-0:31.7.0(001*A)
1-0:21.7.0(00.000*kW)
1-0:22.7.0(00.019*kW)
0-1:24.1.0(003)
0-1:96.1.0(4730303437303030303130393039363230)
0-1:24.2.1(230127112000W)(03010.028*m3)
!CCC2
dupondje commented 1 year ago

@hultenvp : yours can't be parsed because the timestamp is incorrect: 632525252525S This can't be parsed to a date and it fails

@eelcohn : yours fails because 230127112000W is not in the correct format it seems. It's expected to end on a 'S' Docs tell me: ASCII presentation of Time stamp with Year, Month, Day, Hour, Minute, Second, and an indication whether DST is active (X=S) or DST is not active (X=W).

Should fix that :)

dupondje commented 1 year ago

@hultenvp : Fix see above :)

@eelcohn : Seems like I was wrong. It did not fail on that DSMR message right? Guess you sometimes still have a wrong DSMR message with invalid timestamp, and then it fails.

hultenvp commented 1 year ago

Thanks @dupondje . That will at least get rid of the hundreds of errors in my log. I'm working with Enexis finding out why I do not receive gas data, but they have a policy that they'll test all connections somewhere in the first 10 working days after install of new equipment. I have to wait for that one first.