home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.15k stars 30.19k forks source link

gas value is not extracted from the data #101342

Closed fromNL closed 11 months ago

fromNL commented 1 year ago

The problem

I received and got installed a brand new meter, the Landis+Gyr E360 (DMSR 5) Had to try DSMR, right? I know my meter is not supported (yet).

I have an issue with this entity: sensor.gas_meter_gas_consumption

Gas Meter Gas consumption | unknown | state_class: total_increasing device_class: gas friendly_name: Gas Meter Gas consumption

It should update every 15 minutes (and it does do that, as data will show), but it has been monitored almost a week now, but there still not one event/value. I am sure the parser is not parsing it.

Did the protocol for DMSR 5 change? Is the gas information missing or have they changed the layout of the data? I am sharing part og the debug log, you can find my gas value in there. Let me know if you need more.

What version of Home Assistant Core has the issue?

core-2023.9.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

dsmr_reader

Link to integration documentation on our website

https://www.home-assistant.io/integrations/dsmr

Diagnostics information

2023-10-03 13:43:52.319 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: /XMX5LGF0010460963654

2023-10-03 13:43:52.335 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-3:0.2.8(50) 0-0:1.0.0(231003134353S 2023-10-03 13:43:52.351 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: ) 0-0:96.1.1(4530303637303036303936333635343233)

2023-10-03 13:43:52.367 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:1.8.1(000025.413kWh) 1-0:1.8.2(000008.808kWh)

2023-10-03 13:43:52.383 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:2.8.1(000038.871kWh) 1-0:2.8.2(0 2023-10-03 13:43:52.399 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 00030.975kWh) 0-0:96.14.0(0002)

2023-10-03 13:43:52.415 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:1.7.0(00.000kW) 1-0:2.7.0(00.305kW)

2023-10-03 13:43:52.430 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 0-0:96.7.21(00005)

2023-10-03 13:43:52.446 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 0-0:96.7.9(00005) 1-0:99.97.0(3)(0-0:96.7.19) 2023-10-03 13:43:52.462 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: (000101000000 2023-10-03 13:43:52.479 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: W)(0000000226s) 2023-10-03 13:43:52.494 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: (000101000000W)(0000006783s) 2023-10-03 13:43:52.510 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: (000101000000W)(0000241161*s) 2023-10-03 13:43:52.526 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data:

2023-10-03 13:43:52.542 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:32.32.0(00001) 1-0:52.32.0( 2023-10-03 13:43:52.559 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 00003) 1-0:72.32.0(00004)

2023-10-03 13:43:52.574 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:32.36.0(00000) 1-0:52.36.0(00000)

2023-10-03 13:43:52.591 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:72.36.0(00000)

2023-10-03 13:43:52.606 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 0-0:96.13.0() 1-0:32.7.0(237.0*V)

2023-10-03 13:43:52.622 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:52.7.0(236.2V) 1-0:72.7.0( 2023-10-03 13:43:52.638 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 235.5V) 1-0:31.7.0(000*A)

2023-10-03 13:43:52.654 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:51.7.0(001A) 1-0:71.7.0(000A)

2023-10-03 13:43:52.670 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:21.7.0(00.050*kW)

2023-10-03 13:43:52.687 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:41.7.0(00.000kW) 1-0:61.7.0(00.000kW)

2023-10-03 13:43:52.703 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 1-0:22.7.0(00.000kW) 1-0:42.7.0( 2023-10-03 13:43:52.718 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 00.326kW) 1-0:62.7.0(00.028*kW)

2023-10-03 13:43:52.735 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 0-1:24.1.0(003) 0-1:96.1.0(473030393030 2023-10-03 13:43:52.750 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 3034323238393738323233)

2023-10-03 13:43:52.767 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: 0-1:24.2.1(231003134001S)(00003.416*m3)

2023-10-03 13:43:52.894 DEBUG (MainThread) [dsmr_parser.clients.protocol] received data: !75FF

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

I am using a "USB Kabel voor P1 Poort van de Nederlandse slimme meter - 2m", which is a serial to usb converter.

home-assistant[bot] commented 1 year ago

Hey there @robbie1221, @frenck, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `dsmr` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign dsmr` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


dsmr documentation dsmr source (message by IssueLinks)

brxxm commented 12 months ago

I have this problem too. HA up & running since april '23, with the 'slimmelezer ethernet' by Marcel Zuidwijk connected. It has run without a problem ever since. But as of last thursday oct 5th there are no readings anymore, for gas that is. Electricity meter still works fine. I have no idea what happened between Oct 4th and Oct 5th that could have caused it. Last reading is from Oct 4th 20:00 CET, as shown in the screenprint.

Schermafbeelding 2023-10-08 om 08 46 56

Edit: It occurred to me that I also use the 'slimmemeterportal' and when I checked there it turns out there are no readings there too, after Oct. 4.

Schermafbeelding 2023-10-08 om 10 15 15

So it seems that something has changed in a common API. Or in the device itself, can't tell that. I'll inform the 'slimmemeterportal' devs too.

PS: My meter is the Landis Gyr E350 DSMR 4.2 GPRS

brxxm commented 11 months ago

I contacted Stedin. They say the gas meter is in error mode and needs maintenance/repair. So this PR can be dropped.

fromNL commented 11 months ago

I am still waiting for support of the Landis+Gyr E360 (DMSR 5), gas collecting is a problem. All data is in the tx/rx data available, it is just a matter of extracting. Who can pick this up? Winter is coming and this is very important. This is the reason I purchased a RPI and installed hassio. Without integration of gas it is like walking on one leg.

nielsvdc commented 11 months ago

Gas usage reader is working again on my side. I restarted the smart gateway hardware module that is connected to my meter, although electricity usage was read before the restart.

mwullink commented 11 months ago

i have the same issue after getting new DMSR 5 meter, gas metrics are no longer avaiable

dupondje commented 11 months ago

@fromNL : The sensor that should match your DSMR output is hourly_gas_meter_reading Does that one has a value?

fromNL commented 11 months ago

No, there is no value at all for hourly_gas_meter_reading it is uninitialized and still has no value.

The debug data shows the value is accessible and available, however the software does not extract it. The newest version must have changed the datastructure.

MartenVijn commented 11 months ago

https://www.home-assistant.io/integrations/dsmr/ refeers to DSMR 4.0 (not 5) for my Landis+Gyr E360

Screenshot 2023-10-28 at 13 53 54
fromNL commented 11 months ago

@MartenVijn Did you check the version on your meter? Mine was placed recently and has a higher version than the integration offers (as said, I have 5.0). Of course I am forced to use the highest version, 4.0, and this results in having no gas data extracted. If I knew were to find the python code of this integration I'd do the update myself, because the gas data is in the stream, it just seems to have shifted a bit.

dupondje commented 11 months ago

@MartenVijn Did you check the version on your meter? Mine was placed recently and has a higher version than the integration offers (as said, I have 5.0). Of course I am forced to use the highest version, 4.0, and this results in having no gas data extracted. If I knew were to find the python code of this integration I'd do the update myself, because the gas data is in the stream, it just seems to have shifted a bit.

You can use whatever version you want? If some parsing is wrong, open a bug report with the telegram included and mentioning which version you use. That's the only way to debug and resolve the issue. The code for the parser lib is here: https://github.com/ndokter/dsmr_parser

dupondje commented 11 months ago

Why would you configure the dsmr to version 4? The telegram seems to be version 5, so just configure dsmr to version 5...

MartenVijn commented 11 months ago

Where / how can I see to firmware version of the meter?

anyway:: changed it to 5, still seems to work.. I didn't values for gas before but now to did does

nb the pytoh codes lives in /usr/src/homeassistant/homeassistant/components/dsmr (in the docker container)

fromNL commented 11 months ago

I moved my request to here: https://github.com/ndokter/dsmr_parser/issues/51

dupondje commented 11 months ago

@fromNL : I would advice to post your debug log for the dsmr integration. The hourly_gas_meter_reading sensor should have a value imo, as data seems to be parsed correctly. Also please confirm which version you have chosen now in HA for dsmr

mwullink commented 11 months ago

I have the following error in my log:

Logger: dsmr_parser.parsers Source: /usr/local/lib/python3.11/site-packages/dsmr_parser/parsers.py:61 First occurred: 15 oktober 2023 om 09:25:40 (36 occurrences) Last logged: 03:40:45

ignore line with signature \d-\d:1.7.0.+?\r\n, because parsing failed. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/dsmr_parser/parsers.py", line 59, in parse telegram[signature] = parser.parse(match.group(0)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/dsmr_parser/parsers.py", line 201, in parse return CosemObject(self._parse(line)) ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/dsmr_parser/parsers.py", line 151, in _parse raise ParseError("Invalid '%s' line for '%s'", line, self) dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1-0:1.7.0(00.27\r\n', <dsmr_parser.parsers.CosemParser object at 0x7fa7c8850690>)

Op 30 okt 2023, om 09:17 heeft dupondje @.***> het volgende geschreven:

@fromNLhttps://github.com/fromNL : I would advice to post your debug log for the dsmr integration. The hourly_gas_meter_reading sensor should have a value imo, as data seems to be parsed correctly. Also please confirm which version you have chosen now in HA for dsmr

— Reply to this email directly, view it on GitHubhttps://github.com/home-assistant/core/issues/101342#issuecomment-1784687893, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AALSFIFJYIW7CUI5GEJUA43YB5PALAVCNFSM6AAAAAA5RQTQYWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUGY4DOOBZGM. You are receiving this because you commented.Message ID: @.***>

dupondje commented 11 months ago

dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1-0:1.7.0(00.27\r\n', <dsmr_parser.parsers.CosemParser object at 0x7fa7c8850690>)

Please post your full telegram, else unable to check this.

fromNL commented 11 months ago

I am sure I selected 5.0, but if there is a way to check this, I can do that. I am not able to run the config for the plugin, it says 'already installed'. Removing the app will delete all data. H.A. is not very forgiving.

dupondje commented 11 months ago

I am sure I selected 5.0, but if there is a way to check this, I can do that. I am not able to run the config for the plugin, it says 'already installed'. Removing the app will delete all data. H.A. is not very forgiving.

In the .storage folder you have a core.config_entries file that should contain something like: "dsmr_version": "5B",

Eventually you should be able to stop HA, change the version in there, and start it again.

mwullink commented 11 months ago

Gateway dedails from web console gateway

Gateway type NL - DSMR4+5 Mac address C0_49_EF_D1_35_38 Ip address 192.168.178.12 Startup time 2023-10-14T09:25:09Z Firmware running 2023100101 Firmware available 2023100101

core.config_entries f

{ "entry_id": "a886a7a6eced3d0a20bdfe0c57bf0a83", "version": 1, "domain": "dsmr", "title": "192.168.178.12:23", "data": { "host": "192.168.178.12", "port": 23, "dsmr_version": "2.2", "serial_id": "xxxxxx-redacted-xxxxxx", "serial_id_gas": "xxxxxx-redacted-xxxxxx", "protocol": "dsmr_protocol" }, "options": {}, "pref_disable_new_entities": false, "pref_disable_polling": false, "source": "user", "unique_id": "xxxxxx-redacted-xxxxxx", "disabled_by": null }

telnet output gateway

1-3:0.2.8(50) 0-0:1.0.0(231030095056W) 0-0:96.1.1(xxxxxx-redacted-xxxxxx) 1-0:1.8.1(000446.025kWh) 1-0:1.8.2(000511.841kWh) 1-0:2.8.1(000000.000kWh) 1-0:2.8.2(000000.000kWh) 0-0:96.14.0(0002) 1-0:1.7.0(00.181kW) 1-0:2.7.0(00.000kW) 0-0:96.7.21(00004) 0-0:96.7.9(00002) 1-0:99.97.0(1)(0-0:96.7.19)(000101000000W)(0000002132s) 1-0:32.32.0(00001) 1-0:52.32.0(00001) 1-0:72.32.0(00001) 1-0:32.36.0(00000) 1-0:52.36.0(00000) 1-0:72.36.0(00000) 0-0:96.13.0() 1-0:32.7.0(231.4V) 1-0:52.7.0(230.9V) 1-0:72.7.0(230.6V) 1-0:31.7.0(001A) 1-0:51.7.0(000A) 1-0:71.7.0(000A) 1-0:21.7.0(00.132kW) 1-0:41.7.0(00.000kW) 1-0:61.7.0(00.049kW) 1-0:22.7.0(00.000kW) 1-0:42.7.0(00.000kW) 1-0:62.7.0(00.000kW) 0-1:24.1.0(003) 0-1:96.1.0(xxxxxx-redacted-xxxxxx) 0-1:24.2.1(231030095008W)(00068.318m3)

Op 30 okt 2023, om 09:32 heeft dupondje @.***> het volgende geschreven:

dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1-0:1.7.0(00.27\r\n', <dsmr_parser.parsers.CosemParser object at 0x7fa7c8850690>)

Please post your full telegram, else unable to check this.

— Reply to this email directly, view it on GitHubhttps://github.com/home-assistant/core/issues/101342#issuecomment-1784710773, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AALSFIEBQ34QWT4SABLZ3ETYB5QZ3AVCNFSM6AAAAAA5RQTQYWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUG4YTANZXGM. You are receiving this because you commented.Message ID: @.***>

fromNL commented 11 months ago

Thanks. It looks like I have been wrong all the time, it shows 5B Can I edit the config, change 5B into 5, and restart H.A. and keep all data?

dupondje commented 11 months ago

Thanks. It looks like I have been wrong all the time, it shows 5B Can I edit the config, change 5B into 5, and restart H.A. and keep all data?

Shutdown HA, Change it, Start HA. That should do :)

fromNL commented 11 months ago

It worked! Thanks. How a human error can lead to a search .. ..

dupondje commented 11 months ago

"dsmr_version": "2.2",

@mwullink : I think this is wrong ... :)

mwullink commented 11 months ago

Not sure what version I should use instead. After enabling debit I do see HA is getting the data from the p1 gateway

2023-10-30 10:13:15.826 DEBUG (MainThread) [dsmr_parser.clients.protocol] got telegram: /XMX5LGF0010460943029

1-3:0.2.8(50) 0-0:1.0.0(231030111338W) 0-0:96.1.1(—redacted--) 1-0:1.8.1(000446.025kWh) 1-0:1.8.2(000512.155kWh) 1-0:2.8.1(000000.000kWh) 1-0:2.8.2(000000.000kWh) 0-0:96.14.0(0002) 1-0:1.7.0(00.176kW) 1-0:2.7.0(00.000kW) 0-0:96.7.21(00004) 0-0:96.7.9(00002) 1-0:99.97.0(1)(0-0:96.7.19)(000101000000W)(0000002132s) 1-0:32.32.0(00001) 1-0:52.32.0(00001) 1-0:72.32.0(00001) 1-0:32.36.0(00000) 1-0:52.36.0(00000) 1-0:72.36.0(00000) 0-0:96.13.0() 1-0:32.7.0(231.7V) 1-0:52.7.0(231.2V) 1-0:72.7.0(231.4V) 1-0:31.7.0(001A) 1-0:51.7.0(000A) 1-0:71.7.0(000A) 1-0:21.7.0(00.126kW) 1-0:41.7.0(00.000kW) 1-0:61.7.0(00.050kW) 1-0:22.7.0(00.000kW) 1-0:42.7.0(00.000kW) 1-0:62.7.0(00.000kW) 0-1:24.1.0(003) 0-1:96.1.0(—redacted--) 0-1:24.2.1(231030111002W)(00068.318m3) !0891

Op 30 okt 2023, om 10:00 heeft dupondje @.***> het volgende geschreven:

"dsmr_version": "2.2",

@mwullinkhttps://github.com/mwullink : I think this is wrong ... :)

— Reply to this email directly, view it on GitHubhttps://github.com/home-assistant/core/issues/101342#issuecomment-1784753576, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AALSFIAI7VHD44EH3N3X3IDYB5UEVAVCNFSM6AAAAAA5RQTQYWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUG42TGNJXGY. You are receiving this because you were mentioned.Message ID: @.***>

dupondje commented 11 months ago

Not sure what version I should use instead.

This telegram seems to be V5. If you have the type of the meter you should know that for sure :)