EDL21 seems to be running (debug shows data) but not creating sensors or entities

[kliemek]

The problem

I added a EDL21 integration with the config frontend in core 2023.4.3 and rebootet some times etc, tried to yaml configuarzion, where I was shown an error, and advised to remove this from yaml. so I did. After many reeboots , removes and readding of the edl21 integration I have not found a solution. Debug always shows flowing data (attached)

What version of Home Assistant Core has the issue?

2023-4.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

edl21

Link to integration documentation on our website

https://www.home-assistant.io/integrations/edl21

Diagnostics information

home-assistant_edl21_2023-04-16T10-14-45.861Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

SO I tried different ways, always got the data in the Debug info, but never a single entity. As you can see in the debug, the Q3B Easymeter provides every 2 seconds the SML Message, what is the documented feature that the information is pushed every 2000ms with the coverage that is shown in the messagte. But it looks like that the parser is not getting the data and so no sensors are created. As I already was on 2023.04. version I cannot tell if this would have worked in the older versions. I just received my USB reader writer for the smart meter this week.

[home-assistant[bot]]

edl21 documentation edl21 source

[witty-monkey]

Hi, started HA a week ago. Since then in try to integrate EDL21 into HA. No success ... IR Reader is working and it is listed "in" the hardware list.

[egennric]

HAOS in a KVM on Debian with update to Home Assistant 2023.4.6 Supervisor 2023.04.0 Operating System 10.0 Frontend 20230411.1 - latest

I run in that problem, "This entity is not available" for all three sensors I have with a previous update I get the note to disable the EDL21 in the configuration.yaml

> - #sensor:
> - #  - platform: edl21
> - #    serial_port: /dev/ttyUSB0
> - #    serial_port: /dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0045-if00-port0
> - #    name: Hausstrom
> 

I did it and all works well until the today update

[Chr1bu]

I can unfortunately confirm this, too.

@StephanU I think it's a result of the new pysml-release and ... ... @mtdcr I've send you a mail with my logs in addition to the logs attached here in this issue.

[mtdcr]

On Sat, 22 Apr 2023 12:25:22 -0700 Chr1bu @.***> wrote:

I can unfortunately confirm this, too.

@StephanU I think it's a result of the new pysml-release and ... ... @mtdcr I've send you a mail with my logs.

I think the offending state_attr "valTime" should just be removed from Home Assistant, because it contains undecoded items. Maybe even all attrs. They don't seem to be very useful.

https://github.com/home-assistant/core/blame/dev/homeassistant/components/edl21/sensor.py#L418

[Chr1bu]

@StephanU here the corresponding logentry:

Unable to serialize to JSON. Bad data found at $[251](State: sensor.smartmeter_metering_point_id_1).attributes.val_time[0]=ListItem(length=2, value=1, bits=ConstBitStream('0x01'))(<class 'sml.ListItem'>, $[251](State: sensor.smartmeter_metering_point_id_1).attributes.val_time[1]=ListItem(length=2, value=0, bits=ConstBitStream('0x00'))(<class 'sml.ListItem'>, $[252](State: sensor.smartmeter_positive_active_energy_total).attributes.val_time[0]=ListItem(length=2, value=1, bits=ConstBitStream('0x01'))(<class 'sml.ListItem'>, $[252](State: sensor.smartmeter_positive_active_energy_total).attributes.val_time[1]=ListItem(length=2, value=0, bits=ConstBitStream('0x00'))(<class 'sml.ListItem'>, $[253](State: sensor.smartmeter_sum_active_instantaneous_power).attributes.val_time[0]=ListItem(length=2, value=1, bits=ConstBitStream('0x01'))(<class 'sml.ListItem'>, $[253](State: sensor.smartmeter_sum_active_instantaneous_power).attributes.val_time[1]=ListItem(length=2, value=0, bits=ConstBitStream('0x00'))(<class 'sml.ListItem'>

[egennric]

Is this the problem discussed here?

Logger: homeassistant.components.recorder.table_managers.state_attributes Source: components/recorder/table_managers/state_attributes.py:56 Integration: Recorder (documentation, issues) First occurred: 22. April 2023 um 12:49:46 (2812 occurrences) Last logged: 12:26:39

State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=161.9; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:24:37.302793+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.positive_active_energy_total=3398501.0; state_class=total_increasing, status=1868036, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=Wh, device_class=energy, friendly_name=Hausstrom Positive active energy total @ 2023-04-23T12:25:38.301282+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=176.68; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:25:38.302067+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.positive_active_energy_total=3398503.8; state_class=total_increasing, status=1868036, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=Wh, device_class=energy, friendly_name=Hausstrom Positive active energy total @ 2023-04-23T12:26:39.299684+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=164.69; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:26:39.300480+02:00>: Type is not JSON serializable: ConstBitStream

[StephanU]

On Sat, 22 Apr 2023 12:25:22 -0700 Chr1bu @.***> wrote: I can unfortunately confirm this, too. @StephanU I think it's a result of the new pysml-release and ... ... @mtdcr I've send you a mail with my logs. I think the offending state_attr "valTime" should just be removed from Home Assistant, because it contains undecoded items. Maybe even all attrs. They don't seem to be very useful. https://github.com/home-assistant/core/blame/dev/homeassistant/components/edl21/sensor.py#L418

Ok, I will remove those attributes.

[StephanU]

Those of you who have the Type is not JSON serializable: ConstBitStream problem: What type/brand of smart meter do you own?

[Chr1bu]

Meter Model: "DZG DVS7412.1" Meter Serialno: starts with "DZG 00"

[witty-monkey]

Meter Model: Itron OpenWay 3.HZ

[dsteinkopf]

I am using an EMH mMe4.0.

[StephanU]

Ok, so the problem is not related to a specific type of smartmeter... I am using a Iskraemeco MT691 and I don't have this problem. Anyway, removing the extra_state_attributes should solve this :)

[egennric]

Is this the problem discussed here?

Logger: homeassistant.components.recorder.table_managers.state_attributes Source: components/recorder/table_managers/state_attributes.py:56 Integration: Recorder (documentation, issues) First occurred: 22. April 2023 um 12:49:46 (2812 occurrences) Last logged: 12:26:39

State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=161.9; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:24:37.302793+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.positive_active_energy_total=3398501.0; state_class=total_increasing, status=1868036, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=Wh, device_class=energy, friendly_name=Hausstrom Positive active energy total @ 2023-04-23T12:25:38.301282+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=176.68; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:25:38.302067+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.positive_active_energy_total=3398503.8; state_class=total_increasing, status=1868036, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=Wh, device_class=energy, friendly_name=Hausstrom Positive active energy total @ 2023-04-23T12:26:39.299684+02:00>: Type is not JSON serializable: ConstBitStream
State is not JSON serializable: <state sensor.hausstrom_sum_active_instantaneous_power=164.69; state_class=measurement, val_time=[ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=2, value=0, bits=ConstBitStream('0x00'))], unit_of_measurement=W, device_class=power, friendly_name=Hausstrom Sum active instantaneous power @ 2023-04-23T12:26:39.300480+02:00>: Type is not JSON serializable: ConstBitStream

I went back to 2023.4.5 to get my DVS74/DWS74 working again problem started with 2023.4.6

[witty-monkey]

Ok, so the problem is not related to a specific type of smartmeter... I am using a Iskraemeco MT691 and I don't have this problem. Anyway, removing the extra_state_attributes should solve this :)

Hi StephanU,

no Idea how to remove the "extra state attributes". Which file do i have to edit. Where do i find it ... etc. Would be very nice of you to give me some hints.

[Chr1bu]

@witty-monkey you only have to wait for the next HA-Release 2023.4.7 or 2023.5.0 😉

[lukas-kd]

I have the same problem, but only with one integration. I have two edl21 integrations. One has its entities and the second not.

Can there be any other issue?

[StephanU]

I have the same problem, but only with one integration. I have two edl21 integrations. One has its entities and the second not.

Can there be any other issue?

Do you have two different types/brand of smart meter? This could explain that one is working and the other one is not working... Anyway, let's wait for the release which contains #91922

[lukas-kd]

I have the same problem, but only with one integration. I have two edl21 integrations. One has its entities and the second not. Can there be any other issue?

Do you have two different types/brand of smart meter? This could explain that one is working and the other one is not working... Anyway, let's wait for the release which contains #91922

Now working... just switch to another usb port... Strange, why the one is no more working...

I am using the Hichi one for both integrations in HA Core 2023.4.6.

[b3n3d1kt1337]

I have the same problem, but only with one integration. I have two edl21 integrations. One has its entities and the second not. Can there be any other issue?

Do you have two different types/brand of smart meter? This could explain that one is working and the other one is not working... Anyway, let's wait for the release which contains #91922

Now working... just switch to another usb port... Strange, why the one is no more working...

I am using the Hichi one for both integrations in HA Core 2023.4.6.

I'm using the one from Hichi too but switching USB Ports didn't change anything for me. Is there any concrete date when the problem will be fixed?

[Chr1bu]

@lukaskuerzdoerfer and @b3n3d1kt1337 it has nothing todo with the hichi-Infrared-Sensor (i'm also using it). It depends on the used smartmeter and the ha-version. I think that the new HA-Release would be released in the next 7-10 days, because the Milestone 2023.5.0. is actually scheduled till 3. May: https://github.com/home-assistant/core/milestone/593 :)

[lukas-kd]

Yeah, i need to wait too, only worked for a few hours..

[dsteinkopf]

@lukaskuerzdoerfer and @b3n3d1kt1337 it has nothing todo with the hichi-Infrared-Sensor (i'm also using it). It depends on the used smartmeter and the ha-version. I think that the new HA-Release would be released in the next 7-10 days, because the Milestone 2023.5.0. is actually scheduled till 3. May: https://github.com/home-assistant/core/milestone/593 :)

I don't see "our" #91922 in this milestone... 🤔

[dsteinkopf]

BTW. Anyone suffering from the problem in the current version: Downgrading is very easily done by calling from the HA console: ha core update --version 2023.4.5. (Okay, it's easy when you are familiar with command line and are already able to open the HA console.)

[Chr1bu]

@lukaskuerzdoerfer and @b3n3d1kt1337 it has nothing todo with the hichi-Infrared-Sensor (i'm also using it). It depends on the used smartmeter and the ha-version. I think that the new HA-Release would be released in the next 7-10 days, because the Milestone 2023.5.0. is actually scheduled till 3. May: https://github.com/home-assistant/core/milestone/593 :)

I don't see "our" #91922 in this milestone... 🤔

Patience ☺️

[Chr1bu]

@StephanU and @mtdcr
The new HA-Release 2023.5.0 is out now and -for me- everthing is working fine again 🎉 Thanks!! 👍

[lukas-kd]

My problem is still not fixed. One edl21 is working and the second not. I don`t get why, its the same manufacturer... any ideas, how i can relate this issue to any error?

[kliemek]

after update to 2023.5.0 no change - the data is flowing, debug dump shows the output of the ESY 3Q Smartmeter as SML recognized - but no sensor or entity is created, rebooting / removing / re-adding does not change the situation So without understanding the code, the entity creation seems to have the issue.

home-assistant_edl21_2023-05-04T09-19-36.989Z.log

[dsteinkopf]

2023.5.0 fixed the problem for me, too.

[lukas-kd]

I think it can only be one Edl21 Integration.. Maybe something in the Code overrides the other Integration... Can anyone verify this?

[egennric]

2023.5.0 fixed the problem for me, too.

for me too

[b3n3d1kt1337]

My problem is still not fixed. One edl21 is working and the second not. I don`t get why, its the same manufacturer... any ideas, how i can relate this issue to any error?

Did you try to remove the edl21 Integration and reinstall it after restarting Home Assistant? This worked fine for me since it wasn't working after the update to the recent version.

[StephanU]

I have two edl21 integrations running without problems. Maybe try reinstalling the edl21 integration as suggested by @b3n3d1kt1337. And also test whether your two smartmeter are working separatly (so only add the integration for the first one, see if it is working, then remove it and install it for the second one and see if it is working).

[lukas-kd]

I've tried both of your suggestions. Sometime ago, i had both up and running, but lost it with the earlier update (Moved from configuration.yaml to Config-Flow).

But currently always the same is failing... Maybe i can test the sensor with another tool, like hterm.

If anyone knows a Bretter idea to verify the integrity of the device or configuring it correclty or enabling a netter debug log (because i think the protocol handler does not get a event), i will try it out! :)

[StephanU]

Did you try switching the usb ports of the smart meters?

[lukas-kd]

Yeah, commented it earlier in this thread. Worked for a few hours.

[StephanU]

Hmm, I could try adding a few more debug messages... this might also help to identify @kliemek 's problem...

[lukas-kd]

That would be very nice. So i can put it in here or find it on my own. 👍

[olu12345]

Hello everyone,

I have the problem that as soon as the "Sum active instantaneous power" values ​​become negative, they jump to over 160,000W (positive value).

I have a photovoltaic system and during the day the values ​​are negative.

Does anyone else have this problem?

[kliemek]

@StephanU Hi made some tests today, actually on the latest HA version - 23.5.2 - changed the USB on the PI, removed edl, rebootet the whole system, added edl21 again - doing some tests, always the same, the output of the logging shows the data but no entity or sensor is created ... strange - no errors, only the full messages from the Easymeter - any idea, what might can help in finding the issue?

home-assistant_edl21_2023-05-07T09-47-35.812Z.log

[Chr1bu]

@olu12345 I think you have to open a separate issue, with full informations (SmartMeter-model, HA-Version, Logs etc.).

[thomasletsch]

It seems that I have the same issue. I have two meters, one is working correctly, the other one is not creating any items. The first meter is a Easymeter Q3MA1170 V6.03 and works perfectly, the other one is a Easy Meter Q3BA1000 V4.03 and does not work. I can see entries in the log file which seems to be from this meter: 2023-05-16 18:48:44.418 DEBUG (MainThread) [sml] [] RESULT (136 bits left): {'transactionId': ListItem(length=2, value=b'\xa5', bits=ConstBitStream('0xa5')), 'groupNo': ListItem(length=2, value=0, bits=ConstBitStream('0x00')), 'abortOnError': ListItem(length=2, value=0, bits=ConstBitStream('0x00')), 'messageBody': {'serverId': b'\x06ESY\x01\x02rhW\xb8', 'actSensorTime': [ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=5, value=272361718, bits=ConstBitStream('0x103be8f6'))], 'valList': [{'objName': '129-129:199.130.3255', 'value': 'ESY'}, {'objName': '1-0:1.8.0255', 'unit': 'Wh', 'value': 48241724.8794}, {'objName': '1-0:1.8.1255', 'unit': 'Wh', 'value': 48241400}, {'objName': '1-0:1.8.2255', 'unit': 'Wh', 'value': 320}, {'objName': '1-0:1.7.0255', 'unit': 'W', 'value': 23.18}, {'objName': '1-0:21.7.0255', 'unit': 'W', 'value': 0}, {'objName': '1-0:41.7.0255', 'unit': 'W', 'value': 0}, {'objName': '1-0:61.7.0255', 'unit': 'W', 'value': 23.18}, {'objName': '1-0:96.5.5255', 'value': 400}]}, 'crc16': ListItem(length=3, value=48336, bits=ConstBitStream('0xbcd0')), 'endOfSmlMsg': ListItem(length=0, value=None, bits=None)}

But no devices or entities are created

[lukas-kd]

Is there an easy way to ensure that the smart Meter etc is working correctly? Like any bash Tool or something? So we can be safe that it is the edl Integration and Not a Hardware problem?

[lukas-kd]

I've now created a simple Python script to read the messages. One is working the other seems to be not working, because i get no messages (this is now a New one...). If i use 'cat' i receive content... Does anyone have an idea?

[mtdcr]

If you create a raw log file (about 30 seconds with cat or dd) and send it to me by mail (obi at saftware dot de), I'd take a look at it.

Am 17. Mai 2023 20:18:30 MESZ schrieb Lukas @.***>:

I've now created a simple Python script to read the messages. One is working the other seems to be not working, because i get no messages (this is now a New one...). If i use 'cat' i receive content... Does anyone have an idea?

-- Reply to this email directly or view it on GitHub: https://github.com/home-assistant/core/issues/91494#issuecomment-1551856578 You are receiving this because you were mentioned.

Message ID: @.***>

[mtdcr]

There's no electricity ID OBIS entry in your SML message. I think the integration should fall back to using the serverId field for this case. However, I'd prefer if pysml did the conversion from raw bytes to a string, because it looks like a parseable value.

Does your meter have an ID printed on it?

Am 16. Mai 2023 19:07:30 MESZ schrieb Thomas Letsch @.***>:

It seems that I have the same issue. I have two meters, one is working correctly, the other one is not creating any items. The first meter is a Easymeter Q3MA1170 V6.03 and works perfectly, the other one is a Easy Meter Q3BA1000 V4.03 and does not work. I can see entries in the log file which seems to be from this meter: 2023-05-16 18:48:44.418 DEBUG (MainThread) [sml] [] RESULT (136 bits left): {'transactionId': ListItem(length=2, value=b'\xa5', bits=ConstBitStream('0xa5')), 'groupNo': ListItem(length=2, value=0, bits=ConstBitStream('0x00')), 'abortOnError': ListItem(length=2, value=0, bits=ConstBitStream('0x00')), 'messageBody': {'serverId': b'\x06ESY\x01\x02rhW\xb8', 'actSensorTime': [ListItem(length=2, value=1, bits=ConstBitStream('0x01')), ListItem(length=5, value=272361718, bits=ConstBitStream('0x103be8f6'))], 'valList': [{'objName': '129-129:199.130.3255', 'value': 'ESY'}, {'objName': '1-0:1.8.0255', 'unit': 'Wh', 'value': 48241724.8794}, {'objName': '1-0:1.8.1255', 'unit': 'Wh', 'value': 48241400}, {'objName': '1-0:1.8.2255', 'unit': 'Wh', 'value': 320}, {'objName': '1-0:1.7.0255', 'unit': 'W', 'value': 23.18}, {'objName': '1-0:21.7.0255', 'unit': 'W', 'value': 0}, {'objName': '1-0:41.7.0255', 'unit': 'W', 'value': 0}, {'objName': '1-0:61.7.0255', 'unit': 'W', 'value': 23.18}, {'objName': '1-0:96.5.5255', 'value': 400}]}, 'crc16': ListItem(length=3, value=48336, bits=ConstBitStream('0xbcd0')), 'endOfSmlMsg': ListItem(length=0, value=None, bits=None)}

But no devices or entities are created

-- Reply to this email directly or view it on GitHub: https://github.com/home-assistant/core/issues/91494#issuecomment-1550054837 You are receiving this because you were mentioned.

Message ID: @.***>

[thomasletsch]

@mtdcr : Yes, my meter does have several IDs printed on it. The Server ID: 06-45-53-59-01-02-72-68-57-B8 Then a number with Barcode: 1005206 And a Serial Number (S/N): 1121003192

[mtdcr]

@StephanU pysml 0.0.12 decodes the serverId field to match either the meter ID with the three-letter-acronym followed by a number, or the dash-separated hex format shown by @thomasletsch. This is the same behavior as used for the OBIS values.

In sensor.py event(), you'd get the serverId using electricity_id = message_body["serverId"]. This field is mandatory for SmlGetListResponse, and for the limited test cases I have, the values were exactly the same as in 1-0:0.0.9*255 or 1-0:96.1.0*255. So I guess you could just remove the loop that tries to find one of these two values, if you're feeling like taking the risk. :-)