ZLinky attributes sent to MQTT are not restricted to sensor value filling HA database at 3 GB / day !

[capi92]

What happened?

Lixee Linky running firmware date 20220217 version 4000-0005 Linky in standard mode, single phase, no production.

Setup : Zlinky - > ZigBee2MQTT -> Home Assistant

Database get filled at 3 GB / day.

1/ First part of the graph no linky -- > 40 MB /day 2/ linky no filtering --> 3 GB /day 3/ linky filtering on 3 sensors --> 350 MB / day 4/ linky with 2 sensors custom based taking 1 single attributes--> 50 MB / day

Each record in HA database contain as state the value of the sensor and as attribute a full dump of all data Linky can produce.

Database content for 2 records from HA:

SELECT * FROM states where entity_id = "sensor.linky_easd02";

state_id	domain	entity_id	state	attributes
35275309	sensor	sensor.linky_easd02	15970.036	{"state_class":"measurement", "active_enerfy_out_d01":7791.507,"active_enerfy_out_d02":15970.036,"active_enerfy_out_d03":null,"active_enerfy_out_d04":null,"active_power":2098,"active_power_max":6663,"active_power_ph_b":3250,"apparent_power":6002,"available_power":9,"average_rms_voltage_meas_period":240,"current_date":"H220306224851","current_index_tarif":1,"current_price":"HEURE CREUSE","current_summ_delivered":23761.519,"current_tarif":"H PLEINE/CREUSE","current_tier10_summ_delivered":0,"current_tier1_summ_delivered":7791.483,"current_tier2_summ_delivered":15970.036,"current_tier3_summ_delivered":0,"current_tier4_summ_delivered":0,"current_tier5_summ_delivered":0,"current_tier6_summ_delivered":0,"current_tier7_summ_delivered":0,"current_tier8_summ_delivered":0,"current_tier9_summ_delivered":0,"days_number_current_calendar":null,"days_number_next_calendar":null,"days_profile_current_calendar":null,"days_profile_next_calendar":null,"drawn_v_a_max_n1":5563,"linkquality":75,"message1":"PAS DE ###MESSAGE","message2":"","meter_serial_number":"06206118xxxx","power_threshold":9,"relais":1,"rms_current":25,"rms_current_max":65535,"rms_voltage":240,"site_id":"2145658453xxxx","software_revision":2,"start_mobile_point1":null,"start_mobile_point2":null,"start_mobile_point3":null,"status_register":"003A0000","stop_mobile_point1":null,"stop_mobile_point2":null,"stop_mobile_point3":null,"tomorrow_color":"","update":{"state":"idle"},"update_available":false,"unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Linky EASD02","last_reset":"1970-01-01T00:00:00+00:00"**}	36272829	2022-03-06 21:50:03.031440	2022-03-06 21:50:03.031440	2022-03-06 21:50:03.031440	35275287
35275336	sensor	sensor.linky_easd02	15970.036	{"state_class":"measurement","active_enerfy_out_d01":7791.507,"active_enerfy_out_d02":15970.036,"active_enerfy_out_d03":null,"active_enerfy_out_d04":null,"active_power":2098,"active_power_max":6663,"active_power_ph_b":3250,"apparent_power":6002,"available_power":9,"average_rms_voltage_meas_period":240,"current_date":"H220306224851","current_index_tarif":1,"current_price":"HEURE CREUSE","current_summ_delivered":23761.519,"current_tarif":"H PLEINE/CREUSE","current_tier10_summ_delivered":0,"current_tier1_summ_delivered":7791.483,"current_tier2_summ_delivered":15970.036,"current_tier3_summ_delivered":0,"current_tier4_summ_delivered":0,"current_tier5_summ_delivered":0,"current_tier6_summ_delivered":0,"current_tier7_summ_delivered":0,"current_tier8_summ_delivered":0,"current_tier9_summ_delivered":0,"days_number_current_calendar":null,"days_number_next_calendar":null,"days_profile_current_calendar":null,"days_profile_next_calendar":null,"drawn_v_a_max_n1":5563,"linkquality":78,"message1":"PAS DE MESSAGE","message2":"","meter_serial_number":"06206118xxxx","power_threshold":9,"relais":1,"rms_current":25,"rms_current_max":65535,"rms_voltage":240,"site_id":"2145658453xxxx","software_revision":2,"start_mobile_point1":null,"start_mobile_point2":null,"start_mobile_point3":null,"status_register":"003A0000","stop_mobile_point1":null,"stop_mobile_point2":null,"stop_mobile_point3":null,"tomorrow_color":"","update":{"state":"idle"},"update_available":false,"unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Linky EASD02","last_reset":"1970-01-01T00:00:00+00:00"}

Everything that is not in bold shall not be there.

What did you expect to happen?

state_id	domain	entity_id	state	attributes	event_id	last_changed	last_updated	created	old_state_id
36736491	sensor	sensor.linky_easd02	7847.601	{"unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Conso HCHC","state_class":"total_increasing","last_reset":"1970-01-01T00:00:00+00:00"}	37780631	2022-03-10 21:29:41.844866	2022-03-10 21:29:41.844866	2022-03-10 21:29:41.844866	36698044
36736612	sensor	sensor.linky_easd02	7847.706	{"unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Conso HCHC","state_class":"total_increasing","last_reset":"1970-01-01T00:00:00+00:00"}	37780756	2022-03-10 21:31:12.421301	2022-03-10 21:31:12.421301	2022-03-10 21:31:12.421301	36736491

How to reproduce it (minimal and precise)

Looks like this problem is only visible in standard mode not in historic mode.

Zigbee2MQTT version

1.24.0

Adapter firmware version

20210120

Adapter

SonOff ZIGBEE 3.0 USB DONGLE PLUS

Debug log

No response

[Koenkk]

@vk496 could you check this?

[vk496]

@Koenkk that's something I was wondering too. If you go to Developer tools under HA for any device that is reporting Z2M, you can see that every attribute contains a full list of the attributes as properties

In this example of Tuya device (5 attributes), I expect to have state_class, device_class, unit_of_mesurement, friendly name... under temperature. Not the rest of the 5 attributes too.

That is usually not appreciated for small devices. But in case of Zlinky (easily 30 attributes in some configs), where each attribute contains all attributes for EVERY message... without the limits I put, I have over 5G/day too.

I don't found the use case or history of where this behaviour is required. So I would be interested in know this too

[capi92]

As per exchange with other users of Zlinky, looks like this problem doesn't appear when having historic mode instead of standard.

I have Tarif set to auto as my tarif option is missing in the list i.e. standard HCHP.

Might be an explanation ?

[vk496]

The problem is with standard because there are pretty more attributes that in historique. But the issue is the same, just different scale

[antoinelibert]

It's legacy_entity_attributes parameter of Z2M. Set it to False and reboot Z2M and HA and it should be good.

https://www.zigbee2mqtt.io/guide/configuration/homeassistant.html#advanced-configuration

Edit: might remove some attributes of other devices unfortunately ...

[capi92]

Excellent, @antoinelibert thanks !!!!

This solved the problem.

[vk496]

I've tried @antoinelibert suggestion, but I'm still have all values as attribute of the entity. @capi92 could you please share your configuration.yaml of Z2M? Maybe I'm missing something...

[capi92]

@vk496

configuration.yaml

mqtt:
  broker: 127.0.0.1

[capi92]

Finally, after few hours, behavior remains the same...

I restarted both Z2M and HA to see if it's better now (did not force restart Z2M initially , Z2M being a container I was thinking it would be restated together with HA when asking for HA to restart).

[capi92]

Workaround for the time being, create a new sensor with only the required attribute:

sensor.yaml

- platform: template
  sensors:
    conso_hchc:
      friendly_name: "Conso HCHC"
      unit_of_measurement: kWh
      device_class: energy
      value_template: "{{ states('sensor.linky_easd01') }}"

    conso_hchp:
      friendly_name: "Conso HCHP"
      unit_of_measurement: kWh
      device_class: energy
      value_template: "{{ states('sensor.linky_easd02') }}"

[capi92]

Working with solution above. Force Z2M restart and HA to ensure parameter are took into account.

[kevincaradant]

Hi everyone,

Just a little question, do you know if it's possible to reduce the fields answers on this device from Z2M to avoid to overload the coordinator and also HA ( DB ) ?

I have this

But I would remove all this fields that I think are useless ?

current_tiers_2_summ_delievered
current_tiers_3_summ_delievered
current_tiers_4_summ_delievered
until 
current_tiers_10_summ_delievered
message1 and message2 also

Do you know if it's possible in Z2M to avoid the report of them ?

My config:

Thanks

[antoinelibert]

Hello,

A week ago, I have added a new setting to filter the number of exposed values. You would need to upgrade to a more recent DEV version of z2m first. This setting acts as a "whitelist" of Linky TIC command to show. Like this:

[kevincaradant]

@antoinelibert, oh I'm running in the 1.24.0 version. You said that I need to run on the dev branch?

Your options seem nice, I will throw an eye on it thanks! 😁

[kevincaradant]

@antoinelibert , unfortunatly the message send on the network still contains all data but I guess we can't avoid it ? Indeed we can limit the exposed fields for HA ect, it's a good point :) , but I expected also to avoid the message with all data as payload :). I guess it's the Lixee it self which send these data so we can't limit it.

If I switch to "Standard mode" insteaf of Historic, it's worse because there are a lot of more fields :/

Historic mode:

MQTT publish: topic 'zigbee2mqtt/linky', payload '{"active_power":-32768,"active_power_max":-32768,"active_power_ph_b":-32768,"active_register_tier_delivered":"TH..","apparent_power":440,"available_power":30,"current_date":"","current_price":"","current_summ_delivered":8452.3,"current_tarif":"BASE","current_tier10_summ_delivered":0,"current_tier1_summ_delivered":8443.1,"current_tier2_summ_delivered":0,"current_tier3_summ_delivered":0,"current_tier4_summ_delivered":0,"current_tier5_summ_delivered":0,"current_tier6_summ_delivered":0,"current_tier7_summ_delivered":0,"current_tier8_summ_delivered":0,"current_tier9_summ_delivered":0,"linkquality":120,"message1":"","message2":"","meter_serial_number":"xxxx","rms_current":2,"rms_current_max":90,"rms_voltage":65535,"site_id":"","status_register":"","update":{"state":"idle"},"update_available":false,"warn_d_p_s":null}'

[antoinelibert]

You have to remove the cached value in state.json of z2m. Either edit the file or remove it completely and z2m will recreate it with only the needed values. and dont forget to hit the "Reconfigure" button of the device so the reporting get corrected.

[SilentT-FR]

You have just to disable HomeAssistant legacy attribute : https://www.zigbee2mqtt.io/guide/configuration/homeassistant.html legacy_entity_attributes: false or via GUI

and reboot Z2M & HA

if you dont want avoid to see certain value, you just need to disable the value in HA

[kevincaradant]

Thanks for yours answers, I combined both of them so I don't know what really works but it's good now :)

Last question (Off Topic) sorry, does someone have the price in euro as attribute ? I mean I'm not yet in 'Standard mode' but someone told me I need to be in 'standard' mode to get the price ( consumption ) at home. I'm running in historic mode with this:

{
    "active_register_tier_delivered": "TH..",
    "apparent_power": 610,
    "available_power": 30,
    "current_summ_delivered": 8464,
    "current_tarif": "BASE",
    "linkquality": 102,
    "meter_serial_number": "xxxxxxx",
    "rms_current": 2,
    "rms_current_max": 90,
    "update": {
        "state": "idle"
    },
    "update_available": false,
    "warn_d_p_s": null
}

I thought in current_tarif to see the price in KWh. Not the 'BASE' text :)

But even with examples saw on others threads, I'm not sure to see the price displayed somewhere. Do you have this ?

The goal is to have something like that ( from Saniho plugin using Enedis api ) but refresh every minute for example :D (The Enedis API works not everytime that's why also I want something from the linky it self)

Thanks

EDIT: OK in fact after to edit my configuration.yaml under HA, I saw the Energy dashboard, that's answer to my question about price ect. It's working as expected.

[vk496]

does someone have the price in euro as attribute ?

That is not reported by Linky. Linky is related to Enedis stuff, not your electricity company (which bills you). What is reported (and make it useful) is what tarif you are currently. With that information, you could associate prices under HA with automatizations, so your current price would change based on what is reported from Linky

[kevincaradant]

@vk496 hi again,

Yes thank you, that's what I understood later. I had edited my previous answer but maybe I was not clear. Indeed, using the energy dashboard, I got my consumption in kwh and in euro thanks to data from linky and the plugin. It's perfect, sorry for my previous misunderstanding about the price. Regards