search

UtilitechAS / amsreader-firmware

ESP8266 and ESP32 compatible firmware to read, interpret and publish data to MQTT from smart electrical meters, both DLMS and DSMR is supported
Other
381 stars 73 forks source link

No data is displayed/decoded although connection to meter is OK #615

Closed TRIROG closed 1 year ago

TRIROG commented 1 year ago

Describe your problem no data is displayed although DLMS is received

Telnet debug output:

DLMS
No HAN data received last 30s, single blink
HDLC frame:
7E A0 81 CF  02 23 13 85  E1 E6 E7 00  0F 00 00 0D
7E 0C 07 E7  07 18 01 13  28 19 00 FF  88 80 02 12
09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
30 36 09 08  31 36 38 32  30 30 30 35  06 00 00 05
71 06 00 00  00 00 06 00  00 00 00 06  00 00 02 E2
12 09 4B 12  09 4B 12 08  F8 12 00 67  12 00 7D 12
02 55 06 00  00 00 10 06  00 00 00 33  06 00 00 05
2D 06 00 00  00 00 06 00  00 00 00 06  00 00 00 00
DF 42 7E
LLC frame:
E6 E7 00 0F  00 00 0D 7E  0C 07 E7 07  18 01 13 28
19 00 FF 88  80 02 12 09  10 49 53 4B  31 30 33 30
37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
30 30 35 06  00 00 05 71  06 00 00 00  00 06 00 00
00 00 06 00  00 02 E2 12  09 4B 12 09  4B 12 08 F8
12 00 67 12  00 7D 12 02  55 06 00 00  00 10 06 00
00 00 33 06  00 00 05 2D  06 00 00 00  00 06 00 00
00 00 06 00  00 00 00
DLMS frame:
0F 00 00 0D  7E 0C 07 E7  07 18 01 13  28 19 00 FF
88 80 02 12  09 10 49 53  4B 31 30 33  30 37 38 39
33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 35
06 00 00 05  71 06 00 00  00 00 06 00  00 00 00 06
00 00 02 E2  12 09 4B 12  09 4B 12 08  F8 12 00 67
12 00 7D 12  02 55 06 00  00 00 10 06  00 00 00 33
06 00 00 05  2D 06 00 00  00 00 06 00  00 00 00 06
00 00 00 00
Received valid DLMS at 30
Using application data:
02 12 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
35 36 30 36  09 08 31 36  38 32 30 30  30 35 06 00
00 05 71 06  00 00 00 00  06 00 00 00  00 06 00 00
02 E2 12 09  4B 12 09 4B  12 08 F8 12  00 67 12 00
7D 12 02 55  06 00 00 00  10 06 00 00  00 33 06 00
00 05 2D 06  00 00 00 00  06 00 00 00  00 06 00 00
00 00
DLMS

Hardware information:

Relevant firmware information:

gskjold commented 1 year ago

This looks like a proprietary payload. I can sort of guess the fields here, but if you have some documentation from your grid company, that would be best. In any case, I can make this work for you.

TRIROG commented 1 year ago

Would this help? IMG_7012 Push 5sec interval IMG_7010 Push 15 min interval IMG_7013

EDIT: Added HDLC settings

IMG_7016

gskjold commented 1 year ago

Yes, this is exactly what I am looking for. In your original post there is only a dump of the 5s frame, are you also able to capture the 15m frame?

TRIROG commented 1 year ago

This should be it:

No HAN data received last 30s, single blink
Serving /data.json over http...
HDLC frame:
7E A0 6B CF  02 23 13 CB  86 E6 E7 00  0F 00 00 30
EB 0C 07 E7  07 19 02 09  0F 00 00 FF  88 80 02 0C
09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
30 36 09 08  31 36 38 32  30 30 30 36  16 01 09 02
00 01 06 00  00 CE 65 06  00 00 35 19  06 00 00 99
4C 06 00 02  B7 07 06 00  01 33 17 06  00 01 83 F0
06 00 01 A5  6A 06 00 00  A8 F5 5D B7  7E
LLC frame:
E6 E7 00 0F  00 00 30 EB  0C 07 E7 07  19 02 09 0F
00 00 FF 88  80 02 0C 09  10 49 53 4B  31 30 33 30
37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
30 30 36 16  01 09 02 00  01 06 00 00  CE 65 06 00
00 35 19 06  00 00 99 4C  06 00 02 B7  07 06 00 01
33 17 06 00  01 83 F0 06  00 01 A5 6A  06 00 00 A8
F5
DLMS frame:
0F 00 00 30  EB 0C 07 E7  07 19 02 09  0F 00 00 FF
88 80 02 0C  09 10 49 53  4B 31 30 33  30 37 38 39
33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 36
16 01 09 02  00 01 06 00  00 CE 65 06  00 00 35 19
06 00 00 99  4C 06 00 02  B7 07 06 00  01 33 17 06
00 01 83 F0  06 00 01 A5  6A 06 00 00  A8 F5
Received valid DLMS at 30
Using application data:
02 0C 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
35 36 30 36  09 08 31 36  38 32 30 30  30 36 16 01
09 02 00 01  06 00 00 CE  65 06 00 00  35 19 06 00
00 99 4C 06  00 02 B7 07  06 00 01 33  17 06 00 01
83 F0 06 00  01 A5 6A 06  00 00 A8 F5
DLMS
TRIROG commented 1 year ago

Here are the definitions based on OBIS:

5 seconds

1.7.0 kW IMPORT Positive active instantaneous power (A+) [kW] 2.7.0 kW EXPORT Negative active instantaneous power (A-) [kW] 3.7.0 kvar IMPORT Positive reactive instantaneous power (Q+) [kvar] 4.7.0 kvar EXPORT Negative reactive instantaneous power (Q-) [kvar] 32.7.0 V Instantaneous voltage (U) in phase L1 [V] 52.7.0 V Instantaneous voltage (U) in phase L2 [V] 72.7.0 V Instantaneous voltage (U) in phase L3 [V] 31.7.0 A Instantaneous current (I) in phase L1 [A] 51.7.0 A Instantaneous current (I) in phase L2 [A] 71.7.0 A Instantaneous current (I) in phase L3 [A] 21.7.0 kW IMPORT Positive active instantaneous power (A+) in phase L1 [kW] 41.7.0 kW IMPORT Positive active instantaneous power (A+) in phase L2 [kW] 61.7.0 kW IMPORT Positive active instantaneous power (A+) in phase L3 [kW] 22.7.0 kW EXPORT Negative active instantaneous power (A-) in phase L1 [kW] 42.7.0 kW EXPORT Negative active instantaneous power (A-) in phase L2 [kW] 62.7. kW EXPORT Negative active instantaneous power (A-) in phase L3 [kW]

15 minute 96.3.10 Disconnect control 96.14.0 Currently active energy tariff 1.8.0 Wh IMPORT Positive active energy (A+) total [kWh] 1.8.1 Wh IMPORT Positive active energy (A+) in tariff T1 [kWh] 1.8.2 Wh IMPORT Positive active energy (A+) in tariff T2 [kWh] 2.8.0 Wh EXPORT Negative active energy (A+) total [kWh] 2.8.1 Wh EXPORT Negative active energy (A+) in tariff T1 [kWh] 2.8.2 Wh EXPORT Negative active energy (A+) in tariff T2 [kWh] 3.8.0 kvarh IMPORT Positive reactive energy (Q+) total [kvarh] 4.8.0 kvarh EXPORT Negative reactive energy (Q-) total [kvarh]

gskjold commented 1 year ago

Thanks, perfect! I will let you know when I have a test firmware for you

TRIROG commented 1 year ago

Is there any way to include reading timestamp to JSON message?

for above i see 

<DataNotification>
  <LongInvokeIdAndPriority Value="12523" />
  <!--2023-07-25 09:15:00-->
  <DateTime Value="07E7071902090F0000FF8880" />
  <NotificationBody>
    <DataValue>
      <Structure Qty="12" >
        <!--ISK1030789375606-->
        <OctetString Value="49534B31303330373839333735363036" />
        <!--16820006-->
        <OctetString Value="3136383230303036" />
        <Enum Value="1" />
        <OctetString Value="0001" />
        <UInt32 Value="52837" />
        <UInt32 Value="13593" />
        <UInt32 Value="39244" />
        <UInt32 Value="177927" />
        <UInt32 Value="78615" />
        <UInt32 Value="99312" />
        <UInt32 Value="107882" />
        <UInt32 Value="43253" />
      </Structure>
    </DataValue>
  </NotificationBody>
</DataNotification>
gskjold commented 1 year ago

Yes, this will be included

gskjold commented 1 year ago

Please test attached firmware and report back to me esp32.zip esp32c3.zip esp32s2.zip esp32solo.zip esp8266.zip

TRIROG commented 1 year ago

first off import data is off and L1 VOltage and currents are missing in GUI image

the 5s readout

(W) No HAN data received last 30s, single blink
(D) Serving /data.json over http...
(V) HDLC frame:
(V) 7E A0 81 CF  02 23 13 85  E1 E6 E7 00  0F 00 01 A1
(V) 2F 0C 07 E7  07 1E 07 13  33 1E 00 FF  88 80 02 12
(V) 09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
(V) 30 36 09 08  31 36 38 32  30 30 30 35  06 00 00 03
(V) C8 06 00 00  00 3C 06 00  00 00 00 06  00 00 02 ED
(V) 12 09 5D 12  09 15 12 09  2C 12 00 6C  12 00 82 12
(V) 01 B8 06 00  00 00 00 06  00 00 00 00  06 00 00 03
(V) C8 06 00 00  00 15 06 00  00 00 26 06  00 00 00 00
(V) E9 AF 7E
(V) LLC frame:
(V) E6 E7 00 0F  00 01 A1 2F  0C 07 E7 07  1E 07 13 33
(V) 1E 00 FF 88  80 02 12 09  10 49 53 4B  31 30 33 30
(V) 37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
(V) 30 30 35 06  00 00 03 C8  06 00 00 00  3C 06 00 00
(V) 00 00 06 00  00 02 ED 12  09 5D 12 09  15 12 09 2C
(V) 12 00 6C 12  00 82 12 01  B8 06 00 00  00 00 06 00
(V) 00 00 00 06  00 00 03 C8  06 00 00 00  15 06 00 00
(V) 00 26 06 00  00 00 00
(V) DLMS frame:
(V) 0F 00 01 A1  2F 0C 07 E7  07 1E 07 13  33 1E 00 FF
(V) 88 80 02 12  09 10 49 53  4B 31 30 33  30 37 38 39
(V) 33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 35
(V) 06 00 00 03  C8 06 00 00  00 3C 06 00  00 00 00 06
(V) 00 00 02 ED  12 09 5D 12  09 15 12 09  2C 12 00 6C
(V) 12 00 82 12  01 B8 06 00  00 00 00 06  00 00 00 00
(V) 06 00 00 03  C8 06 00 00  00 15 06 00  00 00 26 06
(V) 00 00 00 00
(D) Received valid DLMS at 30
(V) Using application data:
(V) 02 12 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
(V) 35 36 30 36  09 08 31 36  38 32 30 30  30 35 06 00
(V) 00 03 C8 06  00 00 00 3C  06 00 00 00  00 06 00 00
(V) 02 ED 12 09  5D 12 09 15  12 09 2C 12  00 6C 12 00
(V) 82 12 01 B8  06 00 00 00  00 06 00 00  00 00 06 00
(V) 00 03 C8 06  00 00 00 15  06 00 00 00  26 06 00 00
(V) 00 00
(V) DLMS
(V) (EnergyAccounting)  calculating threshold, currently at 0
(V) (EnergyAccounting)  new threshold 0

The MQTT JSON is this

{"id":"BC:DD:C2:7A:CE:F3","name":"AM550_2_MQTT","up":405,"t":1690746910,"vcc":3.211,"rssi":-81,"temp":-127.00,"data":{"lv":"ISK1030789375606","id":"ISK1030789375606","type":"","P":137442872,"Q":32,"PO":1190,"QO":0,"I1":23.40,"I2":1.08,"I3":1.30,"U1":0.00,"U2":238.70,"U3":236.80},"realtime":{"h":0.00,"d":0.0,"t":5,"x":0.00,"he":0.00,"de":0.0}}
TRIROG commented 1 year ago

Here is the 15 minute update followed by 5 second update and both JSON outputs

No HAN data received last 30s, single blink
HDLC frame:
7E A0 6B CF  02 23 13 CB  86 E6 E7 00  0F 00 01 A2
4A 0C 07 E7  07 1E 07 14  0F 00 00 FF  88 80 02 0C
09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
30 36 09 08  31 36 38 32  30 30 30 36  16 01 09 02
00 02 06 00  01 B6 E3 06  00 00 96 05  06 00 01 20
DE 06 00 07  E2 6D 06 00  04 85 B6 06  00 03 5C B7
06 00 04 E0  F8 06 00 01  70 04 A7 EB  7E
LLC frame:
E6 E7 00 0F  00 01 A2 4A  0C 07 E7 07  1E 07 14 0F
00 00 FF 88  80 02 0C 09  10 49 53 4B  31 30 33 30
37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
30 30 36 16  01 09 02 00  02 06 00 01  B6 E3 06 00
00 96 05 06  00 01 20 DE  06 00 07 E2  6D 06 00 04
85 B6 06 00  03 5C B7 06  00 04 E0 F8  06 00 01 70
04
DLMS frame:
0F 00 01 A2  4A 0C 07 E7  07 1E 07 14  0F 00 00 FF
88 80 02 0C  09 10 49 53  4B 31 30 33  30 37 38 39
33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 36
16 01 09 02  00 02 06 00  01 B6 E3 06  00 00 96 05
06 00 01 20  DE 06 00 07  E2 6D 06 00  04 85 B6 06
00 03 5C B7  06 00 04 E0  F8 06 00 01  70 04
Received valid DLMS at 30
Using application data:
02 0C 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
35 36 30 36  09 08 31 36  38 32 30 30  30 36 16 01
09 02 00 02  06 00 01 B6  E3 06 00 00  96 05 06 00
01 20 DE 06  00 07 E2 6D  06 00 04 85  B6 06 00 03
5C B7 06 00  04 E0 F8 06  00 01 70 04
DLMS
(AmsDataStorage) Day data timestamp age 1690740904 - 1690736400 > 3600
Its time to update data storage
(EnergyAccounting)  calculating threshold, currently at 0
(EnergyAccounting)  new threshold 0
HDLC frame:
7E A0 81 CF  02 23 13 85  E1 E6 E7 00  0F 00 01 A2
4B 0C 07 E7  07 1E 07 14  0F 00 00 FF  88 80 02 12
09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
30 36 09 08  31 36 38 32  30 30 30 35  06 00 00 05
6E 06 00 00  00 00 06 00  00 00 00 06  00 00 03 5A
12 09 53 12  09 1B 12 09  17 12 00 6B  12 00 88 12
02 51 06 00  00 00 37 06  00 00 00 13  06 00 00 05
22 06 00 00  00 00 06 00  00 00 00 06  00 00 00 00
9D A4 7E
LLC frame:
E6 E7 00 0F  00 01 A2 4B  0C 07 E7 07  1E 07 14 0F
00 00 FF 88  80 02 12 09  10 49 53 4B  31 30 33 30
37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
30 30 35 06  00 00 05 6E  06 00 00 00  00 06 00 00
00 00 06 00  00 03 5A 12  09 53 12 09  1B 12 09 17
12 00 6B 12  00 88 12 02  51 06 00 00  00 37 06 00
00 00 13 06  00 00 05 22  06 00 00 00  00 06 00 00
00 00 06 00  00 00 00
DLMS frame:
0F 00 01 A2  4B 0C 07 E7  07 1E 07 14  0F 00 00 FF
88 80 02 12  09 10 49 53  4B 31 30 33  30 37 38 39
33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 35
06 00 00 05  6E 06 00 00  00 00 06 00  00 00 00 06
00 00 03 5A  12 09 53 12  09 1B 12 09  17 12 00 6B
12 00 88 12  02 51 06 00  00 00 37 06  00 00 00 13
06 00 00 05  22 06 00 00  00 00 06 00  00 00 00 06
00 00 00 00
Received valid DLMS at 30
Using application data:
02 12 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
35 36 30 36  09 08 31 36  38 32 30 30  30 35 06 00
00 05 6E 06  00 00 00 00  06 00 00 00  00 06 00 00
03 5A 12 09  53 12 09 1B  12 09 17 12  00 6B 12 00
88 12 02 51  06 00 00 00  37 06 00 00  00 13 06 00
00 05 22 06  00 00 00 00  06 00 00 00  00 06 00 00
00 00
DLMS
(AmsDataStorage) Day data timestamp age 1690740905 - 1690736400 > 3600
Its time to update data storage
(EnergyAccounting)  calculating threshold, currently at 0
(EnergyAccounting)  new threshold 0

And JSON OUTPUT

{"id":"BC:DD:C2:7A:CE:F3","name":"AM550_2_MQTT","up":900,"vcc":3.205,"rssi":-82,"temp":-127.00,"version":"e801602"}
{"id":"BC:DD:C2:7A:CE:F3","name":"AM550_2_MQTT","up":900,"t":1690748100,"vcc":3.207,"rssi":-80,"temp":-127.00,"data":{"lv":"ISK1030789375606","id":"","type":"","P":0,"Q":0,"PO":0,"QO":0,"I1":0.00,"I2":0.00,"I3":0.00,"U1":0.00,"U2":0.00,"U3":0.00,"tPI":33554.950,"tPO":73.950,"tQI":220.343,"tQO":319.736,"rtc":0},"realtime":{"h":0.00,"d":0.0,"t":5,"x":0.00,"he":0.00,"de":0.0}}
{"id":"BC:DD:C2:7A:CE:F3","name":"AM550_2_MQTT","up":901,"t":1690748100,"vcc":3.205,"rssi":-82,"temp":-127.00,"data":{"lv":"ISK1030789375606","id":"ISK1030789375606","type":"","P":137442872,"Q":0,"PO":1390,"QO":0,"I1":23.27,"I2":1.07,"I3":1.36,"U1":0.00,"U2":238.70,"U3":233.10},"realtime":{"h":0.00,"d":0.0,"t":5,"x":0.00,"he":0.00,"de":0.0}}
gskjold commented 1 year ago

Sorry, I misaligned the data for the 5s update. There will still be an issue with 15m data, as the internal structures expect to have receive a full dataset when receiving accumulated counters. But I will adapt this

New version where alignment for 5s should be fixed: esp8266.zip

TRIROG commented 1 year ago

5s updates seem correctly parsed now: ![image](https://github.com/UtilitechAS/amsreader-firmware/assets/10112912/02a965f5-e8f3-4e2d-9677-151e012ed966

{"id":"BC:DD:C2:7A:CE:F3","name":"AM550_2_MQTT","up":61,"t":1690757625,"vcc":3.225,"rssi":-83,"temp":-127.00,"data":{"lv":"ISK1030789375606","id":"16820005","type":"","P":636,"Q":0,"PO":0,"QO":786,"I1":1.00,"I2":1.20,"I3":2.52,"U1":237.40,"U2":234.60,"U3":238.50},"realtime":{"h":0.00,"d":0.0,"t":5,"x":0.00,"he":0.00,"de":0.0}}

telnet debug output:

DLMS
(EnergyAccounting)  calculating threshold, currently at 0
(EnergyAccounting)  new threshold 0
No HAN data received last 30s, single blink
Serving /data.json over http...
HDLC frame:
7E A0 81 CF  02 23 13 85  E1 E6 E7 00  0F 00 01 AA
1D 0C 07 E7  07 1E 07 17  00 37 00 FF  88 80 02 12
09 10 49 53  4B 31 30 33  30 37 38 39  33 37 35 36
30 36 09 08  31 36 38 32  30 30 30 35  06 00 00 02
92 06 00 00  00 00 06 00  00 00 00 06  00 00 03 24
12 09 49 12  09 50 12 09  53 12 00 64  12 00 7A 12
01 07 06 00  00 00 49 06  00 00 00 3C  06 00 00 02
0B 06 00 00  00 00 06 00  00 00 00 06  00 00 00 00
8C D0 7E
LLC frame:
E6 E7 00 0F  00 01 AA 1D  0C 07 E7 07  1E 07 17 00
37 00 FF 88  80 02 12 09  10 49 53 4B  31 30 33 30
37 38 39 33  37 35 36 30  36 09 08 31  36 38 32 30
30 30 35 06  00 00 02 92  06 00 00 00  00 06 00 00
00 00 06 00  00 03 24 12  09 49 12 09  50 12 09 53
12 00 64 12  00 7A 12 01  07 06 00 00  00 49 06 00
00 00 3C 06  00 00 02 0B  06 00 00 00  00 06 00 00
00 00 06 00  00 00 00
DLMS frame:
0F 00 01 AA  1D 0C 07 E7  07 1E 07 17  00 37 00 FF
88 80 02 12  09 10 49 53  4B 31 30 33  30 37 38 39
33 37 35 36  30 36 09 08  31 36 38 32  30 30 30 35
06 00 00 02  92 06 00 00  00 00 06 00  00 00 00 06
00 00 03 24  12 09 49 12  09 50 12 09  53 12 00 64
12 00 7A 12  01 07 06 00  00 00 49 06  00 00 00 3C
06 00 00 02  0B 06 00 00  00 00 06 00  00 00 00 06
00 00 00 00
Received valid DLMS at 30
Using application data:
02 12 09 10  49 53 4B 31  30 33 30 37  38 39 33 37
35 36 30 36  09 08 31 36  38 32 30 30  30 35 06 00
00 02 92 06  00 00 00 00  06 00 00 00  00 06 00 00
03 24 12 09  49 12 09 50  12 09 53 12  00 64 12 00
7A 12 01 07  06 00 00 00  49 06 00 00  00 3C 06 00
00 02 0B 06  00 00 00 00  06 00 00 00  00 06 00 00
00 00
DLMS
(EnergyAccounting)  calculating threshold, currently at 0
(EnergyAccounting)  new threshold 0

for 15 minute intervals i guess it makes no sense to test.

HAN icon in GUI is still red.

gskjold commented 1 year ago

Ah yeah, I forgot to add the updated timestamp to the data, so for firmware the data looks old. Fixed:

esp32.zip esp32c3.zip esp32s2.zip esp32solo.zip esp8266.zip

TRIROG commented 1 year ago

HAN is now green in GUI.

I did notice however that energy use seems to display inverted data e.g. at 02AM when heatpump was heating up DHW it shows -1,5 kWh use instead of +1,5 kWh. I dont know if that is connected to the 15 min interval parsing or not. image

Do you need me to test anything else for now?

gskjold commented 1 year ago

The graph data is based on the 15min payload, so maybe I have misaligned or swapped the counters

TRIROG commented 1 year ago

Can i help?

gskjold commented 1 year ago

Thanks, but the last change here is some internal structure I have to re-arrange to better support these different formats. Yours is not the first in need of this, so I will need to adapt to more than one format at this point.

Anyway, the 15m payload reading was misaligned, so here we go again, hopefully your kwh counters will match up with what you have on your meter now.

esp8266.zip

TRIROG commented 1 year ago

Uploaded the fix. energy usage seems to count correctly now. Will check the counters in the following day or two and confirm. image

gskjold commented 1 year ago

Returning to this with a modified firmware that should preserve current values (voltage etc) when counters are received from meter every 15m.

esp8266.zip

TRIROG commented 1 year ago

Hi! What needs to be tested in this version?

gskjold commented 1 year ago

Just that voltage etc is not 0 when you receive the 15m payload that contains the accumulated kwh

TRIROG commented 1 year ago

Uploaded , seems to work OK, but can't check thoroughly as i'm using HEX dump output and decode it elswhere.

TRIROG commented 1 year ago

Is the realtime W graph only in this version?

saddfox commented 1 year ago

@TRIROG I am also trying to read AM550 in Slovenia, using a NodeMCU. So far without success. Mind sharing details about your setup & wiring? Thanks

ArnieO commented 1 year ago

Is the realtime W graph only in this version?

Yes, test version only. You can expect realtime power graph to come in v2.3.x .

TRIROG commented 1 year ago

@TRIROG I am also trying to read AM550 in Slovenia, using a NodeMCU. So far without success. Mind sharing details about your setup & wiring? Thanks

Sure, but we might be more effective if you contact me directly or leave contact so that i can get in touch with you.

saddfox commented 1 year ago

@TRIROG I am also trying to read AM550 in Slovenia, using a NodeMCU. So far without success. Mind sharing details about your setup & wiring? Thanks

Sure, but we might be more effective if you contact me directly or leave contact so that i can get in touch with you.

I couldn't find any contact info in your profile. Feel free to reach out to me at mail listed on my profile.

rvetrsek commented 11 months ago

@TRIROG, @saddfox I'm also from Slovenia trying to get this to work for the Iskra AM550. The data seems to be encrypted, but I have the keys. Looking at the debug log I get similar results as @TRIROG, but no data is shown in the graphs and there is a warning message saying that no HAN data was received for 30s in spite of receiving "Received valid DLMS at 46". I'm using a SlimmeLezer+ board btw.

I would be eternally grateful if you could help me out as this has been keeping me up for months at this point 😅

gskjold commented 11 months ago

Assuming you have configured the keys you got, enable debug and set it to VERBOSE, then telnet to your device and send me the final payload from one of the packets, annotated by "DLMS Frame:"

ArnieO commented 11 months ago

Like this: https://amsleser.no/blog/post/24-telnet-debug

rvetrsek commented 11 months ago

Hey, of course, here it is:

(D) Received valid DLMS at 46
(V) Using application data:
(V) 02 09 09 10  49 53 4B 31  30 33 30 37  37 37 34 37
(V) 38 35 38 36  06 00 00 03  1B 06 00 00  00 00 06 00
(V) 4D CE BB 06  00 22 75 EC  06 00 2B 58  CF 06 00 00
(V) 00 AD 06 00  00 00 58 06  00 00 00 55
(V) DLMS
(V) HDLC frame:
(V) 7E A0 77 CF  02 23 13 BB  45 E6 E7 00  DB 08 49 53
(V) 4B 67 74 9E  3A BA 5F 30  00 08 45 FA  09 36 A7 26
(V) 10 7B B0 1F  7E 2B 9A 16  24 6E 83 57  58 E5 D0 19
(V) CF B1 E7 D4  3E FE 3D 19  F8 92 FD 49  E4 2E DA 30
(V) 5A 8A 25 63  A9 E8 80 C3  FF A5 E4 9C  11 14 07 1D
(V) 72 F0 15 01  92 AF 60 D7  E3 47 E1 2B  87 88 41 2A
(V) 8E 74 7D EB  43 C9 79 D6  8A CB 5B 78  87 1A 6A 42
(V) CA D9 E1 6D  B9 37 C3 EE  7E
(V) LLC frame:
(V) E6 E7 00 DB  08 49 53 4B  67 74 9E 3A  BA 5F 30 00
(V) 08 45 FA 09  36 A7 26 10  7B B0 1F 7E  2B 9A 16 24
(V) 6E 83 57 58  E5 D0 19 CF  B1 E7 D4 3E  FE 3D 19 F8
(V) 92 FD 49 E4  2E DA 30 5A  8A 25 63 A9  E8 80 C3 FF
(V) A5 E4 9C 11  14 07 1D 72  F0 15 01 92  AF 60 D7 E3
(V) 47 E1 2B 87  88 41 2A 8E  74 7D EB 43  C9 79 D6 8A
(V) CB 5B 78 87  1A 6A 42 CA  D9 E1 6D B9  37
(V) GCM frame:
(V) DB 08 49 53  4B 67 74 9E  3A BA 5F 30  00 08 45 FA
(V) 0F 00 08 45  FB 0C 07 E7  0A 1C 06 0D  06 30 00 FF
(V) 88 80 02 09  09 10 49 53  4B 31 30 33  30 37 37 37
(V) 34 37 38 35  38 36 06 00  00 03 0E 06  00 00 00 00
(V) 06 00 4D CE  BB 06 00 22  75 EC 06 00  2B 58 CF 06
(V) 00 00 00 AD  06 00 00 00  58 06 00 00  00 55 5B 78
(V) 87 1A 6A 42  CA D9 E1 6D  B9 37
(V) DLMS frame:
(V) 0F 00 08 45  FB 0C 07 E7  0A 1C 06 0D  06 30 00 FF
(V) 88 80 02 09  09 10 49 53  4B 31 30 33  30 37 37 37
(V) 34 37 38 35  38 36 06 00  00 03 0E 06  00 00 00 00
(V) 06 00 4D CE  BB 06 00 22  75 EC 06 00  2B 58 CF 06
(V) 00 00 00 AD  06 00 00 00  58 06 00 00  00 55
(D) Received valid DLMS at 46
TRIROG commented 11 months ago

Who is your provider? As it seems different providers have different settings. I published data for Elektro Ljubljana. But for example Elektro Primorska outputs data in plain text.

rvetrsek commented 11 months ago

It is a distribution area of Elektro Ljubljana, but the supplier is GEN-i if that makes a difference - it probably shouldn’t, but I'm not an expert 😅

TRIROG commented 11 months ago

Distributor is what i meant. Ok, so same settings should work…

TRIROG commented 11 months ago

Recheck the keys for decoding? You know, you could also as Elektro Ljubljana to turn off encryption :)

TRIROG commented 11 months ago

This tool was helpfull to me in debuging and testing: https://www.gurux.fi/GuruxDLMSTranslator I think it also has support for encryption.

rvetrsek commented 11 months ago

Ok, thank you @TRIROG, I'll try and contact them to see if they can turn off the encryption or help me out... Currently I'm using the Global_encryption_unicast_key and the Authentication_key values I got from an XML I was sent by a company that sends you keys for your specific meter based on a serial number you provide, so hopefully they are the right ones, but I'll recheck with the distributor.

In any case the AMS Reader software only says Received valid DLMS at 46 with the mentioned keys...which leads me to believe they are the right ones 🤞

Not exactly sure how to use the mentioned tool, any tips? 🤷‍♂️

gskjold commented 11 months ago

Your keypair is correct, otherwise it would complain. You are getting a decoded payload out of this, but it is not like the one in from @TRIROG. I would need to know the payload format and a longer debug log to implement

svaliv49 commented 11 months ago

Hello all, my first time posting here. Sorry in advance for my beginner-mistakes. @gskjold : great communication/knowledge, thank You!

@TRIROG, @saddfox @rvetrsek - I am from Slovenia too, but my energy disti is ECE Celje. They installed me a a month ago a brand new meter from Iskra AM550. According to the ECE-installer all communication is done through the RJ12 port, push. No authentication. Looks like the same proprietary payload data as the above, slightly less data (attached). For now I am using a generic ESP8266. I get the data (at least it looks like, through telnet getting: Received valid DLMS at 30_ ) But in the AMS Reader UI no data is being updated.
Is all working on Your side? Would You pls be so kind and share?

Will feel so free to paste my telnet debug:

(V) 00 56 06 00 00 08 F6 06 00 00 00 00 06 00 00 00 (V) 61 06 00 00 00 D6 (V) 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 34 37 (V) 33 31 09 05 55 50 5F 33 66 12 09 09 12 08 FF 12 (V) 09 2C 12 01 E3 12 02 0F 12 00 56 06 00 00 09 47 (V) 06 00 00 00 00 06 00 00 00 00 06 00 00 01 19 82 (V) 6B 7E (V) LLC frame: (V) E6 E7 00 0F 00 0B D5 8F 0C 07 E7 0B 01 03 0C 0C (V) 11 00 FF C4 00 02 0C 09 10 49 53 4B 31 30 33 30 (V) 37 38 39 30 31 34 37 33 31 09 05 55 50 5F 33 66 (V) 12 09 09 12 08 FF 12 09 2C 12 01 E3 12 02 0F 12 (V) 00 56 06 00 00 09 47 06 00 00 00 00 06 00 00 00 (V) 00 06 00 00 01 19 (V) DLMS frame: (V) 0F 00 0B D5 8F 0C 07 E7 0B 01 03 0C 0C 11 00 FF (V) C4 00 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 (V) 30 31 34 37 33 31 09 05 55 50 5F 33 66 12 09 09 (V) 12 08 FF 12 09 2C 12 01 E3 12 02 0F 12 00 56 06 (V) 00 00 09 47 06 00 00 00 00 06 00 00 00 00 06 00 (V) 00 01 19 (D) Received valid DLMS at 30 (V) Using application data: (V) 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 (V) 34 37 33 31 09 05 55 50 5F 33 66 12 09 09 12 08 (V) FF 12 09 2C 12 01 E3 12 02 0F 12 00 56 06 00 00 (V) 09 47 06 00 00 00 00 06 00 00 00 00 06 00 00 01 (V) 19 (V) DLMS (V) (EnergyAccounting) calculating threshold, currently at 0 (V) (EnergyAccounting) new threshold 0 (V) HDLC frame: (V) 7E A0 60 CF 02 23 13 27 C1 E6 E7 00 0F 00 0B D5 (V) 90 0C 07 E7 0B 01 03 0C 0C 12 00 FF C4 00 02 0C (V) 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 34 37 (V) 33 31 09 05 55 50 5F 33 66 12 09 0A 12 08 FF 12 (V) 09 2C 12 01 C3 12 02 0D 12 00 58 06 00 00 09 5F (V) 06 00 00 00 00 06 00 00 00 04 06 00 00 01 08 BC (V) 94 7E (V) LLC frame: (V) E6 E7 00 0F 00 0B D5 90 0C 07 E7 0B 01 03 0C 0C (V) 12 00 FF C4 00 02 0C 09 10 49 53 4B 31 30 33 30 (V) 37 38 39 30 31 34 37 33 31 09 05 55 50 5F 33 66 (V) 12 09 0A 12 08 FF 12 09 2C 12 01 C3 12 02 0D 12 (V) 00 58 06 00 00 09 5F 06 00 00 00 00 06 00 00 00 (V) 04 06 00 00 01 08 (V) DLMS frame: (V) 0F 00 0B D5 90 0C 07 E7 0B 01 03 0C 0C 12 00 FF (V) C4 00 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 (V) 30 31 34 37 33 31 09 05 55 50 5F 33 66 12 09 0A (V) 12 08 FF 12 09 2C 12 01 C3 12 02 0D 12 00 58 06 (V) 00 00 09 5F 06 00 00 00 00 06 00 00 00 04 06 00 (V) 00 01 08 (D) Received valid DLMS at 30 (V) Using application data: (V) 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 (V) 34 37 33 31 09 05 55 50 5F 33 66 12 09 0A 12 08 (V) FF 12 09 2C 12 01 C3 12 02 0D 12 00 58 06 00 00 (V) 09 5F 06 00 00 00 00 06 00 00 00 04 06 00 00 01 (V) 08 (V) DLMS (V) (EnergyAccounting) calculating threshold, currently at 0 (V) (EnergyAccounting) new threshold 0 (D) Serving /data.json over http... (V) HDLC frame: (V) 7E A0 60 CF 02 23 13 27 C1 E6 E7 00 0F 00 0B D5 (V) 91 0C 07 E7 0B 01 03 0C 0C 13 00 FF C4 00 02 0C (V) 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 34 37 (V) 33 31 09 05 55 50 5F 33 66 12 09 0A 12 08 FF 12 (V) 09 2B 12 01 CB 12 02 0E 12 00 54 06 00 00 09 60 (V) 06 00 00 00 00 06 00 00 00 01 06 00 00 01 0C 60 (V) 64 7E (V) LLC frame: (V) E6 E7 00 0F 00 0B D5 91 0C 07 E7 0B 01 03 0C 0C (V) 13 00 FF C4 00 02 0C 09 10 49 53 4B 31 30 33 30 (V) 37 38 39 30 31 34 37 33 31 09 05 55 50 5F 33 66 (V) 12 09 0A 12 08 FF 12 09 2B 12 01 CB 12 02 0E 12 (V) 00 54 06 00 00 09 60 06 00 00 00 00 06 00 00 00 (V) 01 06 00 00 01 0C (V) DLMS frame: (V) 0F 00 0B D5 91 0C 07 E7 0B 01 03 0C 0C 13 00 FF (V) C4 00 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 (V) 30 31 34 37 33 31 09 05 55 50 5F 33 66 12 09 0A (V) 12 08 FF 12 09 2B 12 01 CB 12 02 0E 12 00 54 06 (V) 00 00 09 60 06 00 00 00 00 06 00 00 00 01 06 00 (V) 00 01 0C (D) Received valid DLMS at 30 (V) Using application data: (V) 02 0C 09 10 49 53 4B 31 30 33 30 37 38 39 30 31 (V) 34 37 33 31 09 05 55 50 5F 33 66 12 09 0A 12 08 (V) FF 12 09 2B 12 01 CB 12 02 0E 12 00 54 06 00 00 (V) 09 60 06 00 00 00 00 06 00 00 00 01 06 00 00 01 (V) 0C (V) DLMS (V) (EnergyAccounting) calculating threshold, currently at 0 (V) (EnergyAccounting) new threshold 0

TRIROG commented 11 months ago

Your data can be decoded with doma tanslator so it seems valid:

<HDLC len="95" >
<TargetAddress Value="103" />
<SourceAddress Value="145" />
<!--S frame.-->
<FrameType Value="13" />
<PDU>
<DataNotification>
  <LongInvokeIdAndPriority Value="775569" />
  <!--2023-11-01 12:12:19-->
  <DateTime Value="07E70B01030C0C1300FFC400" />
  <NotificationBody>
    <DataValue>
      <Structure Qty="12" >
        <!--ISK1030789014731-->
        <OctetString Value="49534B31303330373839303134373331" />
        <!--UP_3f-->
        <OctetString Value="55505F3366" />
        <UInt16 Value="2314" />
        <UInt16 Value="2303" />
        <UInt16 Value="2347" />
        <UInt16 Value="459" />
        <UInt16 Value="526" />
        <UInt16 Value="84" />
        <UInt32 Value="2400" />
        <UInt32 Value="0" />
        <UInt32 Value="1" />
        <UInt32 Value="268" />
      </Structure>
    </DataValue>
  </NotificationBody>
</DataNotification>
</PDU>
</HDLC>

This is the second message push - every 5secs? Its just a question of wheter they have standardized the messages to be parsed with amsreader or code will need to be adjusted for each distributor (which would suck)…

ArnieO commented 11 months ago

Its just a question of wheter they have standardized the messages to be parsed with amsreader or code will need to be adjusted for each distributor (which would suck)…

It would definitively be a disadvantage if each grid company in your country has a different message format. Can one of you guys try to find out if there is some kind of standardization?

svaliv49 commented 11 months ago

Your data can be decoded with doma tanslator so it seems valid:

<HDLC len="95" >
<TargetAddress Value="103" />
<SourceAddress Value="145" />
<!--S frame.-->
<FrameType Value="13" />
<PDU>
<DataNotification>
  <LongInvokeIdAndPriority Value="775569" />
  <!--2023-11-01 12:12:19-->
  <DateTime Value="07E70B01030C0C1300FFC400" />
  <NotificationBody>
    <DataValue>
      <Structure Qty="12" >
        <!--ISK1030789014731-->
        <OctetString Value="49534B31303330373839303134373331" />
        <!--UP_3f-->
        <OctetString Value="55505F3366" />
        <UInt16 Value="2314" />
        <UInt16 Value="2303" />
        <UInt16 Value="2347" />
        <UInt16 Value="459" />
        <UInt16 Value="526" />
        <UInt16 Value="84" />
        <UInt32 Value="2400" />
        <UInt32 Value="0" />
        <UInt32 Value="1" />
        <UInt32 Value="268" />
      </Structure>
    </DataValue>
  </NotificationBody>
</DataNotification>
</PDU>
</HDLC>

This is the second message push - every 5secs? Its just a question of wheter they have standardized the messages to be parsed with amsreader or code will need to be adjusted for each distributor (which would suck)…

Thank You @TRIROG - looks like I have to dive into DLSMTranslator lessons :) @ArnieO "...grid company in your country has a different message format..." - that's something I am going to find out, hopefully, next week after the national days-off. Thank You all!

TRIROG commented 11 months ago

The order in which the data is presented is definitly different than was is expected as itndiffers from what elektro ljubljana is sending. You can circumvent this limitation by using RAW data output and decoding the data with a simple python script using DSLM library and sending it via MQTT to you HA system. Of course you won’t have the visualisation the AMSreader offers but at lest you’ll have the data.

alternative is to ask elektro Celje to change the AM500 message settings or to wait to have amsreader updated to supoort elektro Celje message format - the question is does the message provide the identifier to determine the distributor (elektro celje/elektro ljubljana…)

ArnieO commented 11 months ago

the question is does the message provide the identifier to determine the distributor (elektro celje/elektro ljubljana…)

Yes. And another question is: How many local message variants are there for this meter? If the answer is "very many" it is not evident that we can continue adding to the firmware. 😑

svaliv49 commented 11 months ago

If the answer is "very many" it is not evident that we can continue adding to the firmware.

all good, reasonable, understandable. Lets see next week if I get the info.

saddfox commented 11 months ago

This should all be covered by article 177 of the current legislation, if my reading is correct. Specifically, message format should conform to SIST EN62056-7-5 standard, which I believe should be identical to EN 62056-7-5:2016. It would also seem that companies don't actually follow it.

svaliv49 commented 11 months ago

HI All; I did some calls today - and its true. Each of the grid disti's has their own variant. So practically approx. 9 different versions in Slovenia. I will try to talk to my contact in Elektro Celje to get the same config as @TRIROG - will feedback.

gskjold commented 11 months ago

That is unfortunate... But I will keep this in mind in case I can think of a good way to support this

rvetrsek commented 11 months ago

Well that sucks...is there another way I could get my consumption of electricity into HA for the time being?