DSMR - Polish meters - Githubissues

mich909 commented 20 hours ago

The problem

I tried to use dsmr component with polish meter Elgama G15. According to energy distributor it is compliant with P1 companion 5.0.2. The plain communication in hex format looks like this:

2F 45 47 4D 35 47 31 35 0D 0A 
0D 0A
00 82 02 30 30 00 00 00 00 71 D5 81 85 DD 63 F2 52 7D E9 B5 22 34 4B 39 DD 03 71 36 53 A8 96 9A 34 EA 9C 13 F5 E9 B7 EA 83 4F C5 28 63 CD 0E 77 8C C7 F1 84 0A
20 03 EE 27 3D E7 FF A9 36 9E 11 FE A5 01 7F 84 9B 80 DE 54 0B 68 FE 81 15 E3 0E E6 6D 28 A8 BA 21 A4 9F 39 9E 98 C5 28 C4 28 B7 46 75 EF 37 13 63 C8 53 D1 96 2A 8B 69 0D E2 C0 EE F5 77 A7 D8 8E 4C ED 05 93 A5 E6 B1 9D 50 26 58 79 F2 69 1B C3 7F 1E F4 D6 EF C3 32 6D 65 3E 99 4C E4 96 D9 FA 4D 2E 1D 11 8F B8 72 4A F0 D1 4C 59 67 CC A5 F9 5D D1 5F AE 49 19 9A D4 E6 C6 33 C4 13 58 6B D3 64 AA 58 E9 8F D8 0E 61 DF 7A 1B FB 4B 73 FA 51 6D 9F D6 83 9D
DD A6 A4 99 F0 F8 3C 49 CC DD F5 D9 02 AF C6 C4 88 B0 19 D8 56 35 80 23 93 61 1F 72 33 F2 2A C7 B1 4C 04 BD 83 AB EA CD 59 7C 66 D1 24 A2 6D E1 4C 1A 05 77 9E CE C0 08 8B BE 9A CF 20 8F 72 4A 7C 51 AA 0B D5 70 B7 AA 18 48 95 45 02 CF 37 B0 85 D4 B4 29 14 54 F3 81 C7 B1 FA F3 72 2F 6B EC 67 E5 2F 99 52 EA 7E 45 83 92 6B E2 25 B3 D8 CB CE 6E DD 7A 7C CB 86 1B 8C E4 A0 07 BE CE C5 73 72 FF 39 FC B1 56 87 DF D0 42 4F B8 C1 57 2A 36 22 35 CF 14 6E 75
09 60 89 3B 9A D2 E2 27 A9 A2 45 8B 18 E8 7A 4E 4E 9F 5F AD 6A 0E A4 3B C6 AA 79 79 89 9E 78 49 6D 82 6E 85 1B AC 08 48 68 0E A6 52 95 E4 16 DF 8D EC 19 C8 AF B7 46 A2 8E 84 31 AF AA D4 FD 59 B4 63 8C 1C 91 79 93 E1 5A 9F 4B 3D B2 7A 78 00 EE CA 74 11 A5 BF B7 FE 2E 65 2E FF B9 4E 34 83 40 81 30 2D 86 99 5C 82 0D E1 C2 68 10 10 A5 DE F4 41 55 59 B7 A6 DB D7 8C CB 2B 33 0F D8 D3 56 04 9D 82 D6 4B 58 F1 11 E6 E7 94 2D 63 3F 79 73 09 A6 3F AF D6 1D
C4 EF 82 D0 F8 84 1F 09 47 C1 9A AF 63 9A 46 24 24 DF 0E 5B 3D A3 9A D2 9A 85 D8 39 6D D1 18 96 38 45 43 87 6B 05 47 92 45 E6 63 AD D9 44 F8 34 A8 91 F0 2F 02 B1 4F B9 8F 78 87 95 55 21 41 37 30 45 0D 0A

As it can be seen, it has structure as P1 standard state (the header, \r\n, data, and !CRC) however the data is encrypted. I have encryption key but it seems that the bytes corresponding to IV or telegram length are in different places (comparing to the source code).

1) How the author of the dsmr component knew the structure of the data part of the telegram? I mean that i didn't find any information where there is IV in the telegram. Also in the source code there are variables suggesting that AES128 was used but encryption key has 32 characters measinng that AES256 shall be used.

Which version of ESPHome has the issue?

2024.12.0-dev20241123

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

NA

What platform are you using?

ESP8266

Board

D1 mini

Component causing the issue

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Two samples of telegrams loged: 2F 45 47 4D 35 47 31 35 0D 0A 0D 0A 00 82 02 30 30 00 00 00 00 71 D5 81 85 DD 63 F2 52 7D E9 B5 22 34 4B 39 DC 03 70 35 58 AE 97 9A 34 EA 9C 13 F5 E9 B7 EA 83 4F C5 28 63 CD 0E 77 8C C6 F0 84 0A 25 0C EE 27 3D E7 FF A9 36 9E 11 FE A5 01 7F 84 9B 80 DE 54 0B 68 FE 80 14 E3 0E E3 62 28 A8 BA 21 A4 9F 39 9E 98 C5 28 C4 28 B7 46 75 EF 37 13 63 C8 53 D1 96 2A 8B 69 0D E2 C0 EE F5 77 A7 D8 8E 4C ED 05 93 A5 E6 B1 9D 50 26 58 79 F2 69 1B C3 7F 1E F4 D6 EF C3 32 6D 65 3E 99 4C E4 96 D9 FA 4D 2E 1D 11 8F B8 72 4A F0 D1 4C 59 67 CC A5 F9 5D D1 5F AE 49 19 9A D4 E6 C6 33 C4 13 58 6B D3 64 AA 58 E9 8F D8 0E 61 DF 7A 1B FB 4B 73 FA 51 6D 9F D6 83 9D DD A6 A4 99 F0 F8 3C 49 CC DD F5 D9 02 AF C6 C4 88 B0 19 D8 56 35 80 23 93 61 1F 72 33 F2 2A C7 B1 4C 04 BD 83 AB EA CD 59 7C 66 D1 24 A2 6D E1 4C 1A 05 77 9E CE C0 08 8B BE 9A CF 20 8F 72 4A 7C 51 AA 0B D5 70 B7 AA 18 48 95 45 02 CF 37 B0 85 D4 B4 29 14 54 F3 81 C7 B1 FA F3 72 2F 6B EC 67 E5 2F 99 52 EA 7E 45 83 92 6B E2 25 B3 D8 CB CE 6E DD 7B 7A CD 86 1B 8C E4 A0 07 BE CE C5 73 72 FF 39 FC B1 56 87 DF D0 42 4F B8 C1 57 2A 36 22 35 CF 14 6E 75 09 60 89 3B 9A D2 E2 27 A9 A2 45 8B 18 E8 7A 4E 4E 9F 5F AD 6A 0E A4 3B C6 AA 79 79 89 9E 78 49 6D 82 6E 85 1B AC 08 48 68 0E A6 52 95 E4 16 DF 8D EC 19 C8 AF B7 E6 A2 8E 84 31 AF AA D4 FD 59 B4 63 8C 1C 91 79 93 E1 5A 9F 4B 3D B2 7A 78 00 EE CA 74 11 A5 BF B7 FE 2E 65 2E FF B9 4E 34 83 40 81 30 2D 86 99 5C 82 0D E1 C2 68 10 10 A5 DE F4 41 55 59 B7 A6 DB D7 8C CB 2B 33 0F D8 D3 56 04 9D 82 D6 4B 58 F1 11 E6 E7 94 2D 63 3F 79 73 09 A6 3F AF D6 1D C4 EF 82 D0 F8 84 1F 09 47 C1 9A AF 63 9A 46 24 24 DF 0E 5B 3D A3 9A D2 9A 85 D8 39 6D D1 18 96 38 45 43 87 6B 05 47 92 45 E6 63 AD D9 44 F8 34 A8 D6 07 2B E2 0B 63 93 33 D9 94 9C 4A 21 32 34 38 35 0D 0A

2F 45 47 4D 35 47 31 35 0D 0A 0D 0A 00 82 02 30 30 00 00 00 00 71 D5 81 85 DD 63 F2 52 7D E9 B5 22 34 4B 39 DC 03 71 30 53 A8 91 9A 34 EA 9C 13 F5 E9 B7 EA 83 4F C5 28 63 CD 0E 77 8C C6 F0 84 0B 27 02 EE 27 3D E7 FF A9 36 9E 11 FE A5 01 7F 84 9B 80 DE 54 0B 68 FE 80 14 E3 0F E1 6C 28 A8 BA 21 A4 9F 39 9E 98 C5 28 C4 28 B7 46 75 EF 37 13 63 C8 53 D1 96 2A 8B 69 0D E2 C0 EE F5 77 A7 D8 8E 4C ED 05 93 A5 E6 B1 9D 50 26 58 79 F2 69 1B C3 7F 1E F4 D6 EF C3 32 6D 65 3E 99 4C E4 96 D9 FA 4D 2E 1D 11 8F B8 72 4A F0 D1 4C 59 67 CC A5 F9 5D D1 5F AE 49 19 9A D4 E6 C6 33 C4 13 58 6B D3 64 AA 58 E9 8F D8 0E 61 DF 7A 1B FB 4B 73 FA 51 6D 9F D6 83 9D DD A6 A4 99 F0 F8 3C 49 CC DD F5 D9 02 AF C6 C4 88 B0 19 D8 56 35 80 23 93 61 1F 72 33 F2 2A C7 B1 4C 04 BD 83 AB EA CD 59 7C 66 D1 24 A2 6D E1 4C 1A 05 77 9E CE C0 08 8B BE 9A CF 20 8F 72 4A 7C 51 AA 0B D5 70 B7 AA 18 48 95 45 02 CF 37 B0 85 D4 B4 29 14 54 F3 81 C7 B1 FA F3 72 2F 6B EC 67 E5 2F 99 52 EA 7E 45 83 92 6B E2 25 B3 D8 CB CE 6E DD 7B 7A CF 86 1B 8C E4 A0 07 BE CE C5 73 72 FF 39 FC B1 56 87 DF D0 42 4F B8 C1 57 2A 36 22 35 CF 14 6E 75 09 60 89 3B 9A D2 E2 27 A9 A2 45 8B 18 E8 7A 4E 4E 9F 5F AD 6A 0E A4 3B C6 AA 79 79 89 9E 78 49 6D 82 6E 85 1B AC 08 48 68 0E A6 52 95 E4 16 DF 8D EC 19 C8 AF B7 46 A2 8E 84 31 AF AA D4 FD 59 B4 63 8C 1C 91 79 93 E1 5A 9F 4B 3D B2 7A 78 00 EE CA 74 11 A5 BF B7 FE 2E 65 2E FF B9 4E 34 83 40 81 30 2D 86 99 5C 82 0D E1 C2 68 10 10 A5 DE F4 41 55 59 B7 A6 DB D7 8C CB 2B 33 0F D8 D3 56 04 9D 82 D6 4B 58 F1 11 E6 E7 94 2D 63 3F 79 73 09 A6 3F AF D6 1D C4 EF 82 D0 F8 84 1F 09 47 C1 9A AF 63 9A 46 24 24 DF 0E 5B 3D A3 9A D2 9A 85 D8 39 6D D1 18 96 38 45 43 87 6B 05 47 92 45 E6 63 AD D9 44 F8 34 A8 C6 15 7C C0 52 90 8C 6C 86 50 6D 4D 21 41 31 34 46 0D 0A

Additional information

Encryption key: 8a4e5d79f5c82ce90246b7730b1f46c7

nagisa commented 14 hours ago

Hey,

How the author of the dsmr component knew the structure of the data part of the telegram? I mean that i didn't find any information where there is IV in the telegram.

Generally this is entirely dependent on the operator of the meters. They write down the (usually vague) product requirements and the meter manufacturers implement them to the letter. It may very well be that the meter operator specified that encryption is to be used for P1, without any specifics and engineers at Elgama Elektronika went the easiest route and reused the encryption routines they have for DLMS encryption.

Until very recently only a couple regions (out of the few which have deployed P1) used encryption for their P1 telegrams: Belgium and Luxemburg. So the decryption implementation has been tailored and implemented based on what's deployed there. Another complication is that meters deployed in those regions are made by Sagemcom. Given that encryption is not standardised by the P1 standard -- or any standard -- it wouldn't be surprising if Elgama's implementation differed, or if the implementation for the same manufacturer differed across regions.

Since the payload is not huge, and you have the decryption key, in principle you could figure out the IV construction and whatnot by brute force, but if your operator has supplied you with the decryption key in the first place, they most likely have some documentation available to provide you on how to decrypt as well.

Also in the source code there are variables suggesting that AES128 was used but encryption key has 32 characters measinng that AES256 shall be used.

More specifically AES-GCM-128 and, interestingly, ESPHome's documentation suggests that the key for this algorithm should be 32-bytes long. I strongly doubt Elgama used anything other than AES-GCM-128, as this is one of the few encryption schemes supported for DLMS encryption. So in absence of documentation, I would start with a brute force program to try all possible combinations of the IV key.

mich909 commented 2 hours ago

thanks for fast reply I have doubt if AES128 encryption was used because the data that I received seems to be very similar, there are some differences in values in few bytes. From what I understand regarding cryptography such situation shouldn't occur as if there is small change in decrypted data, the after encryption there shall be all bytes changed similarly to CRC. Could you confirm my thoughts? Also I tried to verify CRC with CRC16-IBM but without succes which looks like all telegram that i received could be corrupted. I have attached two telegrams with encryption key if someone would like to check them. One more: I received two keys: AK and EK - looks like Authentication and Encryption keys, but for the purpose of reading telegram I should use only EK. Am I correct?

esphome / issues

DSMR - Polish meters #6500