Add support for Austrian meter

[gskjold]

Slightly different format

218_9_SmartMeter_Kundenschnittstelle_lektoriert_1410_Web.pdf Beschreibung_Kundenschnittstelle_Smart_Meter_TINETZ.pdf

[manfi001]

The attachment contains 10 consecutive packets of 282 bytes ea received from my meter. maybe this helps to clarify the format. Sagemcom.txt

[gskjold]

Looking at it, the 282 bytes frame actually consist of two frames, one 263 (259+4) and a 19 byte frame which I am not sure what is for. The specified length 0x0d0d makes no sense. Makes me wonder if 0x0101 is actually a length, although it is consistent with a normal HDLC frame.

// 19b 68 0D 0D 68 53 FF 11 01 67 CD 6B CB 69 13 53 FF 98 34 16

// 263b 68 01 01 68 53 FF 00 01 67 DB 08 53 41 47 59 05 E6 D9 FD 81 F8 20 00 69 D1 50 55 28 2C E9 97 46 82 61 19 3E 23 78 8A E6 E2 42 D1 D6 44 BA 2C 3C 55 0E 59 47 02 DC 8D D4 10 91 67 6B 76 9B F0 2F 42 BF D9 D2 FE A2 B3 AA 11 B1 BF 7B 8B B3 36 FE 7E B0 22 D7 60 10 48 1B 77 AA C2 DC 99 8D C2 C4 5D 78 83 53 92 E8 66 44 CC 32 43 A9 E8 22 B2 0E DF D8 39 B3 21 5B E6 A8 F1 83 5E 85 5A A3 5D 2B 92 ED 59 D7 24 2C CC 26 AB A6 0A FE 78 B0 E9 D3 7C 6D B8 32 0F 36 C0 A0 9B A2 56 73 08 56 EE 9B AD 7C CC F3 6B EC 13 63 55 2A 28 0E 7A 9B D9 2A 62 08 D5 9C AD E8 43 6D 7A CA 8B DD BF DB 3F E1 88 3F 9D B9 7C 19 D3 68 8C 57 AB 82 46 4B 75 B8 F3 9E 2C 22 06 2A 93 78 56 56 76 51 65 11 C7 12 78 AB F2 97 97 51 2A 16 70 56 30 C9 12 00 08 BC 80 55 6E 44 51 A1 93 CD CF BA 9A DA CA 48 19 74 E4 70 1E AD 63 99 16

[gskjold]

A detailed description of all the fields in the frame:

68 // MBUS start flag 0x68 01 01 // Format (0x00) and total length (257) (Green book 8.4.1.3)

68 // Start 53 // Control field FF // Address (Broadcast address)

00 // Control information field 01 // Source SAP 67 // Destination SAP

DB // Encrypted 08 53 41 47 59 05 E6 D9 FD // System title, first byte specifies length 81 // Prefix for 1-byte length F8 // Length (248) 20 // Security tag 0010 0000, 0=Compression off, 0=Unicast, 1=Encryption, 0=No auth, 0000= Security Suite ID 00 72 00 76 // Frame counter // Encrypted frame follows 00 // 1 byte CRC 16 // End flag

This last part, from 0xDB and until the end is consistent with format from encrypted Kamstrup in Denmark, only they have authentication, adding 12 extra bytes to the footer, before CRC and end flag.

[manfi001]

According to other Mbus protocol documentations, byte 3 is a repetition of byte 2, which would mean hex 01 01 is not dec 257. consequently hex 0D 0D = dec 13 would make sense at least for the second frame.

[gskjold]

Which doc are you referring to? Not able to find this in green book

[manfi001]

this here: Anleitung_M-Bus_Protokoll_V1.0.pdf sorry to bother you with all this German docs

[gskjold]

No worries, I can deal with German :) Thanks!

[manfi001]

Another one: M-Bus_DOC48.PDF

[manfi001]

Found another project on Github dealing with the same meter: https://github.com/culvermelanie/SmartMeter

[gskjold]

Thanks, I've actually managed to make it work with the example payload from the document for your meter. A lot of the values are 0 though, so not very useful. But it works, so you can test it when you get the key

[manfi001]

Great! The doc is based on a different meter (Kaifa MA309), but I hope, it will work with my Sagemcom too. will keep you informed.

[manfi001]

Q: Is the debug data in the frame dump the raw data from the meter or is it already decrypted?

[gskjold]

It will be the decrypted frame. I just release v2.0.0 which I think should work with your meter. Let me know how it turns out.

[manfi001]

thanks for your efforts 👍 Still waiting for the key :-( btw: is the missing verbose debug level in the ui intentional? and what is the difference between debug and verbose level?

[gskjold]

I have not taken advantage of verbose logging in the code, so I have not included it in the config either. It is up to the developer to use the difference levels of logging, and I guess verbose logging would just be an extended debugging log.

[manfi001]

Just received my key, entered it, rebooted and >> Hurraaa, it's working. Thanks for your work! Wish you merry X'mas and Happy & healthy new Year!

[gskjold]

That is awesome! Thank you for the good words. Merry Christmas!

[gskjold]

By the way, could you provide me with meter model, baud and parity from the meter configuration page?

[ArnieO]

Just received my key, entered it, rebooted and >> Hurraaa, it's working.

Great!!

Would you be interested in sharing your hardware setup? I am curious to find out whether the Pow-U unit I currently sell from my webshop could be modified to work on your meter.

I see from the documentation in this thread that your meter is:

• Sagemcom Siconia smart meter installed/owned by Netz NÖ
• M-bus
• RJ12 6P6C connector, data on pins 3(+) and 4(-), other pins unconnected.
• Maximum power draw from M-bus: 6mA at 32V

The Pow-U pulls its energy from the M-bus line; works on a Kamstrup meter where maximum power draw is 144 mW, so this should be OK. However, meters in Norway have their base voltage at 24V ("zero"), dropping to 12V ("one") during data transmission - so a minor modification would be needed to read data (I do not use the TSS721 chip).

From what I see in the above documentation, your meter seems to have a base voltage at 36V ("zero"), dropping to 24V ("one") during data transmission - can you confirm this?

Please contact me on post@amsleser.no if you are interested in discussing the possibility of me sending you a modified Pow-U unit for testing on your meter.

[manfi001]

Hi,

the meter config page looks like this:

The meter is a Sagemcom T210-D. The grid operator is Netz NÖ

I think, my enthusiasm was a bit premature. There are still some issues:

1. The reactive power data is always zero:

2. The device is restarting every couple of hours

3. In the log I get these messages

(I) (readHanPort) Valid HDLC, start at 44

(W) (readHanPort) Invalid HDLC, returned with -9

(I) (readHanPort) Valid HDLC, start at 44

(W) (readHanPort) Invalid HDLC, returned with -9

(I) (readHanPort) Valid HDLC, start at 44

(W) (readHanPort) Invalid HDLC, returned with -9

Are the 2 frames re-assembled before decoding, as described in the document ….TINETZ.pdf pg 6?

Regards

Manfred

From: Gunnar Skjold @.*** Sent: Dienstag, 21. Dezember 2021 11:43 To: gskjold/AmsToMqttBridge Cc: manfi001; Comment Subject: Re: [gskjold/AmsToMqttBridge] Add support for Austrian meter (Issue #155)

By the way, could you provide me with meter model, baud and parity from the meter configuration page?

— Reply to this email directly, view it on GitHub https://github.com/gskjold/AmsToMqttBridge/issues/155#issuecomment-998671244 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ATQ5AAFLFPUKGXFNTPVSKZDUSBK3ZANCNFSM5JNDB7YA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you commented. https://github.com/notifications/beacon/ATQ5AAHSP2HVUP6LTGC5SBLUSBK3ZA5CNFSM5JNDB7YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHODIHDA.gif Message ID: @.***>

[gskjold]

Reopening to have a look at that problem.

[ArnieO]

The setup is missing from his message, but his settings should be configured like this, according to https://github.com/gskjold/AmsToMqttBridge/files/7656628/Beschreibung_Kundenschnittstelle_Smart_Meter_TINETZ.pdf

[gskjold]

Are the 2 frames re-assembled before decoding, as described in the document ….TINETZ.pdf pg 6?

Nice, this is the problem, will fix when I have time

[gskjold]

Try this firmware: esp8266.zip

I have tested it with data and key from first PDF in this issue. It looks like it is working. Tested with some of your data and it does at least assemble the data correctly, so lets hope it works for you.

It it doesn't work, enable debugger and send me the log.

[manfi001]

Doesn't look good, HAN indicator goes from red to orange to green back to orange and red. Remember: data and key from pdf is for a Kaifa meter, not Sagemcom ams2mqttAustria.txt

[gskjold]

Doesn't look good, HAN indicator goes from red to orange to green back to orange and red.

I see there is a lot of buffer overflow here.. Could you try disabling temperature sensors? Save blank temp pin in GPIO.

Remember: data and key from pdf is for a Kaifa meter, not Sagemcom

I know it is not the same, but I need to implement this as generic as possible. Also, running the dump file you previously sent to me through my setup works just fine, only I do not have your key. Assembly is correct at hex level though.

[manfi001]

This is my GPIO setup

[gskjold]

Aha, sorry I thought I saw a temp update message in log > 0ms, but I was mistaken.

I moved around on some of the code, here is a new one: esp8266.zip

[manfi001]

Same situation :-( ams2mqttAustria2.txt

[gskjold]

I would suggest trying UART2 and hook your meter up to GPIO13 so that a actual hardware serial is used. I feel like this could be related to some instability using software serial. Judging from the log, assembly is working fine now, but for some reason you are getting overflow in between.

[manfi001]

The problem is, that I am using a Wemos D1 mini and it has the hw serial connected to the ftdi chip

[manfi001]

Would my key help you for debugging?

[gskjold]

If you choose UART2 as HAN pin, the hardware serial pins will be swapped to pin 13 (RX) an 15 (TX) and will prevent USB chip from interfering

If it still does not work and you are OK with sharing your key, please send it to gunnar.skjold@gmail.com

[gskjold]

Try this firmware: esp8266.zip

[manfi001]

Thanks! Looks better now, but: reactive part still all zeroes, btw: how can I delete the false historic data?

Meter manufacturer and model not appearing HDLC errors solved: ams_reader_211227_log.txt (readHanPort) System title: (D) 53 41 47 59 05 E6 D9 FD converted to ascii/dec =SAGY 99015165 could be the meter manufacturer/type, the number in bold are the last 6 digits of my counter no.

[gskjold]

Good to hear! I see in your decrypted frame that reactive values are not present (OBIS 3.7.0, 4.7.0, 3.8.0 and 4.8.0) There is no function to manually clear the graph other than erasing flash and re-flashing. The meter model detection currently does not use system title, but I will definitely implement that.

[manfi001]

Still restarting sporadically. Could this be caused by weak wifi signal? Seems that restart also causes weird values (+14.4, -22.2) and that hourly and monthly data are not updated correctly. ( compare to yesterday's screenshot)

[gskjold]

Looks like it is never saving graph data? Would have to look at log to figure that one out. Not sure about that reboot..

[gskjold]

Added checksum verification and cleaned up some code. It may or may not help with reboot. esp8266.zip

[gskjold]

Updated a few core functions in the code, new firmware file:

esp8266.zip

[manfi001]

After erasing and re-flashing version 211228.1 the hourly values seem to be updated correctly but daily values are still missing: reboots less frequently, longest interval was 2 days.

Updated to ver. 220105.02 just now and noticed first reboot after only 15 min, 2nd after 18 min. previous version transmitted mqtt data every 5 sec. (same interval as meter data) but the new firmware has breaks (see MQTTfx log below) 2022-01-05 21:20:58,375 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:03,377 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:17,993 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:18,407 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:23,377 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:28,376 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:41,797 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:43,382 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:21:48,399 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:01,899 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:03,382 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:08,384 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:23,513 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:28,378 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA 2022-01-05 21:22:33,384 INFO --- MqttFX ClientModel : messageArrived() with topic: tele/Sagemcom/DATA

and sometimes only transmits ESP data: { "id" : "84:CC:A8:A1:FB:B0", "name" : "", "up" : 1084, "vcc" : 3.3, "rssi" : -78, "temp" : -127.0 }

Telnet debugging stops after 2 blocks: *** Remote debug - over telnet - for ESP8266 (NodeMCU) - version 3.0.5

• Host name: ams-fbb0 IP:10.1.1.192 Mac address:84:CC:A8:A1:FB:B0

• Free Heap RAM: 16864

• ESP SDK version: 2.2.2-dev(38a443e)

  ---

• Commands: ? or help -> display these help of commands q -> quit (close this connection) m -> display memory available v -> set debug level to verbose d -> set debug level to debug i -> set debug level to info w -> set debug level to warning e -> set debug level to errors s -> set debug silence on/off l -> show debug level t -> show time (millis) profiler: p -> show time between actual and last message (in millis) p min -> show only if time is this minimal P time -> set debug level to profiler c -> show colors filter: filter -> show only debugs with this nofilter -> disable the filter cpu80 -> ESP8266 CPU a 80MHz cpu160 -> ESP8266 CPU a 160MHz

• Please type the command and press enter to execute.(? or h for this help)

  ---

  (D) (AmsWebServer)Serving /data.json over http... (I) (readHanPort) Intermediate segment (D) (readHanPort) Frame dump (263b): (D) 68 01 01 68 53 FF 00 01 67 DB 08 53 41 47 59 05 (D) E6 D9 FD 81 F8 20 00 04 15 3C 8F 1C A8 50 0F 75 (D) 5D A9 FF 25 ED 8B 71 40 08 54 96 C8 B6 1A D4 88 (D) FD 33 0D 80 BE 26 08 C9 F5 6F 52 85 90 90 D3 66 (D) 3A 73 20 37 C7 F0 42 3D B4 E1 BB 40 90 55 5D E0 (D) 62 32 4F 89 61 2C 85 8D 19 80 15 1E 61 8C F2 FD (D) 90 C4 AB D4 BE 8E F3 CE 21 EE 7F 05 15 B4 7E A7 (D) 65 82 06 F7 EB 6F 94 26 FD 9C 3D BA 67 1A 2A B8 (D) 8E F5 0F B5 1E B6 23 61 08 9C E3 B3 EC F3 7E 85 (D) 33 C5 F1 29 43 EF 7B 0C A8 BC 46 D1 98 29 74 96 (D) 2E CA A8 9B D8 C0 D1 A8 F9 0A 60 0C F2 78 38 2D (D) 65 C8 12 6A 42 89 53 88 E3 EC 57 17 E2 8C 18 02 (D) D7 C8 FA 1B AF F4 2B D4 0A 7F AC 2E 55 52 E3 36 (D) 5D DE D5 2A 58 CD F6 FC E4 28 00 69 6E 02 24 90 (D) 13 A2 13 84 2B 2D 63 56 09 72 BC 07 20 1C 5A 76 (D) B3 07 65 4A 37 AA 77 DB 7A A4 0D BB B2 F9 6F 53 (D) 0D 52 B6 FB DE 22 16 (I) (readHanPort) Final segment (D) (readHanPort) Frame dump (19b): (D) 68 0D 0D 68 53 FF 11 01 67 38 48 3B 91 A5 B9 0A (D) 32 B1 16 (D) (readHanPort) Frame dump (271b): (D) 68 00 00 68 53 FF 10 01 67 DB 08 53 41 47 59 05 (D) E6 D9 FD 81 F8 20 00 04 15 3C 0F 80 40 AB CD 0C (D) 07 E6 01 05 03 15 19 2D 00 FF C4 02 02 23 09 0C (D) 07 E6 01 05 03 15 19 2D 00 FF C4 02 09 06 01 00 (D) 01 08 00 FF 06 00 44 BD 7F 02 02 0F 00 16 1E 09 (D) 06 01 00 02 08 00 FF 06 00 00 01 03 02 02 0F 00 (D) 16 1E 09 06 01 00 01 07 00 FF 06 00 00 02 10 02 (D) 02 0F 00 16 1B 09 06 01 00 02 07 00 FF 06 00 00 (D) 00 00 02 02 0F 00 16 1B 09 06 01 00 20 07 00 FF (D) 12 09 1D 02 02 0F FF 16 23 09 06 01 00 34 07 00 (D) FF 12 09 24 02 02 0F FF 16 23 09 06 01 00 48 07 (D) 00 FF 12 09 1E 02 02 0F FF 16 23 09 06 01 00 1F (D) 07 00 FF 12 00 D2 02 02 0F FE 16 21 09 06 01 00 (D) 33 07 00 FF 12 00 11 02 02 0F FE 16 21 09 06 01 (D) 00 47 07 00 FF 12 00 17 02 02 0F FE 16 21 09 06 (D) 01 00 0D 07 00 FF 10 03 E6 02 02 0F FD 16 FF 09 (D) 0C 31 37 38 32 31 30 30 31 35 31 36 35 18 16 (D) (readHanPort) System title: (D) 53 41 47 59 05 E6 D9 FD (D) (readHanPort) Initialization vector: (D) 53 41 47 59 05 E6 D9 FD 00 04 15 3C (D) (readHanPort) Additional authenticated data: (D) 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (D) 00 (D) (readHanPort) Authentication tag: (D) 00 00 00 00 00 00 00 00 00 00 00 00 (I) (readHanPort) Valid data, start at byte 44 (D) (AmsDataStorage) Time is: 1641414343 (D) (AmsDataStorage) It is only 1540 seconds since last update, ignoring (D) (loop) Used 0 ms to update temperature

Strange observation: 1st frame of 2nd block starts with 68 00 00 68, normally it's 68 01 01 68

[gskjold]

Thanks for the feedback. Attached is a new firmware that may or may not fix the graph ;) Still not sure about those reboots... Regarding 68 00 00 68, this is a "fake" (271b) frame assembled by the two partial frames (263b + 19b). I just chose to use 00 in length parameter to make the code ignore the length field.

esp8266.zip

[manfi001]

New firmware solved the irregular mqtt message intervals and the debug log freeze. still no daily data in graph, to exclude weak wifi as reason for reboots, I connected an external antenna. wifi is green now, but reboots still occur appr. every hour. observation: free mem sometimes drops to 12k. Is it normal, that the telnet log closes after 10 min.?

[gskjold]

Telnet has 10min timeout, yes. Upstream library, so I have no control over it.

Try disabling debugging, I've had many cases lately where having telnet/serial debugger enabled causes reboot. Set level to ERROR and disable both telnet and serial.

Regarding month plot, it need to pass two midnights before it can start calculating.

[manfi001]

disabled debugging, but still 5-6 restarts/day. hourly values in graph are o.k. but daily values still erratic. will try a second unit connected in parallel, to see if it is hardware related. In principle it does what I need: mbus to mqtt. The graphical interface is a nice feature, but not essential for me. Thanks for your efforts.

[gskjold]

Still working on why the month graph is acting up for some user, but I have made stability improvements on the attached firmware

esp32.zip esp8266.zip

[manfi001]

updated with the new firmware. 1st impression: apart from restarts every couple of hours, seems to work fine. will continue observing. another challenge: tried the reader with a Sagemcom T210-D from another Austrian grid operator. The user port follows the DSMR 5.0.2 standard with additional encryption ( like meters in Luxembourg) . Data is pushed every 10 sec. with 115200 baud, 8N1. I tried the 220106.1 and the standard 2.0.6 firmware, but get "unknown data format 8C"

[gskjold]

Although DSMR should be supported, I have not implemented decryption for it. Do you know of any documentation on that encryption?

[manfi001]

Found something here: http://www.weigu.lu/microcontroller/smartyReader/index.html

[manfi001]

With the new firmware it reached the longest interval between restarts (>20h) but since yesterday morning HAN is red and log shows "Frame timestamp is not correctly formatted"