Add Kamstrup protocol to SML

[TheChatty]

Have you looked for this feature in other issues and in the docs?
Yes.

Is your feature request related to a problem? Please describe.
Native support for Kamstrup protocol as it requires CRC calc and a few char escapings.

Describe the solution you'd like
Like o is for OBIS I suggest k for Kamstrup. In the script the user should supply requested variable (int32 xxyy). This will be expanded to the request msg 3F1001xxyyCRCR. 5 special chars need then to be escaped and the message will finally be sent. Here are hexdumps for all variables meaningful for heating meters like mine:

Wr:b'803f1001003cb25f0d'
Rd:b'403f10003c0304430000d6de03220d'
Heat Energy (E1),55.006,MWh
Wr:b'803f100100501f750d'
Rd:b'403f10005016024100002f5c0d'
Power,0.0,kW
Wr:b'803f100100567fb30d'
Rd:b'403f10005625024214f569dc0d'
Temp1,5.365e-08,C
Wr:b'803f100100576f920d'
Rd:b'403f10005725024211e3a17e0d'
Temp2,4.579e-08,C
Wr:b'803f100100598e5c0d'
Rd:b'403f100059260242031255200d'
Tempdiff,7.86e-09,K
Wr:b'803f1001004aac0e0d'
Rd:b'403f10004a2902000000f0500d'
Flow,0.0,l/h
Wr:b'803f100100444dc00d'
Rd:b'403f1000442804420002dca47c0a0d'
Volume,1875.56,m3
Wr:b'803f1001008d05a50d'
Rd:b'403f10008d290200000086210d'
MinFlow_M,0.0,l/h
Wr:b'803f1001008b65630d'
Rd:b'403f10008b29020000f5b47a0d'
MaxFlow_M,245.0,l/h
Wr:b'803f1001008c15840d'
Rd:b'403f10008c30040000035ee58a930d'
MinFlowDate_M,220901.0,yy:mm:dd
Wr:b'803f1001008a75420d'
Rd:b'403f10008a30040000035ee50a580d'
MaxFlowDate_M,220901.0,yy:mm:dd
Wr:b'803f10010091d6180d'
Rd:b'403f1000911602410000d4cc0d'
MinPower_M,0.0,kW
Wr:b'803f1001008f25e70d'
Rd:b'403f10008f16024100235aea0d'
MaxPower_M,3.5,kW
Wr:b'803f10010095969c0d'
Rd:b'403f1000952501004509e20d'
AvgTemp1_M,69.0,C
Wr:b'803f10010096a6ff0d'
Rd:b'403f1000962501003d18af0d'
AvgTemp2_M,61.0,C
Wr:b'803f10010090c6390d'
Rd:b'403f10009030040000035ee5a6900d'
MinPowerDate_M,220901.0,yy:mm:dd
Wr:b'803f1001008e35c60d'
Rd:b'403f10008e30040000035ee635560d'
MaxPowerDate_M,220902.0,yy:mm:dd
Wr:b'803f1001007edad90d'
Rd:b'403f10007e2902000000d97d0d'
MinFlow_Y,0.0,l/h
Wr:b'803f1001007cfa9b0d'
Rd:b'403f10007c290200015fca160d'
MaxFlow_Y,351.0,l/h
Wr:b'803f1001007deaba0d'
Rd:b'403f10007d30040000035c2dbeeb0d'
MinFlowDate_Y,220205.0,yy:mm:dd
Wr:b'803f1001007b8a7c0d'
Rd:b'403f10007b30040000035bd099050d'
MaxFlowDate_Y,220112.0,yy:mm:dd
Wr:b'803f10010082f44a0d'
Rd:b'403f100082160241000000a80d'
MinPower_Y,0.0,kW
Wr:b'803f1001001b7fd4080d'
Rd:b'403f10001b7f160241009ee99f0d'
MaxPower_Y,15.8,kW
Wr:b'803f10010092e67b0d'
Rd:b'403f1000922501004b8ff80d'
AvgTemp1_Y,75.0,C
Wr:b'803f10010093f65a0d'
Rd:b'403f10009325010035baf00d'
AvgTemp2_Y,53.0,C
Wr:b'803f10010081c4290d'
Rd:b'403f10008130040000035c2cf2f00d'
MinPowerDate_Y,220204.0,yy:mm:dd
Wr:b'803f1001007fcaf80d'
Rd:b'403f10007f30040000035bd7e68f0d'
MaxPowerDate_Y,220119.0,yy:mm:dd
Wr:b'803f1001006139070d'
Rd:b'403f1000612b0400000231e7dcfc0d'
Temp1xm3,143847.0,m3xC
Wr:b'803f1001006ec8e80d'
Rd:b'403f10006e2b04000001777bb9310d'
Temp2xm3,96123.0,m3xC
Wr:b'803f100100712b360d'
Rd:b'403f1000713304000000000c511f0d'
Infoevent,12.0,
Wr:b'803f100103ec2c710d'
Rd:b'403f1003ec2e030000787e6ff90d'
HourCounter,30846.0,h

Describe alternatives you've considered
Non-Tasmota python script. I'd rather avoid this.

Additional context
There is plenty of python source code: 1 2 3

One request with the third code example:

readvar( b'003c' )
--> send( 0x80, (0x3f, 0x10, 0x01, 0x00, 0x3c) )
----> wr( b'803f1001003cb25f0d' )
<---- recv: b'403f10003c0304430000d6de03220d'
<-- decode mantissa & exponent and return

(Please, remember to close the issue when the problem has been addressed)

[gemu2015]

normally i only implement drivers for devices i personally use. however at least i would need a hexdump of meter values to evaluate how much effort it would be. since berry is very similar to python you may also consider to try that path.

[TheChatty]

@gemu2015 I added the infos to the issue description.

[gemu2015]

ok i added kamstrup, give it a try i can not test since i don't have such a meter, at least the transmission is correct the decoder code is kstr it works similar to modbus and you must also use indexes i0 ...

>M 1 +1,3,k,0,9600,KS,1,10,3F1001003C 1,3F10003Ckstr@i0:1,Volts,V,VP,2 #

[TheChatty]

Incredible but it will take two weeks till I can test. I‘ll report.

[TheChatty]

I eventually found time to test your code which was obviously already merged upstream.

I configured the meter as:

+1,3,kN2,0,1200,KSMC403,1,10,3F1001003C
1,3F10003Ckstr@i0:1,Heat Energy (E1),MWh,HeatEnergyE1,3

Because it runs ar 1200 8N2 and 3C returns total heat energy used in MWh at a precision of 3.

The request is created in the correct manner and answered from the meter like this (seen by sniffing the serial line):

> 80 3F 10 01 00 3C B2 5F 0D
< 40 3F 10 00 3C 03 04 43 00 00 D9 2B AC A6 0D

But console shows no receiption / webUI shows "KSMC403 Heat Energy (E1) | null MWh".

When defining "MODBUS_DEBUG" I see transmits only as well.

Even when I connected my FTDI to act as a fake IR transmitter I received all requests from ESP but no response was shown. When I configured "Serial TX/RX" instead of script I could see all request/response pairs.

[gemu2015]

try new version in my repo

[TheChatty]

1. That worked to some extend. I can now define all measures and they are requested one after the other but only the first requested measure is decoded correctly (e.g. actual value shown in UI).
2. I still only see transmits in console with define MODBUS_DEBUG - no responses even in the case of the first telegram.
3. Some values represent a date. The number 221001 (YYMMDD) should be shown as 01.10.22... is this achievable?

Current meter definition is:

>D
>B
=>sensor53 r
>M 1
+1,3,kN2,0,1200,KSMC403,1,10,3F1001003C,3F10010056,3F10010057,3F10010059,3F1001004A,3F1001008A
1,3F10003Ckstr@i0:1000,Heat Energy (E1),MWh,HeatEnergyE1,3
1,3F100056kstr@i0:100,Vorlauftemp.,°C,PreTemp,2
1,3F100057kstr@i0:100,Rücklauftemp.,°C,AftTemp,2
1,3F100059kstr@i0:100,Temp.diff.,°C,DifTemp,2
1,3F10004Akstr@i0:1,Fließgeschw.,l/h,Flow,2
1,3F10008Akstr@i0:1,Datum max. Fluss,d,maxFlowDate
#

All values remain zero except the one from the first telegram.

[gemu2015]

You must increment the index i0, i1 etc like with modbus

give an example response for time

[TheChatty]

403f10008a30040000035ee50a580d decodes to 220901 meaning 01.09.22.

[TheChatty]

I found that I can request multiple values at once. But skip N bytesdoes not work whereas enough xx work. Following entries should be similar but are not.

+1,3,kN2,0,1200,KSMC403,1,10,3F1002003C0056
1,3F10003Ckstr@i0:1000,Heat Energy (E1),MWh,HeatEnergyE1,3
1,3F10x90056kstr@i0:100,Vorlauftemp.,°C,PreTemp,2
1,3F10xxxxxxxxxxxxxxxxxx0056kstr@i0:100,Vorlauftemp.,°C,PreTemp,2

Request of a single value is responded within 316ms, 2 values within 399ms, 3 values within 501 ms, 4 values within 619ms and 6 values in 765ms. And starting from 4 values all values no longer get decoded. Probably a timeout (baudrate 1200 is so slow)?

[gemu2015]

1. the decoder skip with xn requires a not number delimiter, so x90056kstr would assume you would skip 90056 chars while x11kstr would work
2. the serial buffer is limited to SML_BSIZ with default = 48 bytes, you may increase this value in user config override to fit more data

[TheChatty]

I just noticed those "escape bytes" make the fixed number skip algo somewhat useless. You'd apply the unescapeing before counting.

SerialSend5 80 3f 10 03 00 3c 00 56 00 57 c8 d9 0d 
SerialReceived 403F10003C0304430000D94800562502421C700057250242171BE4521BF90D

SerialSend5 80 3f 10 03 00 3c 00 56 00 57 c8 d9 0d 
SerialReceived 403F10003C0304430000D94800562502421C72005725024217205A980D

[gemu2015]

no, what you see is the raw received value, the decoder gets the corrected data without escapes

[TheChatty]

So it should work already? I will try.

[TheChatty]

I raised SML_BSIZ to 90 and are now able to request at least 6 values. A few examples would be:

> 80 3f 10 1b f9 00 3c 00 56 00 57 00 59 00 4a 00 44 03 1b bf 0d
< 403F10003C0304430000D949005625024217C400572502420AC000592602421BF204004A290200000700442804420002ECA858770D
< 403F10003C0304430000D949005625024217C100572502420A4B00592602421BF276004A290200000900442804420002ECA85F690D
< 403F10003C0304430000D949005625024217CA00572502420A4100592602421BF289004A290200000900442804420002ECA875710D
        <--------09------><-----07-----><-----07-----><-----07--^^---><-----07-----><--------09------><CR>

But the following meter definition eventually decodes all six measures after adding one after the other to the script.

>D
>B
=>sensor53 r
>M 1
+1,3,kN2,0,1200,KSMC403,1,10,3F1006003C005600570059004A0044
1,3F10003Ckstr@i0:1000,Wärmemenge,MWh,HeatEnergyE1,3
1,3F10x08xx0056kstr@i0:100,Vorlauftemp.,°C,PreTemp,2
1,3F10x15xx0057kstr@i0:100,Rücklauftemp.,°C,AftTemp,2
1,3F10x22xx0059kstr@i0:100,Temp.diff.,°C,DifTemp,2
1,3F10x29xx004Akstr@i0:1,Fließgeschw.,l/h,Flow,0
1,3F10x36xx0044kstr@i0:100,Volumenstrom,m³,Volume,2
#

[TheChatty]

Now all that is missing are the date values. Shall I update the docs already (without date values atm)?

[gemu2015]

i implemented the # indicator for kamstrup as a date string. (see repo) like this

1,3F10003Ckstr@#,Datum,,date,0

however since the driver supports only one string per meter you may only define one such a line per meter

yes update the docs together with an example, i will male a pr in the next few days

[TheChatty]

How to specify index in case of date? "@i0:#" doesn't work.

[gemu2015]

no index with this decoder line

[TheChatty]

It will always be applied to first telegram or on every telegram?

[gemu2015]

yes to any, but the decoder will select the right one, because of the register address given.

in theory we would not need an index in kamstrup decoder since the reply repeats the register address which is not the case on MODBUS.

i guess the decoder will work also whitout any index because the register selects the decoder line.

[TheChatty]

But "@100" is not working like "@i0:100"? Former syntax results in "0.99°" and latter syntax in e.g. "45.12°".

[gemu2015]

ok, then leave the indexes and only for the string use without index.

[TheChatty]

There is still some instability... now the formerly working script decodes nothing (all shown as 0)... sensor53 d1 shows data is sent and received.

[gemu2015]

post the script

[TheChatty]

>D
>B
=>sensor53 r
>M 1
+1,3,kN2,0,1200,KSMC403,1,15,3F1007003C005600570059004A0044007B
1,3F10003Ckstr@i0:1000,Wärmemenge,MWh,HeatEnergyE1,3
1,3F10x08xx0056kstr@i0:100,Vorlauftemp.,°C,PreTemp,2
1,3F10x15xx0057kstr@i0:100,Rücklauftemp.,°C,AftTemp,2
1,3F10x22xx0059kstr@i0:100,Temp.diff.,°C,DifTemp,2
1,3F10x29xx004Akstr@i0:1,Fließgeschw.,l/h,Flow,0
1,3F10x36xx0044kstr@i0:100,Volumenstrom,m³,Volume,2
1,3F10x45xx007Bkstr@#,max. Fluss am,,MaxFlowDate,0
#

[TheChatty]

Working with

• 3F1004003C005600570059
• 3F1004003C00560057004A

This telegram works every other time...

• 3F1004004C005600570059004A --> e.g. 403f10003c0304430000d96000562502421a4500572502420a8300592602420fc2004a290200001bf94ef30d

Asking for register 0x0044 seems to be problematic. 803f1003004a003c007ba19e0d --> 403f10 004a2902000005 003c0304430000d960 007b30040000035bd 008a80d 803f1003004a0044007bd0370d --> 403f10 004a2902000005 00442804420002ed1bf2 007b30040000035bd 07ac40d 803f1003004a0044007bd0370d --> 403f10 004a29020000e7 00442804420002ed10 007b30040000035bd0 f9710d

The response in the middle hinders decoding of the entire response. So I guess it has something to do with the escape character. @gemu2015 Can you check on your side?

[gemu2015]

i did put a hexdump in normal mode, (not dump mode) to show the decode string with removed escapes. see repo

[TheChatty]

I split the telegram into two pieces. So sometimes the first, sometimes the second is working.

That point of code only gets called when decoding works. Thus the error must be happening before.

Maybe you're calculating CRC at the wrong time (before/after escape removal)?

[gemu2015]

ok seems to need escaping BEFORE checksum is checked

try repo

[TheChatty]

All CRCs are verified ok. All values are interpreted ok. Thanks a million!

I just created the PR for docs.

[TheChatty]

All done.