[REQUEST] Water flow metering reading of IEC 62056-21 protocol smart water meters (Kamstrup Meter Protocol / KMP) via photodiode/infrared optical eye reader header?

[Hedda]

Describe the problem you have/What new integration you would like

Like to get water flow metering from smart water meters with IR eye via hardware similar to Home Assistant Glow or SlimmeLezer.

Optical photodiode/infrared read and write header hardware for IEC 62056-21 protocol (formerly IEC 61107 / IEC 1107) like these:

https://www.mysensors.org/build/ir

http://jheyman.github.io/blog/pages/WirelessWaterMeter/

A hackerspace group in Denmark is selling these IR/photodiode optical reading heads for about $14 US (Google translate as test written in Danish) and they wrote a note that using magnets is required as the IR port transmitter / reader eye does not activate unless it sense a magnet:

https://wiki.hal9k.dk/projects/kamstrup

https://github.com/Hal9k-dk/kamstrup

I believe that a such IR reader adapter could be used with newer Kamstrup water meters with IR optical heads common in Europe.

Specifically, I myself live in Sweden where I believe that the Kamstrup MULTICA 21 / Kamstrup flowIQ 210x ultrasonic water meter is the most common model installed by the municipality's utility companies (e.i. local government agencies) in normal private residential houses here in Scandinavia (Sweden, Norway, Denmark, Finland, and Island) to track water usage for their service fees.

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/multical-21

They currently also have Kamstrup flowIQ 2200 "advanced model with acoustic leak detection":

and Kamstrup flowIQ 3100 "Commercial and industrial" model

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/flowiq-2200

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/flowiq-3100

Please describe your use case for this integration and alternatives you've tried:

I believe many new water meters offer such two-way bidirectional infrared port (IR receiver and sender header) for communication?

https://www.mysensors.org/build/ir

http://jheyman.github.io/blog/pages/WirelessWaterMeter/

Kamstrup seems to only provide some technical documentation about all their products here but nothing about their protocols:

https://products.kamstrup.com/index.php

Kamstrup apparently calls their DLMS implementation for Kamstrup Meter Protocol (KMP) it has been reverse-engineered here:

https://github.com/bsdphk/PyDLMS

https://github.com/RuntimeError123/hass-mc66c

https://github.com/KenanV/KamstrupMultical66

https://github.com/Hal9k-dk/kamstrup/tree/master/Software%20eksempler/kamstrup_multical402

https://github.com/Hal9k-dk/kamstrup/tree/master/Software%20eksempler/kamstrup_powermeter

https://github.com/bsdphk/PyKamstrup

https://frack.nl/wiki/Stadsverwarming

https://forum.mysensors.org/topic/3525/district-heating-city-heating-stadsverwarming-mysensor-ir-sender-receiver/

https://www.domoticz.com/forum/viewtopic.php?t=11333

As I understand most of those optical probes should work with optical infrared waves be compliant with the international standard IEC 62056-21 (formerly IEC 61107 / IEC 1107) or ANSI C12.18 communications protocols for energy metering for reading utility meters?

https://github.com/pwitab/iec62056-21

https://en.wikipedia.org/wiki/IEC_62056

https://en.wikipedia.org/wiki/Smart_meter#Protocols

Kamstrup official (and expensive) optical read-out head tools are called 6699-099 (USB) and 699-102 (9F D-sub plug serial)

https://www.termonet.rs/pdf/Kamstrup/Optical%20Read-out%20(IR)%20Head%20-%20Data%20Sheet%20-%20English.pdf

Additional context

FYI, most of the older and newer Kamstrup electricity meters look to feature a similar or same type of bi-directional optical header, so the hardware could probably be repurposed for other meters as well

PS: Off-topic but FYI; some if not all of these models of Kamstrup water meters can also be read remotely via Sigfox network RF (Radio Frequency) but that communication traffic is most of the time encrypted, at least if the water meter itself is owned and operated by a utility company or a service provider.

https://www.kamstrup.com/en-en/water-solutions/water-meter-reading

https://www.kamstrup.com/en-en/water-solutions/water-meter-reading/usb-meter-reader

If the case is that the RF radio communication is not encrypted then there look to be projects that can work with that:

https://github.com/adams-okode/kamstrup-integration

https://github.com/weetmuts/wmbusmeters

https://github.com/tobiasrask/wmbus-client

[mroek]

You should be able to find a similar dongle from other suppliers that will ship to your location.

[mdrmdrmdr]

I'm pretty sure that a wake-up of the IR port of my flowIQ 2200 only is possible by lifting the IR head. I don't think that there is a secret wake-up sequence... I could not find any hint for this. Sad!

Are there IR heads with a USB controllable electrical magnet? ;-)

[mroek]

I'm pretty sure that a wake-up of the IR port of my flowIQ 2200 only is possible by lifting the IR head. I don't think that there is a secret wake-up sequence... I could not find any hint for this. Sad!

Are there IR heads with a USB controllable electrical magnet? ;-)

You are most likely correct. I am in the process of building a reader (ESP8266 based) that automates this by having a servo-controlled magnet to trigger the meter to wake up. The magnet sensor in the meter seems to be in one particular spot, so only one powerful magnet is needed, but it needs to be placed in the correct location.

Once I have this in place, I can test the max frequency of readout, which I suspect is a handful each day, tops. I also suspect (or perhaps I should say hope) that the meter counts the number of reads, so it might be good to read multiple registers in each read, instead of one read per register.

[mdrmdrmdr]

Funny, did not think that my idea is taken seriously 😅 Would a electrical magnet not be easier than a construction with a servo? Especially for those already with a IR head? Do you already know the exact required location of the magnet?

[copystring]

Electro magnets use a lot of power to work correctly and might need to be calibrated?

[mroek]

I have experimented with an electromagnet, and I'm quite sure it would be possible, but it takes very much power to generate a strong enough field to trigger the sensor. A neodymium magnet is very strong, and the one I have been using is 10x5 mm circular. It is just really simple to use a servo to move the magnet.

I'm doing this as en exercise to develop something interesting with an ESP, but I am still going to push to gain access to the encryption key. If I succeed, this contraption is of course no longer of any use, but that's OK. As I said, I am doing it to educate myself, and I do have all the necessary components lying around anyways.

@mdrmdrmdr I had already started down this path before you mentioned it, so we both had the same idea.

[mroek]

I now have a working prototype of my reader. It does use a servo to move a magnet to wake the meter, and then performs a reading before moving the magnet out of the way again. I am currently running tests to find the limits of the polling frequency, but it will take some days until I (hopefully) have an answer.

[mdrmdrmdr]

Sounds promising. I'd prefer to connect the servo direct to my Raspberry Pi. GPIO 18 supports PWM. I already have a power IR head and a gas tick counter hooked up to it. Looking forward to your construction of servo and magnet 👍🏻

[glance-]

There's some magic modes in the meter, evidently by the addon: https://www.kamstrup.com/en-en/water-solutions/meters-devices/accessories/pulse-adapter

It can put the meter on some mode where it emits a "pulse" over the ir eye which then is turned into a open collector pulse output by the addon. The addon looks to be powered by a saft14500.

[mroek]

I have also connected the servo directly to a GPIO on the ESP8266, no problem there. The servo is pretty cheap, and it isn't all that precise. Good enough for this, though. I found out that just one 10x5 mm magnet wasn't quite enough, so I stacked two of them to get a stronger field to reliably trigger the meter.

The IR-circuit is also exceedingly simple. I've connected the TX LED to a GPIO through a resistor (the GPIO can sink around 20 mA, which is good enough), and the RX is a NPN phototransistor where the emitter is connected to ground, the collector to a GPIO (I'm using softwareserial, but it could just as well be an UART pin), and with a 1.5 k resistor to VCC (3.3V) to pull it up when there's no light.

Using a phototransistor makes for a simpler circuit, since it is more sensitive than a diode. I used a phototransistor called PT334-6C, which comes in a standard 5 mm package just like a LED.

[mroek]

There's some magic modes in the meter, evidently by the addon: https://www.kamstrup.com/en-en/water-solutions/meters-devices/accessories/pulse-adapter

It can put the meter on some mode where it emits a "pulse" over the ir eye which then is turned into a open collector pulse output by the addon. The addon looks to be powered by a saft14500.

There's no documentation on that adapter. However, even if we could put the meter into that pulse mode, it still might not stay in that mode forever. It seems that the pulse interface for the flowIQ2200 is different (or it is a newer version):

The pulse interface uses the same plastic bracket as the normal Kamstrup optical reading head, as you can see (or maybe it is the same, it's just that control box that is different). The meter also stays in the pulse mode for 9 hours after removal of the pulse interface, but there's no telling if it could stay in that mode indefinitely if the interface is left on.

And if it could indeed stay in that mode, since it is a test mode, it might not actually update the normal registers, and it might also stop broadcasting the radio telegrams. That would also be a problem when the municipality does their drive-by to read the meters...

[mdrmdrmdr]

I found out that just one 10x5 mm magnet wasn't quite enough, so I stacked two of them to get a stronger field to reliably trigger the meter.

I have such Neodyms from my last Carrera tuning actions 😊 Can you please share the location the magnet needs to touch the meter?

[mroek]

I found out that just one 10x5 mm magnet wasn't quite enough, so I stacked two of them to get a stronger field to reliably trigger the meter.

I have such Neodyms from my last Carrera tuning actions 😊 Can you please share the location the magnet needs to touch the meter?

Here's a picture:

The grey dot indicates the magnet position that I have found to work best. You can easily test this by holding the magnet to that spot for a few seconds. The display should then change mode, and by taking the magnet away and then replacing it, you can cycle the different display views. The orientation of the magnet also matters, for my meter the south pole of the magnet must face the meter. This is opposite of what the 62056-21 standard says, oddly enough.

[mdrmdrmdr]

Great, thanks a lot. I'm currently not at home, so I can't measure it, but it seems that this position is slightly outside of my 30 mm ø IR head, which would make it easier.

[mroek]

Great, thanks a lot. I'm currently not at home, so I can't measure it, but it seems that this position is slightly outside of my 30 mm ø IR head, which would make it easier.

Different shapes of magnets would also work, as long as they are strong enough. The sensor inside probably sits at a radius of around 26 mm from the center point between the LEDs, and to me it looked like it was more or less right above the RX LED, but it could be offset slightly from the position I found. I don't think it is far off, though.

[mroek]

I have been polling the device every 30 minutes, with a read of 3 registers (in one single read command) each time. After 15 responses, it went silent after 15:00 (I got a reading at 15:00, but nothing at 15:30). The next time I had a response was 01:00 the next day, and then I had 4 subsequent responses (last one at 03:00, so 5 i in total) until it went silent again.

My current best guess is that at some time of day (in my case 01:00) the meter will add to the "reading quota", and when it has been spent, it stays silent until the next time it adds to the quota. If a meter hasn't been read for a long time, it is possible to do many reads, as the quota builds every day, no matter if it is spent or not.

I do not know whether I could have had more reads if I had asked for just one register each time, but as of now I am abandoning the project, because I have received the key to my meter. I will therefore be able to pick up the radio telegrams instead, which is of course the better approach.

[mdrmdrmdr]

What documentation do you have for the KMP protocol for flowIQ 2200? So far I did read out just one register at a time for the total volume. Btw, I'll try now a linear servo. Just ordered it.

[mroek]

There is a protocol document here:

https://github.com/nabovarme/iphonedatalogger/blob/master/KMP%20Kamstrup%20Heat%20Meter%20Protocol%20MULTICAL%20601%20and%20MULTICAL%20801.pdf

It is old, and not for the flowIQ2200, but the protocol itself is the same. Creating a request for multiple registers is simple enough, and described in there.

The registers that I have been reading, are these:

0x0044: "Volume register"  (m3, as shown on the display of the meter)
0x004A: "Current flow" (liters per hour)
0x00EF: "Volume" (liters)
0x03EA: "Time"
0x03EB: "Date"
0x03E9: "Serial number"

Good luck!

[mdrmdrmdr]

That's the doc I use as well. Did you try the things in chapter 6.3? It seem that the flowIQ stores a hourly value of register 0044 and all those values can be read with one request. I think I read about that somewhere but just can't remember were.

[mroek]

That's the doc I use as well. Did you try the things in chapter 6.3? It seem that the flowIQ stores a hourly value of register 0044 and all those values can be read with one request. I think I read about that somewhere but just can't remember were.

No, I did not test any of that. I planned to, but I no longer have the need for it.

[mdrmdrmdr]

Now my servo successfully moves using the pigpio C library with PWM signals. Next is to attach the 2 magnets somehow to the servo and the servo to the IR head (without the permanent magnet). Already curious if this all works ;-)

[mdrmdrmdr]

Maybe someone here can help me on this:

This KMP protocol request "x80 x3f x10 x01 x00 x44 x4d xc0 x0d" works perfectly well for a flowIQ 2200 reading (x10) one (x01) register (x00 x44) with the total volume using an USB connected IR head. I get the correct number of m3, which is also shown on the the display of the meter.

On the other hand, this request "x80 x3F xA0 x01 x01 x00 x44 x0C x80 x17 x05 x09 x17 x2D x00 x00 x00 x00 xF1 x0D" does not work. It should read using GetLogTimePresent (xA0) from the data logger 01 (x01) one (x01) variable (x00 x44) with 12 values (x0C) and a max. respond buffer of 128 records (x80) starting at timestamp "09.05.2023 23:45:00" (x17 x05 x09 x17 x2D x00) or later.

In this case, I do not get an meaningful answer and the meter responds with almost the same bytes as the request "x80 x3F xA0 x01 x01 x00 x44 x0C x1B x7F x17 x05 x09 x17 x2D x00 x00 x00 x00 xF1 x0D", which makes no sense.

Any idea what's wrong?

And, does anybody have a explicit KMP protocol description for the flowIQ 2200? I only have it for the MULTICAL 601/801, which seems to be very similar but there might be slight differences which cause the error...

[mroek]

I never tried any of those commands, so I can't help. However, even if it worked, I'd guess that you'd still be cut off by the battery saving functionality of the meter.

I have one potential tip for you, but I don't want to mention it in public, is there some other way I could contact you (email, perhaps)?.

[mdrmdrmdr]

I'd guess that you'd still be cut off by the battery saving functionality of the meter.

Well, therefore I'd like to only wake up the meter once a day and read 24 values... possibly even rarer...

[mroek]

I'll write you an email very soon. You can delete the address again now.

[mdrmdrmdr]

Just a quick update on this:

I could now almost completely analyze and breakdown the requests and responses sent from the Kamstrup LogView HCW software to and from my flowIQ 2200 to read more than just one value. After all investigations it looks even simple :-)

In short:

• The meter stores the hour values (date / time / V1 m3-value) in a 2400 value ring-buffer (= 100 days with 24 values each)
• To find out the correct start index for the 24 hour values (00:00 ... 23:00) of the previous day (or any day within the last 100 days), I just have to read index 0 to get that date and time and then calculate the required offset to reach the date & time range requested
• Then I request in 2 steps (first 16 then 8 values; that’s how LogView does it) the 24 values

I also found out, that many basics of the KMP protocol documentation for Multical 601/801 is still valid. E.g. the calculation of date, time, integers and floats. Also the id's of the registers. And most important, the syntax for the ring-buffer index.

[cenobitedk]

@cfeenstra1024

If this meter uses the 'Kamstrup Meter Protocol' (KMP) we might be able to use the Kamstrup Multical40x component for ESPHome I've created recently.

If you have the hardware to connect an ESP to the optical interface, I can create a version for ESPHome that can be used to request available parameters. You can then test with your meter. If that works, I can extend the ESPHome component functionality.

I don't have this meter myself, but I do have the Kamstrup Multical 403 (district heating meter) which works well with the created component in ESPHome. I noticed that both meters mention a 'METERTOOL HCM' that can be used with the optical interface in their data sheets. This makes me think they might use the same protocol.

Some info about the component:

• ESPHome component for Kamstrup multical 40x
• Electronics and housing I've created for use with the Multical 40x
• Documentation

Dude! This is awesome! I believe with some modifications this can be used as generic kmp component to be used with all Kamstrup meters that supports KMP. 😁 We just need collect a big list of default register values that can be used as reference for available sensors, with option to overwrite the register id value to match any KMP compatible meter.

[tarmor1]

There's some magic modes in the meter, evidently by the addon: https://www.kamstrup.com/en-en/water-solutions/meters-devices/accessories/pulse-adapter

Hi all! Another Multical 21 user here from Estonia without access, despite asking, to the AES key. The reasoning from my local water company is that as the meter get removed+calibrated+randomly replaced to new home once every 5 years, it could be that the next home it'll be fitted to is within the receiving vicinity of my wireless MBus dongle.

Be that as it may, I got the "manual" from local Kamstrup vendor for the Pulse Adapter. As it has not been posted here yet nor seems fully visible on their own website, I'll share here.

With that now, I'm not sure if I can use that one for the registries querying as posted here without the magic of automated magnet? Guess I have to buy and try.

[gbyhtn]

I just printed an adapter to hold an IR adapter on the meter, I am going to give it a try soon. @onkelbeh: How are your experiences with your 3D-printed adapter. I#m currently working with some tube-insulation cutted to fit, but it's very clumsy. So it would be very appreciated if you can publish you 3D-modell for the Kamstrup Multical21. THX Ralf

[BerlinJoker]

@mdrmdrmdr & @cenobitedk & @mroek , this is some great stuff you pulled together here around reading the various registers and the device/protocol overall :) Has anyone of you ever successfully requested values from the long-term logging of the multical21? According to the documentation for the device ist should have storage for certain daily, monthly and even yearly values. I know those are usually not needed for continuous usage monitoring, but I'd still be interested in getting those as well.

[cenobitedk]

@BerlinJoker I don't know, but I would ask at https://github.com/htvekov/esphome_multical402 if he knows how to do that. He was successful in pulling high resolution data from registers on his meter.

[mdrmdrmdr]

@BerlinJoker: I'm pretty sure that works. In my productive setup, I'm reading daily at 01:30 am, the 24 h-values of the previous day. I trigger automatically the IR communication of the flowIQ 2200 with a neodymium magnet moved by a servo. I have to admit that I did not test it, but by just changing the requested register, it should work. I use id 0x44 "Volume register V1". To determine the proper id's, guess one needs to analyze the USB communication from and to the Kamstrup LogView HCW software with a USB sniffer while reading the week, month or year values. That's how I did it with the hour values. If you are interested, I can send you more details how I did it by mail. I won't post it here, as the approach is reverse engineering which might be a problem for Kamstrup.