Norwegian Smartmeter Kaifa MA304H3E

[thomasja27]

Hello

Norway is getting new smart meters by 2019 (Kaifa MA304H3E with 2400baud 8n1 serial mbus). Got mine couple of days ago, now Im trying to read data from it, using using this library.

Is this meter supported? Im getting a lot of data when I run debug - 2. See attachment (PDF) for more info about the meter and data. (got this from my power company)

D0Protocol CREATE SERIALPORT: 2400 8 1 none SERIALPORT OPEN ADD NEW DATA (0 + NEW 116) CURRENT PROCESS STEP 0 IN CHECKMESSAGE MATCH-RESULT SIGNON: "\u0000\u0000\u0012!�1�'\u0001\u0002\u0001\u0010Z���\u0000\u000f@\u0000\u0000\u0000\t\f\u0007�\u0003\u0006\u0002\b#\u001c��\u0000\u0000\u0002\u0001\u0006\u0000\u0000\u0012\u0018B��y\u0001\u0002\u0001\u0010����\u0000\u000f@\u0000\u0000\u0000\t\f\u0007�\u0003\u0006\u0002\b#\u001e��\u0000\u0000\u0002\r\t\u0007KFM_001\t\u00106970631403607026\t\bMA304H" -> null SET MESSAGE TIMEOUT TIMER: 120000

SmlProtocol CREATE SERIALPORT: 2400 8 1 none SERIALPORT OPEN ADD NEW DATA (0 + NEW 106) MATCH-RESULT MESSAGE: "00000a77f93e7e7ea027010201105a87e6e7000f40000000090c07e2030602083904ff80000002010600000a8420fc7e7ea027010201105a87e6e7000f40000000090c07e2030602083906ff80000002010600000a9593ff7e7ea027010201105a87e6e7000f40000000" -> null

Im using this config: var SmartmeterObis = require('smartmeter-obis');

var options = { 'protocol': "D0Protocol", 'transport': "SerialResponseTransport", 'transportSerialPort': "/dev/tty.usbserial", 'transportSerialBaudrate': 2400, 'transportSerialDataBits': 8, 'transportSerialStopBits': 1, 'transportSerialParity': 'none', 'debug': 2, };

function displayData(err, obisResult) { if (err) { // handle error // if you want to cancel the processing because of this error call smTransport.stop() before returning // else processing continues return; } for (var obisId in obisResult) { console.log( obisResult[obisId].idToString() + ': ' + SmartmeterObis.ObisNames.resolveObisName(obisResult[obisId], options.obisNameLanguage).obisName + ' = ' + obisResult[obisId].valueToString() ); }

}

var smTransport = SmartmeterObis.init(options, displayData);

smTransport.process();

setTimeout(smTransport.stop, 60000);

S1001_Kaifa HAN OBIS codes KFM_001.pdf HAN data stream 2017-2-14.pdf

[Apollon77]

This module currently do not support mbus.

I'm currently looking into it in general (also for #4) and was able to read un-encrypted mbus and created a very new nodejs library for this that could be used as base. So if you would like to support I would be happy if you can test this.

create a new directory on your computer and run

npm install node-mbus --debug --save

then you can find in node_modules/node-mbus/example.js a small example script to query data when you know the mbus id. Do you know it?

if not you would need to change the call from "getData" to "scanSecondary" (and remove the first parameter, so only have the callback) to scan the mbus network for the secondary ID. you can use this then for getData.

The "--debug" on npm install above will enable some additional serial logging to exactly see the messages and such. SO it would be great (independently from the result) if you can send the output :-)

[Apollon77]

PS: If it would be a smartmeter with a P1 port and sending DSMR over this then D0 and this library already was proofed to work ... but mbus is different

[thomasja27]

I can make it a try :) I got a error when I try to install: Thomass-Mac-mini:~ thomasjakobsen$ cd mbus Thomass-Mac-mini:mbus thomasjakobsen$ npm install node-mbus --debug --save

node-mbus@0.1.3 install /Users/thomasjakobsen/node_modules/node-mbus node-gyp rebuild

CC(target) Debug/obj.target/libmbus/libmbus/mbus/mbus-protocol-aux.o CC(target) Debug/obj.target/libmbus/libmbus/mbus/mbus-protocol.o CC(target) Debug/obj.target/libmbus/libmbus/mbus/mbus-serial.o CC(target) Debug/obj.target/libmbus/libmbus/mbus/mbus-tcp.o CC(target) Debug/obj.target/libmbus/libmbus/mbus/mbus.o LIBTOOL-STATIC Debug/mbus.a Usage: /usr/local/bin/libtool [OPTION]... [MODE-ARG]... Try 'libtool --help' for more information. libtool: error: unrecognised option: '-static' make: *** [Debug/mbus.a] Error 1 gyp ERR! build error gyp ERR! stack Error: make failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23) gyp ERR! stack at emitTwo (events.js:106:13) gyp ERR! stack at ChildProcess.emit (events.js:191:7) gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12) gyp ERR! System Darwin 16.7.0 gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /Users/thomasjakobsen/node_modules/node-mbus gyp ERR! node -v v6.10.0 gyp ERR! node-gyp -v v3.6.2 gyp ERR! not ok npm WARN enoent ENOENT: no such file or directory, open '/Users/thomasjakobsen/package.json' npm WARN thomasjakobsen No description npm WARN thomasjakobsen No repository field. npm WARN thomasjakobsen No README data npm WARN thomasjakobsen No license field.

npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! node-mbus@0.1.3 install: node-gyp rebuild npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the node-mbus@0.1.3 install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in: npm ERR! /Users/thomasjakobsen/.npm/_logs/2018-03-06T09_10_14_764Z-debug.log Thomass-Mac-mini:mbus thomasjakobsen$

[thomasja27]

If you look at the attachment in the first post you will se the datastream from the meter. (got this from the tech division at the power company)

[Apollon77]

What MacOS version you sue there? What outputs libtool -v? Please install https://github.com/nodejs/node-gyp#on-mac-os-x ... Then try again please

The datastream in the document is not MBus really format that I'm aware of ... or it is incomplete ... I checked it. Also that the device sends data without request is special somehow. So I wanted to see how the device reacts on "real" mbus traffic.

[thomasja27]

Running OS X Sierra 10.12.6

Thomass-Mac-mini:~ thomasjakobsen$ libtool -v Usage: /usr/local/bin/libtool [OPTION]... [MODE-ARG]... Try 'libtool --help' for more information. libtool: error: you must specify a MODE

Wait a moment, i will try to install gyp

[Apollon77]

Not install gyp, install XCode andf commandlinetools ... as written directly on "MacOS" :-)

[Apollon77]

The more I look into it it seems that mbus is only the transport medium and your smartmeter is using "OMS" (see #4) and in this case also special OMS messagetypes like "7E" which transports SML somehow ...

[thomasja27]

Ok, I finally got the node-mbus library installed.

Tried to run the script, but there is no output.

var MbusMaster = require('/Users/thomasjakobsen/node_modules/node-mbus/index.js');

/ var mbusOptions = { host: '127.0.0.1', port: port, autoConenct: true }; / var mbusOptions = { serialPort: '/dev/tty.usbserial', serialBaudRate: 2400, autoConenct: true };

var mbusMaster = new MbusMaster(mbusOptions);

if (!mbusMaster.connect()) { console.log('Connection failed.'); process.exit(); }

// request for data from devide with ID 1 mbusMaster.scanSecondary(function(err, data) { console.log('err: ' + err); console.log('data: ' + JSON.stringify(data, null, 2));

mbusMaster.scanSecondary(function(err, data) {
    console.log('err: ' + err);
    console.log('data: ' + JSON.stringify(data, null, 2));

    mbusMaster.close();
});

});

[thomasja27]

When I run "screen /dev/tty.usbserial 2400" to monitor the serial port, it push a lot of data, every 2 sec it adds more, an every 10 sec it shows these numbers below:

������'Z���@ ����4�y����@ � KFM_001 6970631403607026 MA304H3Ekp$. I P�w�'Z���@ ���e��'KFM_001 6970631403607026���\RMA304H3EOt#�-���y����@ G�� ND��'Z���@ ���L@E�'Z� ��@ ���JKFM_001�6970631403607026��@ MA304H3EB{#A-<�@ ��� F N:��'Z���@ ���D���'Z���@

[thomasja27]

https://github.com/roarfred/AmsToMqttBridge/blob/master/Samples/Kaifa/obisdata.md

[Apollon77]

Can you capture your data in hex tinderbox this matches with the samples ?! Thank you

[thomasja27]

7ea027010201105a87e6e7000f40000000090c07e2030602083904ff80000002010600000a8420fc7e

7ea027010201105a87e6e7000f40000000090c07e2030602083906ff80000002010600000a9593ff7e

This is from my meter

[thomasja27]

the meter pushes out this every 2 secounds

[thomasja27]

10 sec 7ea027010201105a87e6e7000f40000000090c07e2030703081130ff80000002010600000cdb6e677e7ea079010201108093e6e7000f40000000090c07e2030703081132ff800000020d09074b464d5f30303109103639373036333134303336303730323609084d413330344833450600000cd206000000000600000000060000024b0600002efd06000031c706000008b106000009470600000000060000094d80857e

2sec 7ea027010201105a87e6e7000f40000000090c07e2030703081134ff80000002010600000cc98b517e

2sec 7ea027010201105a87e6e7000f40000000090c07e2030703081136ff80000002010600000ccf06367e

2sec 7ea027010201105a87e6e7000f40000000090c07e2030703081138ff80000002010600000cc34bf17e

2sec 7ea027010201105a87e6e7000f40000000090c07e203070308113aff80000002010600000cc9aa5c7e

10sec 7ea079010201108093e6e7000f40000000090c07e2030703081200ff800000020d09074b464d5f303031091 03639373036333134303336303730323609084d413330344833450600000cc20600000000060000000006000002470600002ed606000031b506000008a006000009440600000000060000094cd74c7e7ea027010201105a87e6e7000f40000000090c07e2030703081202ff80000002010600000cc5b5ac7e7ea027010201105a87e6e7000f40000000090c07e2030703081204ff80000002010600000cc34ece7e

There is a telegram every 2sec and 10sec There is a hour telegram also, but i have not catched it yet.

[thomasja27]

Found a program that works with my meter! https://www.dropbox.com/sh/avdtlvaad3jabs9/AAAqdC9z51MllaxBt72-OaMXa?dl=0

[Apollon77]

cool, I will have a look on it.

In general it is no real mbus, it is some kind of OMS/mbus/whatever protocol :-) And in fact it is "manufactorer specific data format" ... I need to think about it how to incorporate this in here (or another library?) ...

[Apollon77]

One more question: how exactly you get the data out of the meter? I read that in Dutch households Kaifa meter also provide DSMR over a do called P1 socket. Is this available too?

[sveip]

@Apollon77, you are correct, it's OBIS over MBUS (just as electrical interface) There is a spec on the OBIS codes, I'll see if I can find them and link it here. All meters in Norway are to use this interface (OBIS over MBUS serial.)

[sveip]

Here is a repo with info that will be useful: https://github.com/roarfred/AmsToMqttBridge/tree/master/Documentation

And at the end of this doc, the OBIS codes are documented: http://docplayer.me/37212360-Informasjon-til-kundene-via-han-grensesnittet-i-ams-maleren-obiskoder.html

[Apollon77]

Yes as you said. Seems so it is using mbus transport layer definition, but transporting custom messages over it. The messages are not mbus but it's own format starting with "7E" ... and return an "ordered list" of data which are then (per hopefully standard definition) mapped to OBIS codes ... How standard is that "Norwegian HAN spesification"? Do all "HAN compatible smartmeter" do this encoding?

So when I see correctly it could be a "SerialResponseTransport" as standard used and only need a "HANProtocol" class/implementation that collects and parses the message.

So if someone can/want to implement a "parser" for those messages for nodejs (as simple class that gets the data as "Buffer" and returns the list or already mapped to OBIS as object with the data or such) it could speed up the topic a lot :-))

[sveip]

Yes, it's a standard that all meeters will have to comply with. About the parsing there are some examples in python and soon go, so should be possible.

[sveip]

Here is a repo with a partial implementation https://github.com/Danielhiversen/AMSreader

[robinsmidsrod]

@Apollon77 Actually, the 7E is the start/stop flag for HDLC frames (related to PPP), as mentioned in the PDF excerpts (BB12, GB8) on roarfred's repo. And the messages itself is is some form of DLMS. They are standardized, but the standard costs around 2000 NOK to buy from standard.no (as described here: https://www.nek.no/info-ams-han-utviklere/).

[gpitzer]

Hi there, I refer to the problem from the iobroker forum: ##https://forum.iobroker.net/topic/4685/adapter-smartmeter/1424 I have a Landis & Gyr E450 smart meter from Netz Burgenland (BEWAG) with a DLMS protocol on the optical interface. The data is not encrypted. Here is a detailed description of the smart meter in German, and a few logs. The log in the debug log is SmartMeter Language 1.0.3 and D0. 1577452355174-netz_burgenland_beschreibung_endkundenschnittstelle_02.pdf iobroker.2019-12-31.log

What needs to be done to get this adapter up and running? I am happy to provide further data. best regards gpitzer

[Apollon77]

@gpitzer could you provide the data as text and not screenshot ;-) Else i need to „convert“ or by hand ;-)

And for the question what needs to be done: I need to find time to look into this. Hopefully during January. But I can not fully promise!

If someone else finds time I’m happy to take PRs

[gpitzer]

Hello, Sorry for that, encluded the data from the RealTearm as text.

7E A0 5A CE FF 03 13 9D 15 E6 E7 00 0F 00 02 A3 13 0C 07 E4 01 02 04 09 10 37 FF 80 00 00 02 0E 12 00 E6 12 00 EC 12 00 E8 12 00 89 12 00 5A 12 00 69 06 00 00 00 00 06 00 00 01 F5 06 00 14 19 8D 06 00 03 90 2D 12 00 F2 12 00 AB 12 00 C4 09 08 33 30 31 33 32 30 31 33 54 B3 7E

No problem if it takes longer, it is not vital ;) regards gpitzer