[sunspec] Add Modbus SunSpec Binding

[mrbig]

Dear openHAB developers!

This is a notification per contribution guidelines, that I started developing a new binding to support the so called "SunSpec" standard PV inverters and meters over the modbus protocol.

What is SunSpec? "SunSpec publishes free, open interoperability specifications and information models that software developers, hardware manufacturers, and integrators use to achieve plug-and-play interoperability between Distributed Energy Resource (DER) components and smart grid applications. All SunSpec specifications are free to download and use. SunSpec also supports international standard communication protocols that adhere to open principles." -- https://sunspec.org/about-sunspec-specifications/

What does that mean to openHAB? SunSpec compatible devices could be connected to openHAB regardless who the manufacturer is. There are already several certified products from various manufacturers like ABB, Fronius, Schneider Electric, SMA, SolarEdge...

Design concepts Original idea came from Thomas Kuehne's post https://community.openhab.org/t/reading-data-from-solaredge-inverters-via-modbus-tcp/26290/1 who set up monitoring with the modbus OH1 binding. But IMHO this method needs lots of tweaking mostly because there are some caveats in the SunSpec protocol. I'd like to make the sunspec binding more user friendly.

My concept is to have the inverters/meters as things, and all the data will be available as channels, OH2 style.

For the transport layer I use ssalonen's work in progress OH2 modbus binding (https://github.com/ssalonen/openhab2-addons/tree/modbus-openhab2-native-binding/addons/binding/org.openhab.binding.modbus ) which includes a very robust modbus transport I/O addon. So my binding uses the transport addon to handle all the modbus related stuff, and the handler mostly does parse and interpret the data and updates the states of the thing channels.

Work in progress code can be found here: https://github.com/mrbig/openhab2-addons/tree/3216-binding-sunspec-modbus/addons/binding/org.openhab.binding.modbus.sunspec

Since this will be my first OH binding, any feedbacks, ideas, comments are welcome!

[mrbig]

@triller-telekom you commentend in PR #2246 :

The sunspec binding however should not be another "binding" but rather a separate OSGi bundle within this generic modbus binding.

Can you please point me to another binding how this is actually done?

I was trying to follow the advices of @kaikreuzer from this conversation: https://github.com/eclipse/smarthome/issues/5301#issuecomment-376188699

[triller-telekom]

@mrbig The explanation from Kai is the proper way to do it.

Basically the extension should use the same binding id as the generic binding and not specify its own binding.xml, see https://github.com/eclipse/smarthome/blob/master/extensions/binding/org.eclipse.smarthome.binding.bluetooth.blukii/src/main/java/org/eclipse/smarthome/binding/bluetooth/blukii/BlukiiBindingConstants.java#L27

[ssalonen]

Not sure if something was changed after @triller-telekom's comment but it seems Sunspec binding seems to do just this?

See https://github.com/mrbig/openhab2-addons/blob/3216-binding-sunspec-modbus/addons/binding/org.openhab.binding.modbus.sunspec/ESH-INF/thing/inverter-types.xml#L2

[mrbig]

Thanks @ssalonen yes I believe I did the same already when @triller-telekom commented. Maybe he was looking at different branch?

Anyways now I'm a bit on hold waiting for the modbus binding's approval.

[RealMalWare]

@mrbig If you start working on this excellent approach again (now that the modbus binding is approved) I'd be happy the to support you by testing the sunspec ids 1 - common 101 - single phase inverter 103 - three phase inverter 203 - three phase meter 801 - energy storage 802 - battery 803 - lithium-ion battery with my devices.

[mrbig]

Dear @PartTimeDev thanks for reminding me. I've rebased my branch on top of the current development thread, and fixed any issues there existed. Currently 101-103 inverters are supported, will you have some time to try it out? I'd like to be interested specially in three phase inverter.

If you're willing to compile openhab yourself you can find the sourcecode here: https://github.com/mrbig/openhab2-addons/tree/3216-binding-sunspec-modbus/addons/binding/org.openhab.binding.modbus.sunspec

Or I can send you the precompilend jars as well.

[RealMalWare]

@mrbig Thanks for the quick reaction! Ran it from the debugger for now.

One phase inverter values seem to be fine - at least for my solar edge device. There might be single phase inverters where the first block is not starting at address 69.

Which brings me to the first issue regarding the three phase inverter. I need to set the start address for the block, as my three phaser is modeled like this:

address	desc	block id
40001	Well-known base address – 0x53756e53
40003	Common Model	1
40071	Energy Storage Base Model	801
40095	Battery Base Model	802
40117	Lithium-Ion Battery Model	803
40151	Inverter (Three Phase)	103
40203	(abcn) meter	203
40310	(abcn) meter external generator input	203

There should be a setting for this. Or the fancy way: parse for 0x53756e53 from the start on. The next register should hold the ID followed by its length. Jump to length-offset-register, get ID + length...

Also I can not set a Bridge as there is no "Bridge Selection" control. Same for split phase.

[mrbig]

Thanks, I've fixed the issue with the bridge not showing up.

And you are right, it was a mistake to hardcode the begin address of the blocks into the binding. I'll change it to autodetect the available blocks, this could also be a nice way to implement autodiscovery.

[mrbig]

Hello @PartTimeDev yesterday I've pushed a new version of the binding wich supports sunspec meters and autodetection of the base address. Unfortunately I do not have the possibility to test the meters against a real device, so please appreciate if there are some bugs. Will you be able to test it out? Do you need the binding jar-s?

[RealMalWare]

Detection is working already. Pretty cool :)

Sunspec detector found block ModelBlock type=1 address=2 length=68
Sunspec detector found block ModelBlock type=801 address=70 length=24
Sunspec detector found block ModelBlock type=802 address=94 length=22
Sunspec detector found block ModelBlock type=803 address=116 length=34
Sunspec detector found block ModelBlock type=103 address=150 length=52
Sunspec detector found block ModelBlock type=203 address=202 length=107
Sunspec detector found block ModelBlock type=203 address=309 length=107

But it leaves me hanging there for the 3 phase device described earlier:

Modbus/Sunspec activity ends after that. (Eclipse hangs randomly while trying to debug - is driving me nuts)

2018-09-27 00:58:18.139 [WARN ] [t.m.internal.ModbusManagerImpl:614  ] - Try 1 out of 3 failed when executing request (BasicModbusReadRequestBlueprint@35b3328d[slaveId=1,functionCode=READ_MULTIPLE_REGISTERS,start=416,length=2,maxTries=3]). Will try again soon. Error was: net.wimpi.modbus.ModbusSlaveException Error Code = 3 [operation ID 3c7cb250-3d67-4e7d-bddb-0fac21e54414]
2018-09-27 00:58:18.201 [WARN ] [t.m.internal.ModbusManagerImpl:614  ] - Try 2 out of 3 failed when executing request (BasicModbusReadRequestBlueprint@35b3328d[slaveId=1,functionCode=READ_MULTIPLE_REGISTERS,start=416,length=2,maxTries=3]). Will try again soon. Error was: net.wimpi.modbus.ModbusSlaveException Error Code = 3 [operation ID 3c7cb250-3d67-4e7d-bddb-0fac21e54414]
2018-09-27 00:58:18.264 [ERROR] [t.m.internal.ModbusManagerImpl:618  ] - Last try 3 failed when executing request (BasicModbusReadRequestBlueprint@35b3328d[slaveId=1,functionCode=READ_MULTIPLE_REGISTERS,start=416,length=2,maxTries=3]). Aborting. Error was: net.wimpi.modbus.ModbusSlaveException Error Code = 3 [operation ID 3c7cb250-3d67-4e7d-bddb-0fac21e54414]
2018-09-27 00:58:18.265 [WARN ] [b.m.s.i.detector.ModelDetector:281  ] - Error with read: org.openhab.io.transport.modbus.internal.ModbusSlaveErrorResponseExceptionImpl Slave responsed with error=3

Full log: https://gist.github.com/PartTimeDev/d116ada37fb00e7a60aa39ff61aab5cc

---

Works great for the Single Phase SolarEdge already (Besides number formatting 😬 ):

https://gist.github.com/PartTimeDev/20515f683f28ba65bccfdf6ffd69f689

[mrbig]

Thanks for the report and the logs. Looks like your device does not have an end model (id 0xffff) at the end of the block list. The detector now tries to find that one, but instead it receives a modbus exception. I'll try to handle this exception as if the end model was received.

btw: the current detector will use the first available block. Your device has 2 three phase meters (203). I'm considering how this should be selectable? Either one could enter the base address for the block but it's prone to errors, or there could be just an instance selector (eg 1, 2, 3) that will tell wichone to use if there are multiple of the same type.

[mrbig]

Hello @PartTimeDev , I've pushed a possible fix for your issue. I had to change a few thing in the modbus transport bundle as well, so make sure you recompile that as well.

[mrbig]

I would like to add auto discovery to this feature set. I basically have everything ready for that, I just have to get hold of any new modbus slave bridge that shows up in the system.

And my problem is this: modbus bridges are handled by the modbus bundle (ModbusHandlerFactory) while my sunspec code resides in the sunspec bundle (SunSpecHandlerFactory). I don't know how the two could communicate with each other? Or is there a way get notification about newly created bridges?

@triller-telekom @ssalonen do you have any hints for me?

Thanks

[RealMalWare]

getting closer... But the/my meter gives odd values. As it's the main/first meter, there should be reasonable values. Added some log lines in getScaled. This somehow looks as if the scale factors are incorrectly set, as 2018-09-29 14:41:33.646 [INFO ] [b.m.s.h.AbstractSunSpecHandler:489 ] - getScaled: 239 * 10^-32768 = 0,0000000000 should be the voltage IMO

Screenshot from PaperUI

Log excerpt truncated, after a views seconds after start.

Regarding your former question, I'd go for the index to select out of multiple identical block types

[ssalonen]

@mrbig I don't now directly but browsing the codebase found these resources:

• documentation on receiving events
• ThingAddedEvent

Perhaps you can hook into those to listen for new things?

[triller-telekom]

@mrbig Regarding your discovery problem: You should not listen to any events here, this is not reliable and also a Thing must not have a handler attached when it is created... So instead you should do it like its done in bluetooth:

The generic Binding (in your case this has to go into the generic modbus binding, i.e. please extend it) should have a ModbusDicoveryParticipant interface like the one in bluetooth:

https://github.com/eclipse/smarthome/blob/master/extensions/binding/org.eclipse.smarthome.binding.bluetooth/src/main/java/org/eclipse/smarthome/binding/bluetooth/discovery/BluetoothDiscoveryParticipant.java

And in the generic Modbus discovery service you collect all participants via OSGi:

https://github.com/eclipse/smarthome/blob/master/extensions/binding/org.eclipse.smarthome.binding.bluetooth/src/main/java/org/eclipse/smarthome/binding/bluetooth/discovery/internal/BluetoothDiscoveryService.java#L110-L119

In your sunspec binding you can implement such a participant as an OSGi service like its done in the blukii binding for example:

https://github.com/eclipse/smarthome/blob/master/extensions/binding/org.eclipse.smarthome.binding.bluetooth.blukii/src/main/java/org/eclipse/smarthome/binding/bluetooth/blukii/internal/BlukiiDiscoveryParticipant.java

If you have any questions, feel free to ask. I hope these explanations are helpful.

[mrbig]

@PartTimeDev ok, I thing I've found the problem, the scale factor was incorrectly handled by my parser. I pushed some updates, hope you'll be able to check this out.

[RealMalWare]

@mrbig tested the latest changes. Looks better now.

Seeing values the values -32768 and 32768, i guess the "not implemented" checks need to be changed? (See SMA page 13 or Sunspec page 14 )

Regarding all the 0s and 32768s I contacted the vendor for clarification if and which fields are field correctly.

Please check the (full) event log. There are way to much zeros. Besides that, some values seem to be reasonable but hiden behind some scale factor: 2018-10-03 23:56:34 - modbus_meter_wye_phase_a2df3b9c_acPhaseB_ac_exported_real_energy

[mrbig]

Hi @PartTimeDev I believe I could fix those issues by now. I refactored a lot of parts of the parsing so unsupported fields should be much better handled by now.

I've also added auto discovery, and this should be the preferred way to add new items. If you update to the branch I've pushed today, then you'll be able to turn on discovery on modbus tcp slave via the gui or by the enableDiscovery=true parameter.

Then your things can be added through discovery: they should show up at the add new thing screen and in the inbox.

If you want to add a thing manually then you should use the address and the length attributes to set the start address and the length of the block you want to get handled.

[grizzlyfred]

Hi @mrbig I am also very interested in getting readings not only from proprietary cloud service of the manufacturer but more fresh local data that is obviously there. So far I have followed Mr. Kuehne's approach and been not lucky, nothing retrieved from the SE9K and its attached Modbus meter... though all should be there... could you maybe offer a .jar - Artifact for the public to check out?

[mrbig]

Hello @grizzlyfred ,

I've attached the current version. I consider it more on less ready, with the documentation updated things seem to work fine for me.

You will have to install the modbus and the modbus io package as well. There are some changes in my version in the modbus package as well, so make sure you use the one in the tarball below: sunspec-devel-20181028.tar.gz

[mrbig]

@grizzlyfred I've update the tar.gz because the previous had some errors referencing a non existing class. Please try this one: sunspec-devel-20181029.tar.gz Also note, that my se4000 needs at least a second between tries. This can result in timeouts during the discovery. This is how I set up my bridge:

Bridge modbus:tcp:se4000 [ host="hostorip", port=502, id=1, timeBetweenReconnectMillis=1300, timeBetweenTransactionsMillis=1500, enableDiscovery=true, connectMaxTries=3]

Alternatively you can set a high reconnectAfterMillis value to keep the connection open:

Bridge modbus:tcp:se4000 [ host="hostorip", port=502, id=1, reconnectAfterMillis=3600000, enableDiscovery=true, connectMaxTries=3]

[flaviodexter]

Hi, I've installed today a solar edge inverter in my new house wich actually is stilla work in progress. I have a bench installation of HO in the old house and plan to move to the new one in the upcoming weeks. I'd love to test this binding. still its not clear to me how to configure the inverter. LAN connection seems to work only with proprietary solaredge protocol, so I guess I'll have to connect OH via RS485

[mrbig]

Hi @flaviodexter , if you have network connection working, then there is a great chance that you will able to use this binding. Take a look at this document from solaredge "MODBUS over TCP Support" chapter describes how to enable modbus support. You can leave the port on default 502 and the device id on 1.

[pernozzoli]

Hi,

trying to get it to work with a SMA Sunny Boy 4.0 inverter. Inverter has been discovered and added as thing. Status is INITIALIZING and doesn't change to ONLINE as it should. I'm currently using the 2.4.0-SNAPSHOT, any hints?

Thank you Arno

Following log entry may be interesting:

2019-01-18 14:14:58.342 [WARN ] [mmon.WrappedScheduledExecutorService] - Scheduled runnable ended with an exception: 
java.lang.NullPointerException: null
    at org.openhab.binding.modbus.sunspec.internal.handler.InverterHandler.handlePolledData(InverterHandler.java:72) ~[?:?]
    at org.openhab.binding.modbus.sunspec.internal.handler.AbstractSunSpecHandler$1.onRegisters(AbstractSunSpecHandler.java:340) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusLibraryWrapper.invokeCallbackWithResponse(ModbusLibraryWrapper.java:279) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusManagerImpl$PollOperation.lambda$1(ModbusManagerImpl.java:163) ~[?:?]
    at org.openhab.io.transport.modbus.internal.SimpleStopWatch.timeRunnable(SimpleStopWatch.java:148) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusManagerImpl$PollOperation.accept(ModbusManagerImpl.java:162) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusManagerImpl$PollOperation.accept(ModbusManagerImpl.java:1) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusManagerImpl.executeOperation(ModbusManagerImpl.java:571) ~[?:?]
    at org.openhab.io.transport.modbus.internal.ModbusManagerImpl.lambda$15(ModbusManagerImpl.java:719) ~[?:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:?]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
    at java.lang.Thread.run(Thread.java:748) [?:?]

[mrbig]

Hi @pernozzoli, thanks for trying out. Seems like a field is not implemented at your inverter, and this caused an error. I've fixed it, here's a new version: sunspec-devel-20190119.tar.gz Only the org.openhab.binding.modbus.sunspec-2.4.0-SNAPSHOT.jar has changed. Please keep me informed: I'd like to know it the binding works with vendors other than solaredge.

[pernozzoli]

Hi,

thank you so much! Unfortunately the automatic discovery doesn’t work any longer. I tried to manually add my inverter as a three phase inverter. Starting address is 40000, block size 52. Inverter goes online but apparently the values are wrong, instead of the AC power value, I get a value that seems to be the AC frequency (around 50 here in Germany). So basically the values are offset around 4 bytes. Am I doing something wrong? May be I selected the wrong address, documentation clearly states, that it starts at 40000. I’m not really sure about the block length...

I would really like to support your development by providing any information or contribution you need. It would also be nice to add the SMA Sunny Island profile to the binding. I could provide the required information and at least it would need a few additional channels for battery related parameters.

Cheers Arno

[mrbig]

Hello @pernozzoli Modbus is a strange animal, and addresses are sometimes offsetted a bit. So it's ok to try 40001 or 40002. The binding does only read, so you can't do any harm. You can turn on logging for org.openhab.io.transport.modbus and the org.openhab.bingind.modbus.sunspec packages then all the communication will be logged to the system log. (See this article how to turn it on: https://www.openhab.org/docs/administration/logging.html#defining-what-to-log ) If you could send me those logs about the auto discovery then I could possibly find out what's going wrong.

[pernozzoli]

Hi,

thanks for the hint, I was able to find out by myself. I was missing the "enableDiscovery" attribute within my manual things configuration. Now the inverters were discovered automatically, as far as I can judge, all available values are correct. The Sunny Island battery inverter was discovered as well. Of course the battery specific values are missing as the binding doesn't support them yet. If you would like to add this to your binding, just tell me what kind of information you need for. I would be glad to provide it. Currently I'm getting the values via default modbus poller and data.

Cheers Arno

[mrbig]

Thanks @pernozzoli I'm glad to hear it works well!

I realized that the enableDiscovery was a bit hidden in the README, so I made it more clear.

Regarding the battery model: what I'm mostly lacking is free time :) I have the documentation and everything available, but first I'd like to get the main binding accepted by openhab.

[fraenki79]

Hi @mrbig ,

I tried your compiled sunspec binding (sunspec-devel-20190119.tar.gz), but I got the following error at startup:

2019-11-13 17:58:47.363 [WARN ] [org.apache.felix.fileinstall ] - Error while starting bundle: file:/usr/share/openhab2/addons/org.openhab.binding.modbus.sunspec-2.4.0-SNAPSHOT.jar org.osgi.framework.BundleException: Could not resolve module: org.openhab.binding.modbus.sunspec [225] Unresolved requirement: Import-Package: org.eclipse.jdt.annotation; resolution:="optional" Unresolved requirement: Import-Package: org.openhab.binding.modbus.discovery

I read that this could happen if it was compiled for another openhab version. I use openhab 2.4.0-1.

If that is the problem, would you please provide a new compiled version of your binding. I would like to try it with my Kostal Plenticore Plus Inverter. Thank you.

Regards Frank

[mrbig]

Hello @fraenki79,

did you install the modbus package as well? The required components are part of that bundle.

[ssalonen]

I think this issue is solved by PRs merged above?

[mrbig]

You are right, I have absolutely forgot about this issue.

This has already been merged, and big thanks everyone for the help!

[AchimWilden]

Hello, I run a PV inverter and a battery inverter (both from SMA) at home, as well as a wallbox and make everything a little smarter. I've never heard of SunSpec before, so I've tried to read it and ended up here. It all reads pretty well, but I can't figure out how to use it. I have an openHab installation running, the Modbus connection is used for the wallbox, for example, but what steps do I have to take to use autodicovery for SunSpec compatible inverters ... I am not sure what to download and where to copy it to.

Then the next question: The last comment is two years old. Is the project still alive?

[mrbig]

Hello @AchimWilden yes, it's still alive, and it has been merged into the main openhab release. Main documentation can be found here: https://www.openhab.org/addons/bindings/modbus.sunspec/ You should install both the modbus and the sunspec bindings. You can do this using the paperui as well. You can the set it up using the paper ui as well: you will need to add first a modbus bridge (tcp or serial based on how you connect to your inverter), then turn on auto discovery on this bridge, then do a scan. If your inverter talks the protocol, then it should appear in the "inbox"

[AchimWilden]

Hi @mrbig, thanks for the answer. This is exactly how I would have imagined the installation. But my problem is, when I search for "sun" under Addon in the PaperUI, there is no match in any category (apart from meetings related to any Samsung addons). What am I doing wrong?

[ssalonen]

@AchimWilden you might be suffering from this: https://github.com/openhab/openhab-addons/issues/8538

Manual installation is one workaround...

That might be fixed with latest 2.5.x release but I am not sure. What version on openHAB do you have?

EDIT: installation might be simpler https://community.openhab.org/t/modbus-binding-with-sunny-boy-sma-inverter/81847/142?u=ssalonen

[AchimWilden]

Hi, openHAB runs at version 2.5.10-1. Okay, manual installation but how do I do it. That was the point of my initial question "what do I copy where"? Sorry if I ask such simple questions, I am still at the beginning with my OH knowledge: - /

[ssalonen]

No problem!

EDIT: installation might be simpler https://community.openhab.org/t/modbus-binding-with-sunny-boy-sma-inverter/81847/142?u=ssalonen

First install modbus binding via paper ui.

The other modbus based addons like modbus.e3dc or modbus.sunspec you need install manually using the below instructions.

You can download the addons from https://www.openhab.org/download/

Under manual installation there is "Download openHAB 2.5.10 stable addons"

• rename the downloaded .kar file to .zip
• Unzip
• find addon jars from the following locations of the extracted files: /repository/org/openhab/addons/bundles/
• copy the jar file to your openhab addons folder

[AchimWilden]

A first hard-fought success. The addon is installed. Now the documentation of the binding finally picks up but the next question comes up.

I have now created a thing file based on the example in the documentation, with the following content

Bridge modbus:tcp:bridge [ host="192.168.133.16", port=502, id=3, enableDiscovery=true ] Thing modbus:inverter-three-phase:bridge:STP10000 "STP10000" (modbus:tcp:modbusbridge) [ address=40069, length=52, refresh=15 ]

What is still unclear to me is, what is this: modbus: inverter-three-phase: bridge: In the example there is SE4000h, which is an inverter. Is that just a label and does it not matter what I write there or is there a lookup table with suppoertet inverter somewhere?

I want to read out an SMA STP10000. Where can I get the starting address? Or is that always register 40069? I have found the definition of the Modbus register but which part the start register should point to is unclear.

Thanks for the help. Without them I would have given up in frustration :-)

[ssalonen]

@AchimWilden you should proceed creating a new thread in the community forums. The question will get more attention there

[AchimWilden]

correct you are right. I do it directly

[mrbig]

@AchimWilden those are only labels. But please avoid setting up the inverter manually unless you really need this. The autodiscovery will try to find the starting address automatically wich can be a bit tricky if done by hand. If the device is found then it will be reported in undert the device properties, so if you really wish, you could later copy them into a file config.