Open nbertram opened 5 years ago
I haven't seen this before, but actually since my server is running happily I haven't paid much attention to newer developments. From your dumps certain parts are recognizable, e.g. 0001 0005 000c 0116 XXXXX is clearly a 'PING'. It seems that 0001 0005 imply the protocol version (1.5) as opposed to the traditional 0001 0002 (1.2) version. It may take some time and patience to reverse engineer this...
Yeah I thought I'd try this as an interim measure before I build something to do modbus over RS232 instead (which is apparently an option). The inverter is a 4200MTL.
If you think there's value in reverse engineering this new protocol, I'm happy to go through it and see how similar it is to what you've implemented. I did see the PING in there, but saw the response from the server was IDENTIFY without an ACK?
I guess the other option is to simply identify the DATA packet format and just continue to proxy all unknown traffic and just extract the bit I actually need.
Anyway, thanks very much for posting your work, it's very refreshing to see a protocol reverse engineering implementation taken far enough to publish!
I stumbled upon this by chance trying to get some useful info out of the newer module. My tcpdump output looked similarly puzzling. I'll post on this issue again if I manage to extract some useful info from it. I'd like to point out that you CAN get the web interface still on this revision of the module, I don't fully recall how I accesed it but you can access it in an unconfigured state. There you can set the target IP etc.
EDIT: at least some part of the messages are xor'd with the dongle key. When I tried this on one of the smaller messages I got GrowattGro. This was pretty obvious from some of the larger messages which read just GrowattGrowattGrowatt when there are large areas with zeroed out fields. Headers of mesages seem to not be xor'd as far as I can tell. I am not sure why they made this change but it shouldn't dissuade anyone attempting to circumvent it.
Possibly a typically IoT approach to encryption... or obfuscation?
I gave up on using the Shine adapter, removed its guts and put in an ESP8266 acting as a modbus TCP bridge instead. Works fine, though this inverter seems unable to have things like a date/time register set, which means a day starts when it sees light after a long pause and ends when the inverter shuts down. 99% of the time it's right...
I think if I buy another one of these inverters I'll skip buying the wifi adapter for it, especially after the Shine app crashing constantly while trying to configure its connection.
Nice to hear you found the web interface. I was completely unable to after a large amount of prodding/resetting it, and had to persevere with the unstable app.
Yea I don't fully trust the adapter itself to be reliable either. For me the main reason I started looking in to it was it seemingly losing connection with the server but not the wifi connection.
I currently have a pretty clear idea of how this protocol works.
The first 2 bytes are a counter, This gets incremented for every time a data update is sent to the server. The next two bytes are the version ID, this seems to be 5 for my dongle. After that we get the type and message ID with 2 bytes each as in the original protocol spec done by Sciurius. The rest of the data is XOR'd with the dongle ID and the inverter ID. Messages seem to have a "random" two bytes trailing data.
I have not yet made sure if this bit of data will be enough to decode the large main data packet but the logic seems to hold for the smaller periodic packets.
message payload is xored with GrowattGrowatt... last 2 bytes seems to be a crc, just no clue how to calculate and over what
Thanks for reminding me about this thread, I wrote an implementation of the protocol which I have been using for the past two weeks or so. The protocol is basically ModbusTCP, growatt uses a different version ID. After the modbus header we have a packet identifier, followed by the packet data. I too could not find the logic for generating the CRC. I wrote the server by intercepting traffic so I don't need to be able to generate good replies for now. I'll probably upload my implementation to Github after some more cleanup.
it's CRC-16/MODBUS over the complete original data (ex crc) found this site for many crc results https://crccalc.com/
I have received my Growatt Inverter today, with a "Shine LAN Box", which seems to be using the same "1.5" protocol. I've written a system in PHP to monitor my "SmartMeter" and I've made an extension to it to monitor the Growatt. I am not intercepting, but have fully taken over the role of the server.
I've come so far as requesting the config information (Query) and I'm getting some Query Replies, but apparently my Acks to those are not good enough, because I'm not getting past that point. Can you, @Soreil share your implementation and possibly a few wireshark dumps?
Sure thing Jeroen, I'll take a look at it over the weekend. I have some dumps but they did not record the system configuration since I wasn't monitoring for those steps, it's all the regular data communcation between dongle and server. I'll also upload those PCAP for you to play with.
@Soreil thanks, but it's no longer needed. I decided to let my inverter talk to "China" for a few minutes and PCAP'ed that, with a little bit of puzzling I managed to get it fully working. It's now (well, not now, sun's set) happily sending data to the database. Once I've removed all the debugging code and cleaned it up a little I will share my code on Gitlab.
@jeroenrnl Could you share your code for protocol v1.5?
@tristan79 yes. It has been working without any problems for the past month or so. I will see if I can upload it to my Gitlab repo this evening, but be aware it is somewhat "alpha" state. I'll be happy to help to get it working though.
Tof! alpha state is ok.
@Tristan79 https://gitlab.com/jeroenrnl/nrg
I received my shinelan today (paid 3 bucks + 11 euro shipping for it ;-), stupid question but what is the login/password for the web interface, I am googling far too long with no answer. I also have the ShineWifi version. For some reason I got 2 times in the web interface after that, whatever I do (reset, cut power, etc) I can't access it... and why it has 5 different mac addresses? (I can not change the dns on the device itself, luckily I have a dns server :-) So anyway, the lan version must have the same options, but what is the password, login?
Will take me a couple of days to test your software, looking forward to it, thanks!
Try admin admin
or just "ok", without filling anything in
For wifi is is there an ssid with the serial of the wifi dongle? If Yes, connect to that password 12345678 After that the gui should be admin:admin as well
I just was going to try that but now the ip/mac is gone..., probably have to push the button like on the wifi to get it into that mode, I notice something different...
even thou 1.5 is not supported this github app still gives raw data,,, and the following i notice
for the wifi I got 5 networks...
84:0d:8e:94:01:1e 84:0d:8e:88:8d:a8 84:0d:8e:88:bd:88 cc:50:e3:27:31:6a 84:0d:8e:88:c9:a5
and this github app for the lan, I got the some mac addresses...
For some strange reason it uses normal mode 5 mac addresses (listed above) to communicate. Why? We will never know :-)
Correction:
For some strange reason it uses in normal mode, 2 of those 5 mac addresses (3th and 5th listed above) to communicate.
98:XX:63:0e:47:03 of the lan is also used
is probably equivalent to wifi cc:50:e3:27:31:6a (guessing)
strange to get the same mac addresses
also strange to get so much mac addresses with either device, you would expect just one :-)
anyway keep you updated on progress!
each (virtual)network interface card must have it's own mac address, for switches & routers to work properly Lan and wifi would each have their own MAC, virtual wifi accesspoints would also likely have their own mac. If 2 devices on the same network would have the same mac routers/switches would send any data for that mac to both devices...
Luckily I do not have 2 growatt devices (the "what it is called?" where you stick in the lan/wifi extension which I got both)... cuz swithcing from wifi to lan giving the same mac addresses, would it clash 2 "the what it is called?" devices sending from the same mac their data..???. that is a problem. And unless someone test 2 growatt "the what it is called?" devices, with either lan or wifi, this is probably a bug. Luckily most of us got only one "the what it is called?" device :-) So don't worry!
the what it is called? -> inverters i think they are called
deep trailer voice: this summer... ...two inverters... ...one shared (virtual) mac address... ...confusing around...
sub starring john cleese
:-)
serious, 2 random bought growatt lan and wifi devices sharing the same mac addresss?? Hopefully the firmware of either compensate for that, or there will be clashes! You can not have two inverters with wifilan on the same network as a result of this behaviour.
Unless the firmware compensate for it, which I highly doubt it. It's probably far less then <1% of the use cases... someone with more then one inverter on the same network :-)
If the Mac address stays the same when plugging in the other interface i would expect the interface to use the serial of the inverter to generate a Mac.
As for the networking layers May I suggest you read up on the OSI model
Seems my lan version is v2 not v5 as my wifi is.
So...
I combined the data & server script and added/hacked in some http to communicate with domoticz, since it is http url to update domoticz you probably could use that with other domotica controllers software like openhab by changing the urls.
I have a ShineWifi-X on a 5000TL-X inverter and it's using this newer protocol so the perl scripts don't work at all. @jeroenrnl your code works (mostly) and I'll pop some notes in your gitlab.
XOR with "Growatt" is a funny touch - I was wondering why it was spewing Growatt over and over again in to the tcpdump for no apparent reason. :D
So a question... I have the ShineLanBox and of course get the same growattgrowattgrowatt data. I saw thaty they used to have an Open Api http://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-Open-API-protocol-standards.pdf but it seem closed now.
I have a few questions:
As far as I know, the API is to talk to Growatt's servers. So, you let your Growatt box send the data to their service and then pull some data from there.
I think the documentation at https://github.com/sciurius/Growatt-WiFi-Tools/blob/master/README is pretty clear
I don't have a battery in my system and my inverter doesn't support connecting to one, so I don't know.
No, I am not connecting Growatt to my smart meter, I just added Growatt support to a tool that I previously built to read out the smart meter. However, my program is a totally different program than the one by @sciurius - I only built on his research to decode the data, but it is a complete separate implementation, that doesn't share any code with his. If you want support for nrg, please head over to nrg's site.
Hi,
I have a newer Growatt Wifi module (PN MR00,0004601, Shine Wifi) which doesn't seem to expose a web interface (or listen on any ports). I've managed to intercept its comms by MITMing DNS for server.growatt.com, but then found I couldn't make it communicate with your server...
Here's an excerpt of the comms if I intercept the real client and server:
etc etc etc
I haven't delved in to try to see if this protocol is similar yet, but the first stumbling block is the header seems to always be 0001 0005 in both directions. If you haven't seen this before, I guess I'll be trying to decipher it tomorrow. Just thought I'd ask first!