search

dpoulson / ginlong-mqtt

Daemon to receive data from a Ginlong solar charge controller and input into MQTT
GNU General Public License v3.0
30 stars 16 forks source link

New Ginlong firmware #2

Open pcmvanveen opened 6 years ago

pcmvanveen commented 6 years ago

I have a Ginlong firmware version H4.01.51Y4.0.02W1.0.57(2017-12-211-D). I found out that the data string len is now 276. Also the format is changed. I already changed the decoding of the serial to get some information str(30:30)

waiting for a connection Hex data: 687c51b0875c9725875c9725810305303030413830303137373036333638200131050904a40000001300130000001500000000096c00000000138801fa00000000000602260032000004a6000000000003000000a8040e004900000021000000000000000000000000000000001027c90100860300f40300c3016c09150000000000000000000009aa16 Serial 000A80017706368 Length 276 MQTT Topic: ginlong/000A80017706368/ vpv1: 0.0 ipv1: 1.9 vac1: 0.4 iac1: 6400.0 pac: 1280.0 fac: 424.96 temp: 0.9 kwhtoday: 0.0 kwhtotal: 0.0 Finally

Please a hint to detect the right positions of the values.

francsw commented 6 years ago

I have firmware version 4.01.51Y4.0.02W1.0.57(2017-07-261-D) So far I managed to figure out the following: Char 1-64 stays constant 31-60 is the Inverter serial no. 305-378 is the firmware version. The bits inbetween I haven't been able to figure out. Here is a line of my stick's output: 687c51b05edcca255edcca258103053031304142303137423137303030322001fb0d640a590000001c00220000004c00000000091a00000000137106ea00000000000c09060852000004920000000000030000beab041f007500000000000000000000000000000000000000001027140700e10b00d40d00ff010000000000000000000b0001000dc816682951b15edcca255edcca25800148342e30312e353159342e302e303257312e302e353728323031372d30372d3236312d44292c00aa16

pcmvanveen commented 6 years ago

Meanwhile, I have made some progress. I found the value below;

vpv1 = float(int(hexdata[66:70],16))/10 vpv2 = float(int(hexdata[70:74],16))/10 ipv1 = float(int(hexdata[78:82],16))/10 ipv2 = float(int(hexdata[82:86],16))/10 vac1 = float(int(hexdata[102:106],16))/10 pac = (vpv1 ipv1 + vpv2 ipv2 ) * 0.975 # not found yet, however the calculation is reasonably accurate temp = float(int(hexdata[62:66],16))/10 fac = float(int(hexdata[114:118],16))/100 kwhtotal = float(int(hexdata[146:150],16))/10

wtreur commented 6 years ago

@pcmvanveen My inverter has the same firmware version as yours (2017-12-211-D)

After a quick look I can confirm these values are correct for me too (not entirely sure about pac as well)

I found these additional values:

kwhyesterday = float(int(hexdata[134:138],16))/100 kwhtoday = float(int(hexdata[138:142],16))/100

Edit Removed kwhlastmonth, since I'm uncertain after some more testing

pcmvanveen commented 6 years ago

I can confirm kwhtoday is working.

francsw commented 6 years ago

I managed to get most of it. So for firmware version: H4.01.51Y4.0.02W1.0.57(2017-07-261-D) These are my values: vpv1 = float(int(hexdata[66:70],16))/10 vpv2 = float(int(hexdata[70:74],16))/10 ipv1 = float(int(hexdata[78:82],16))/10 ipv2 = float(int(hexdata[82:86],16))/10 vac1 = float(int(hexdata[102:106],16))/10 vac2 = float(int(hexdata[106:110],16))/10 vac3 = float(int(hexdata[110:113],16))/10 iac1 = float(int(hexdata[90:94],16))/10 iac2 = float(int(hexdata[94:98],16))/10 iac3 = float(int(hexdata[98:102],16))/10 pac = float(int(hexdata[118:122],16)) fac = float(int(hexdata[114:118],16))/100 temp = float(int(hexdata[62:66],16))/10 kwhtoday = float(int(hexdata[138:142],16))/100 kwhyesterday = float(int(hexdata[134:138],16))/100 kwhmonth = float(int(hexdata[174:178],16)) kwhlastmonth = float(int(hexdata[182:186],16)) kwhtotal = float(int(hexdata[146:150],16)) unk1 = float(int(hexdata[74:78],16))/10 unk2 = float(int(hexdata[86:90],16))/10 unk3 = float(int(hexdata[122:126],16))/10 unk4 = float(int(hexdata[126:130],16))/10 firmware = float(int(hexdata[130:134],16)) unk6 = float(int(hexdata[158:162],16))/10 unk7 = float(int(hexdata[166:170],16))/10 unk8 = float(int(hexdata[170:174],16))/10 unk9 = float(int(hexdata[182:186],16))/10 unk5 = float(int(hexdata[142:146],16))/10

Note: unk? are values I haven't been able to tie up to something.

acmorfe commented 6 years ago

using Modbus protocol function 4, from address 2999 to 3079, these are what I have identified so far on Solis 5K-4G (H4.01.51Y4.0.02W1.0.57(2017-12-211-D)): Address Comment 3005 ac power 3007 pv power 3009 total energy 3011 month energy 3013 last month energy 3014 today energy /10 3015 yesterday energy /10 3017 year energy 3019 last year energy 3021 dc1 voltage /10 3022 dc1 current /10 3023 dc2 voltage /10 3024 dc2 current /10 3035 ac voltage /10 3036 ac current /10 3038 ac current /10 --duplicate of 3036 3041 temperature /10 3042 frequency /100 3043 status 0-2 init, 3 ready, alert 3067 ??? alert_id1 (see 3043) 3069 ??? alert_id2 (see 3043)

system clock 3072 yy 3073 mm 3074 dd 3075 hh24 3076 mi 3077 ss

still trying to find the EPM values...

XtheOne commented 6 years ago

Maybe my protocol analysis from Omnik can help. https://github.com/XtheOne/Inverter-Data-Logger/tree/master/Development

ddaddy commented 6 years ago

The ginlongmonitoring site doesn't have EPM either so i'm not hopeful the data is in there, however it would be awesome if you could find it, as having the EPM data is the only way to work out any excess energy that can be used at any time.

alekslyse commented 6 years ago

Any of you know where to find firmwares for the sticks? Would be easier if we could install the same firmware. The web panel got the option so it has to exist somewhere?

v1ckdesigns commented 4 years ago

how do you actually look for these values? mine DataStick has a way different firmware (MW_08_0501_1.58) hence hexdata is 198 long and looks like : a556001041000354aa38ef0279411400a802000000000000053c780164034d575f30385f303530315f312e35380000000000000000000000000000000000000000000000000098d8638598083139322e3136382e312e3231300000004100010105ee15

cant get any usefull data out of it..

mcrapts commented 4 years ago

I have an inverter with the same firmware. However, the message I'm getting is much shorter:

685951b024ff3a2624ff3a2681011650564c313330304e313843343030373600f005400000000000030000000000010000000008e200000000138c00190000000000000000000200000504000002c3000100000000ffff00000000000000000000000000005316

The instructions above don't seem to help to extract anything meaningful out of this message. Does anybody have an idea on how to deserialize this byte string?

Edit: actually the protocol analysis from @XtheOne did help! I'm now able to parse the messages.

codeworkx commented 4 years ago

@v1ckdesigns

the 198 byte long message is not the one you are looking for. it's the data logger registration message:

14 - 21 Data logger serial 60 - 90 Data logger firmware 140 - 151 Data logger MAC 160 - 183 Data logger IP

There's also some other data which is kind of changing. probably signal strength and so on.

If you wireshark the connection between the data logger and ginlong portal, then you might notice a much bigger message (length of hexdata 492 bytes) which appears every 5 minutes. that's the one you want.

i'm currently able to extract serial number, firmware versions and inverter model: https://github.com/codeworkx/ginlong-mqtt Working on the interesting values right now.

codeworkx commented 4 years ago

For firmware MW_08_0501_1.58: https://github.com/codeworkx/ginlong-mqtt/blob/solis4gmini/Protocol

Sometimes the data is slightly off, but overall it seems to work quite good.

I've redirected the requests from the controller to the ginlong portal to my own server using static dns entries.

data1.solarmanpv.com => yourserverip data2.solarmanpv.com => yourserverip

v1ckdesigns commented 4 years ago

this is awesome, i will try that, thanks !

joegoldman commented 4 years ago

Hi @codeworkx ,

I did similar to you, redirected port 10000 requests from my data logger to my private server. I am getting the 198 byte auth message but never get the bigger 400+ byte data message.

Is there a response to the auth message I should send back to have it start sending the data messages? This firmwares server b option just never seems to work so hijacking appears the only way.

codeworkx commented 4 years ago

@joegoldman please use latest version from here: https://github.com/codeworkx/ginlong-mqtt

joegoldman commented 4 years ago

@joegoldman please use latest version from here: https://github.com/codeworkx/ginlong-mqtt

Oh yess, i was looking at the protocol file over there didn't realise there was a whole revised listener. I appreciate your work! The Solis/Ginlong guys for my area had no idea what I was on about when trying to ask about spec etc!

craigcurtin-dev commented 4 years ago

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

dauheeIRL commented 4 years ago

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

does that mean can leave data1.solarmanpv.com and use the second entry for home server i.e. send to solarmanpv + local server?

Thanks

craigcurtin-dev commented 4 years ago

Yes that’s the theory – unfortunately what has happened now is that they have updated the protocll to V5 on the Igate/SolarMan WIFI sticks.

Based on lots of internet research it appears that the WIFI stick initially sends out a packet and then waits for a correct response packet before it will send any data –no one appears to have found out what the magic wakeup and send me data packet contains yet – it is only a matter of time though.

I am trying some magic with my linux firewall to essentially duplicate everything that is sent and received by the Inverter and to then decode that and try to find out the magic packet sequence etc

Craig

From: dauheeIRL notifications@github.com Sent: Wednesday, September 30, 2020 1:09 AM To: dpoulson/ginlong-mqtt ginlong-mqtt@noreply.github.com Cc: Craig Curtin craigc@prosis.com.au; Comment comment@noreply.github.com Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

does that mean can leave data1.solarmanpv.com and use the second entry for home server i.e. send to solarmanpv + local server?

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-700770678, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AM6OSTZKJMPMYOWLROXXOCLSIHZ67ANCNFSM4ERGQEXA.

codeworkx commented 4 years ago

V5 Stuff is here: https://github.com/codeworkx/ginlong-mqtt/blob/solis4gmini/ginlong-listen.py

craigcurtin-dev notifications@github.com schrieb am So., 4. Okt. 2020, 04:29:

Yes that’s the theory – unfortunately what has happened now is that they have updated the protocll to V5 on the Igate/SolarMan WIFI sticks.

Based on lots of internet research it appears that the WIFI stick initially sends out a packet and then waits for a correct response packet before it will send any data –no one appears to have found out what the magic wakeup and send me data packet contains yet – it is only a matter of time though.

I am trying some magic with my linux firewall to essentially duplicate everything that is sent and received by the Inverter and to then decode that and try to find out the magic packet sequence etc

Craig

From: dauheeIRL notifications@github.com Sent: Wednesday, September 30, 2020 1:09 AM To: dpoulson/ginlong-mqtt ginlong-mqtt@noreply.github.com Cc: Craig Curtin craigc@prosis.com.au; Comment < comment@noreply.github.com> Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

does that mean can leave data1.solarmanpv.com and use the second entry for home server i.e. send to solarmanpv + local server?

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-700770678>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AM6OSTZKJMPMYOWLROXXOCLSIHZ67ANCNFSM4ERGQEXA>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703190809, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJL2VIKRS2Q7VSZNL7FZLSI7MYJANCNFSM4ERGQEXA .

craigcurtin-dev commented 4 years ago

Daniel,

Thanks for that – does that do the magic packet response to get the inverter to actually send the data ?

I will set it up now and test it out

Craig

From: Daniel Hillenbrand notifications@github.com Sent: Sunday, October 4, 2020 5:44 PM To: dpoulson/ginlong-mqtt ginlong-mqtt@noreply.github.com Cc: Craig Curtin craigc@prosis.com.au; Comment comment@noreply.github.com Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

V5 Stuff is here: https://github.com/codeworkx/ginlong-mqtt/blob/solis4gmini/ginlong-listen.py

craigcurtin-dev notifications@github.com<mailto:notifications@github.com> schrieb am So., 4. Okt. 2020, 04:29:

Yes that’s the theory – unfortunately what has happened now is that they have updated the protocll to V5 on the Igate/SolarMan WIFI sticks.

Based on lots of internet research it appears that the WIFI stick initially sends out a packet and then waits for a correct response packet before it will send any data –no one appears to have found out what the magic wakeup and send me data packet contains yet – it is only a matter of time though.

I am trying some magic with my linux firewall to essentially duplicate everything that is sent and received by the Inverter and to then decode that and try to find out the magic packet sequence etc

Craig

From: dauheeIRL notifications@github.com<mailto:notifications@github.com> Sent: Wednesday, September 30, 2020 1:09 AM To: dpoulson/ginlong-mqtt ginlong-mqtt@noreply.github.com<mailto:ginlong-mqtt@noreply.github.com> Cc: Craig Curtin craigc@prosis.com.au<mailto:craigc@prosis.com.au>; Comment < comment@noreply.github.commailto:comment@noreply.github.com> Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

does that mean can leave data1.solarmanpv.com and use the second entry for home server i.e. send to solarmanpv + local server?

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-700770678>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AM6OSTZKJMPMYOWLROXXOCLSIHZ67ANCNFSM4ERGQEXA>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703190809, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJL2VIKRS2Q7VSZNL7FZLSI7MYJANCNFSM4ERGQEXA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703211423, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AM6OSTYR7HOQCKBX73UN4ADSJAKUZANCNFSM4ERGQEXA.

codeworkx commented 4 years ago

For me it does the trick. But the interpretation of the payload is not 100% accurate. Some values seem to be slightly off sometimes.

Am So., 4. Okt. 2020 um 09:09 Uhr schrieb craigcurtin-dev < notifications@github.com>:

Daniel,

Thanks for that – does that do the magic packet response to get the inverter to actually send the data ?

I will set it up now and test it out

Craig

From: Daniel Hillenbrand notifications@github.com Sent: Sunday, October 4, 2020 5:44 PM To: dpoulson/ginlong-mqtt ginlong-mqtt@noreply.github.com Cc: Craig Curtin craigc@prosis.com.au; Comment < comment@noreply.github.com> Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

V5 Stuff is here:

https://github.com/codeworkx/ginlong-mqtt/blob/solis4gmini/ginlong-listen.py

craigcurtin-dev notifications@github.com<mailto:notifications@github.com> schrieb am So., 4. Okt. 2020, 04:29:

Yes that’s the theory – unfortunately what has happened now is that they have updated the protocll to V5 on the Igate/SolarMan WIFI sticks.

Based on lots of internet research it appears that the WIFI stick initially sends out a packet and then waits for a correct response packet before it will send any data –no one appears to have found out what the magic wakeup and send me data packet contains yet – it is only a matter of time though.

I am trying some magic with my linux firewall to essentially duplicate everything that is sent and received by the Inverter and to then decode that and try to find out the magic packet sequence etc

Craig

From: dauheeIRL <notifications@github.com<mailto: notifications@github.com>> Sent: Wednesday, September 30, 2020 1:09 AM To: dpoulson/ginlong-mqtt <ginlong-mqtt@noreply.github.com<mailto: ginlong-mqtt@noreply.github.com>> Cc: Craig Curtin craigc@prosis.com.au<mailto:craigc@prosis.com.au>; Comment < comment@noreply.github.commailto:comment@noreply.github.com> Subject: Re: [dpoulson/ginlong-mqtt] New Ginlong firmware (#2)

If you want to use the Server B option go to the following URL on your Inverter

http:\your_solis_IP\config_hide.html

does that mean can leave data1.solarmanpv.com and use the second entry for home server i.e. send to solarmanpv + local server?

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-700770678>,

or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AM6OSTZKJMPMYOWLROXXOCLSIHZ67ANCNFSM4ERGQEXA>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703190809>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AADJL2VIKRS2Q7VSZNL7FZLSI7MYJANCNFSM4ERGQEXA>

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703211423>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AM6OSTYR7HOQCKBX73UN4ADSJAKUZANCNFSM4ERGQEXA>.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dpoulson/ginlong-mqtt/issues/2#issuecomment-703213595, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADJL2RIWVZOUW2HGCGGSQ3SJANTBANCNFSM4ERGQEXA .

jasonmadigan commented 3 years ago

Trying this out with a fairly recent Solis Hybrid Inverter and stick:

Version: MW_08_0501_1.58

Seeing 476 byte messages:

b'a5e1001042020246cd6fef0105051aff2d0212000000d9a0095e0100f3e3020031313046353031393331383030353320f50052113600330000002300000000007b090000000086139c080000300200001e640000000000004107000000000000000000000000000000000000d5080000470000005f0000001300a60000000d0000007d050000a2060000e8031500020019000d0005001e007a09f7002700000005000902f7000000a909240031006300fc13f400fa00fa00000000003c039900790100000d00010064010000060000007c25000049000c00000000006d2f00007b001001f5001200210001006b15'
hexdata has invalid length
Finally
codeworkx commented 3 years ago

@jasonmadigan try this one: https://github.com/codeworkx/ginlong-mqtt mine has same firmware as yours

jasonmadigan commented 3 years ago

@codeworkx will give it a try, thank you!

planetmarshall commented 3 years ago

@dpoulson This was really helpful thanks. I've written a service to adapt the v1.82 protocol if it's useful to anyone. I also have a blog entry on reverse engineering the protocol.