Closed madether closed 7 years ago
You can monitor passively via ethernet, but not if it is connected to the network with a switch. You would either need to connect the inverter to an ethernet hub (they are getting hard to find) or connect it through a raspberry pi or other device with 2 ethernet interfaces. This is due to the fact that a switch won't route traffic to ports that it is not addressed to, so that's why you aren't seeing anything with wireshark.
Once you have the hardware in place, you can use tcpdump, seextract.py, and semonitor.py to monitor in real time. Somethng like this:
tcpdump -i eth0 -U -w - | python seextract.py | python semonitor.py -k keyfile -o outfile
You will have to temporarily connect to the inverter using RS232 to get the inverter key in order to decrypt the data.
Many thanks. I had already moved the connecting to the other hub, however I think the systems still think it might be connected to the switch. My sma inverter is connected to the same hub and I can see it. Further investigation is required at my end.
Many thanks for the help.
This is what I am receiving with:
tcpdump -U -w - | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -o outfile -t n
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
Traceback (most recent call last):
File "/root/solaredge/seextract.py", line 227, in
I don't know if this will fix it, but I forgot to tell you to filter tcp packets in tcpdump. Also you do not want the -t n option on semonitor.py. Try just displaying the debug data to start with:
tcpdump -U -w - tcp | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -d stdout -vvv
As I mentioned before, you will eventually need to get the inverter key via RS232 and reference it with the -k option, however you will be able to see if you are capturing data without it.
I have reconfigured here and I am not seeing any traffic from the inverter using Ethernet. I have just used tcpdump with the basic settings and nothing. I have put a pi and the inverter on a hub . I can see other data from other devices but nothing from the inverter. Any ideas.
If you are getting data reported to the monitoring server then it obviously is sending it across the network so tcpdump and wireshark should be able to see it through the hub. You might consider getting a second ethernet interface for the RPi and configuring a bridge. I used this configuration in the past.
Thought I had a hub and spent days looking for it, found it. getting stuff but this is one of the errors.
Traceback (most recent call last):
File "/home/pi/solaredge/seextract.py", line 238, in
@madether - can you save the captured data into a file and post it so I can look at it?
tcpdump -U -w - tcp | tee capture.pcap | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -d stdout -vvv
Here you go. capture.zip
There is no SolarEdge data in that pcap file. All I see is SSH traffic between 192.168.0.2 and 192.168.0.85.
The inverter is connected to a hub and so is the Rpi. I have an RS485 connected and i don't get a whole lot out of that either.
Here is a new capture , i know there is solaredge traffic in there, checked it with wireshark. capture2.zip
Jul 12 07:07:41 data: 00 00 3c 00 00 00 b8 27 eb df 1f fe f4 6d 04 09 Jul 12 07:07:41 data: 83 25 08 00 45 00 00 28 6f e1 40 00 80 06 09 47 Jul 12 07:07:41 data: c0 a8 00 02 c0 a8 00 55 ea b7 00 16 4f d9 d9 67 Jul 12 07:07:41 data: f9 22 91 7a 50 10 00 fe 8e 82 00 00 00 00 00 00 Jul 12 07:07:41 data: 00 00 bf 09 84 57 fe 3c 0e 00 78 00 00 00 78 00 Jul 12 07:07:41 data: 00 00 34 81 c4 0d ff bb 00 27 02 13 e4 a6 08 00 Jul 12 07:07:41 data: 45 00 00 6a 02 b5 00 00 40 06 4c a4 c0 a8 00 b2 Jul 12 07:07:41 data: d9 44 90 96 0d be 56 ce 00 02 47 97 dd 31 fd 8b Jul 12 07:07:41 data: 50 10 08 68 0d 7e 00 00 Jul 12 07:07:41 Ignoring this message
Seeing a lot of Jul 12 07:07:41 Ignoring this message for some reason
Jul 12 07:20:06
root@raspberrypi:/home/pi/solaredge# tcpdump -U -w - tcp | tee /mnt/nfs/capture3.pcap | python seextract.py | python semonitor.py -d stdout -vvv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
Jul 12 07:40:44 debugEnable: True
Jul 12 07:40:44 debugFiles: True
Jul 12 07:40:44 debugMsgs: True
Jul 12 07:40:44 debugData: True
Jul 12 07:40:44 debugRaw: False
Jul 12 07:40:44 debugFileName: stdout
Jul 12 07:40:44 haltOnException: False
Jul 12 07:40:44 inFileName: stdin
Jul 12 07:40:44 serialDevice: False
Jul 12 07:40:44 networkDevice: False
Jul 12 07:40:44 networkSvcs: False
Jul 12 07:40:44 following: True
Jul 12 07:40:44 passiveMode: True
Jul 12 07:40:44 commandAction: False
Jul 12 07:40:44 masterMode: False
Jul 12 07:40:44 outFileName: stdout
Jul 12 07:40:44 append: w
Jul 12 07:40:45 opening stdin
Jul 12 07:41:59
Jul 12 07:41:59
Now I see messages going between your inverter 192.168.0.178 and the SolarEdge monitoring server. The 003d message is an encrypted data packet which it ignores until it can decrypt them. You have to wait a short time for the inverter to send a 0503 message which will enable semonitor.py to begin decrypting the 003d messages if you have specified the correct key with the -k option.
This appears to be functioning properly. Let it run for a while and see if you get some data in your output file.
So i ran the monitor for the day and no decernable output.
First of all, make sure you have the latest code from the project. Also you must first extract the inverter key using RS232 into a file and reference it when you run semonitor.py with the -k option. Refer to the description of sekey.py in the README. I don't see where you are doing that in the example that you posted above.
Run semonitor.py with -vvv and watch for a 0503 message. You should see "creating decryption object with key" in the debug output, and after that when you see a 003d message it should be followed by "decrypting message" and then a 0500 message that produces data in the output.
executing the following: root@raspberrypi:/home/pi/solaredge# tcpdump -i eth0 -U -w - | python seextract.py | python semonitor.py -k 07f16e4a6.key -d stdout -vvv tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes Jul 13 20:33:03 debugEnable: True Jul 13 20:33:03 debugFiles: True Jul 13 20:33:03 debugMsgs: True Jul 13 20:33:03 debugData: True Jul 13 20:33:03 debugRaw: False Jul 13 20:33:03 debugFileName: stdout Jul 13 20:33:03 haltOnException: False Jul 13 20:33:03 inFileName: stdin Jul 13 20:33:03 serialDevice: False Jul 13 20:33:03 networkDevice: False Jul 13 20:33:03 networkSvcs: False Jul 13 20:33:03 following: True Jul 13 20:33:03 passiveMode: True Jul 13 20:33:03 commandAction: False Jul 13 20:33:03 masterMode: False Jul 13 20:33:03 outFileName: stdout Jul 13 20:33:03 append: w Jul 13 20:33:03 keyFileName: 07f16e4a6.key Jul 13 20:33:03 key: 80bxxxxxxxxxxxxxxxe59 Jul 13 20:33:04 opening stdin
Jul 13 20:34:01
Jul 13 20:34:01
Nothing is logged after this point. (21:04)
There seems to be a lot of extraneous data that was captured. Try filtering out only the traffic going between the inverter and the SolarEdge server by IP address. You can do it either in tcpdump or seextract.py and by specifying either the address of prod.solaredge.com or your inverter.
tcpdump -i eth0 -U -w - host 217.68.144.150
tcpdump -i eth0 -U -w - host 192.168.0.178
python seextract.py -s 217.68.144.150
python seextract.py -s 192.168.0.178
Be aware that either of these addresses can change in the future.
Using
tcpdump -i eth0 host 192.168.0.178 or host 217.68.144.150 -U -w - | python seextract.py | python semonitor.py -k 07f13e4a6.key -d stdout -vvv.
Getting
Jul 14 20:29:59
This is different
Jul 14 20:37:00
Jul 14 20:37:00
Jul 14 23:22:39 dataLenInv: 7956
Jul 14 23:22:39 sequence: 00c6
Jul 14 23:22:39 source: 0074ff39
Jul 14 23:22:39 dest: 7f13e4a6
Jul 14 23:22:39 function: fffe
Jul 14 23:22:39 Discarding -13128 bytes
Jul 14 23:22:39 Exception: unpack requires a string argument of length 2
Jul 14 23:22:39
Jul 14 23:22:39
Try this and post the debug output and also the pcap file
tcpdump -i eth0 -U -w - tcp and host 217.68.144.150 | tee capture.pcap | python seextract.py | python semonitor.py -k 07f13e4a6.key -d stdout -vvv
There are heaps of 003d in the logs from both days. Even after a powercycle of the inverter I did not see a 0500 . Some from yesterday and some from today 21:45, 0745 AU EST capturex2.zip
Ok, this capture has a 503 which I think is the key exchange and now it is decrypting data but nothing usable as far as I can see. Took it most of the day before I saw the 503.
Jul 16 05:22:52
This is a more complete copy of captur4a in my previous post. This one there is an sudo>ssh>tcp from the inverter to SE and then a port change to 22221. Someone might be interested in this. capture4.zip
It looks like you may not have the latest version of the project. Line 130 of seMsg.py should be
(msgSeq, fromAddr, toAddr, function, data) = validateMsg(dataMsg[4:])
I made a change on 25 Mar that should have fixed the issue that I see in the log that you posted 2 days ago so if your copy of the source was before that date it would explain the problem.
I just made some additional changes to improve the message validation and error messages.
The port change to 22221 is something I have not seen before, but it should not affect passive monitoring.
have you been able to extract any useful information from the pcaps I have uploaded yet? Is there anyway of forcing the inverter to resend a 0503?
See my last post. It looks like you may have an outdated version of the software.
I will not be able to decrypt the data unless I have your inverter key.
Yes, I am seeing decoded inverter data in capture4.pcap using the latest version of the project and your key.
Unfortunately, it looks like the optimizer data is not being decoded. The device type is 0x0022 which is one that nobody has encountered before.
I don't have any optimizers on my system. I have to meters that are connected via rs485. ARe you able post some of the data please.
trying this an it creates an empty .json file python seextract.py /mnt/nfs/capture4.pcap | python semonitor.py -k 07f13e4a6.key -o /mnt/nfs/capture4.json
Do you have the latest version of the software? Specifically seMsg.py?
You should see this capture4.json.txt
I made the changes but not getting that at all. just a zero byte file. see my previous post.
That's exactly the command I am using as well. Here is the debug output capture4.debug.txt
I wonder if there is a way of getting the data from the other meters?
my system has 2 of these connected to the inverter http://www.all-energy.com.au/en/Exhibitors/918833/SolarEdge-Technologies-Australia-Pty-Ltd/Products/821546/SolarEdge-Modbus-Meter as well as one of these http://sol-distribution.com.au/solaredge-storedge-interface. They are all connected to the RS485 interface. would it be possible to access them via passive monitoring?
If the SolarEdge monitoring server displays the data from those devices then it would have to be somewhere in the stream you are monitoring. It's possible that is what the 0022 devices are. In order to decode the format you would have to correlate the data in the message to what you are seeing displayed. Since I don't have those devices I can't help other than to guess at a few things:
2 bytes: device type = 0x0022 4 bytes: inverter ID = 0x7f13e4a6 2 bytes: data length = 0x003a 4 bytes: timestamp 54 bytes: data
The data probably contains a device identifier towards the beginning. It appears to have some 4 byte floating point fields. You could try posting in issue #8 and see if someone who was in that discussion could help.
So were you able to see the data?
I have been able see the data from capture 4, however since I don't have a capture key for the new session, I am not able to decrypt anything since.
I am not a programmer so I don't think i will be able to extract the 0022 stuff . :(
If you have semonitor.py running and then restart the inverter it should send the 0503 message when it wakes up and you would be able to start decrypting the messages right away.
is there a way of forcing it without a powercycle,. I have to discharge the battery levels first, thats why .
You could try disconnecting it from the network for a while.
disconnected the ethernet and boom. :)
Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 06 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 ff ff 7f ff 00 00 Jul 17 01:23:07 data: 00 00 Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 07 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 01:23:07 data: 7f ff Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 08 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 01:23:07 data: 7f ff Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 09 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff
Excellent!
I just had a thought that since your meter devices are made by a different company, the format of the data may be documented somewhere, possibly by the manufacturer http://www.ccontrolsys.com/w/Home.
is there a problem with seextract not working with -f python ./seextract.py -f -o /mnt/nfs/capture8.dat /mnt/nfs/capture8.pcap The dat file does not grow.
I have a 0300 device too.
Jul 17 02:25:00 Unknown device 0x0030
Jul 17 02:25:00 data: 30 00 a6 e4 13 7f 56 00 7a ec 8a 57 54 31 36 43
Jul 17 02:25:00 data: 30 30 30 35 33 39 31 00 cd 8c d1 43 7a d8 d8 40
Jul 17 02:25:00 data: 00 00 c8 45 00 18 cd 45 00 10 09 45 95 20 01 00
Jul 17 02:25:00 data: 00 00 00 00 22 15 01 00 00 00 00 00 ff ff 7f ff
Jul 17 02:25:00 data: ff ff 7f ff 00 00 60 41 03 00 00 00 00 00 00 00
Jul 17 02:25:00 data: 00 00 2c 01 00 00 56 00 00 00 01 00 00 00
Jul 17 02:25:00 Unknown device 0x0022
Jul 17 02:25:00 data: 22 00 a6 e4 13 7f 3a 00 7a ec 8a 57 03 01 00 00
Jul 17 02:25:00 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00
Jul 17 02:25:00 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01
Jul 17 02:25:00 data: 00 00 00 00 00 00 05 00 00 00 ff ff 7f ff 00 00
Jul 17 02:25:00 data: 00 00
Jul 17 02:25:00
Jul 17 02:55:01 Unknown device 0x0022
Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 05 00 70 f8
Jul 17 02:55:01 data: 03 00 00 00 00 00 e5 78 09 00 00 00 00 00 e7 6e
Jul 17 02:55:01 data: fe ff ff ff ff ff 6c 5b 0e 00 00 00 00 00 2c 01
Jul 17 02:55:01 data: 00 00 00 00 00 00 02 00 00 00 1d 23 fe 40 00 00
Jul 17 02:55:01 data: 00 00
Jul 17 02:55:01 Unknown device 0x0022
Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 06 01 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01
Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 ff ff 7f ff 00 00
Jul 17 02:55:01 data: 00 00
Jul 17 02:55:01 Unknown device 0x0022
Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 07 01 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01
Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff
Jul 17 02:55:01 data: 7f ff
Jul 17 02:55:01 Unknown device 0x0022
Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 08 01 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01
Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff
Jul 17 02:55:01 data: 7f ff
Jul 17 02:55:01 Unknown device 0x0022
Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 09 01 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00
Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01
Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff
Jul 17 02:55:01 data: 7f ff
Jul 17 02:55:01
When writing to a file, seextract.py does not continually flush the buffer as it does when writing to stdout. It should grow the output file if you do it like this
python ./seextract.py -f /mnt/nfs/capture8.pcap > /mnt/nfs/capture8.dat
Is it at all possible to passively monitor the inverter via Ethernet and not having the inverter connected directly to a raspberry pi. Strangely wireshark does not detect any traffic from the inverter either. The monitoring site is collecting data so I know the connection works. My setup consists of a se5000, a tesla battery and some solaredge meters connect to CT's. The se5000 is connected via Ethernet to an un-managed switch. The RS-485 connector in the se5000 is connected to the solaredge meters as well as a unit that connects to the tesla battery. If I can avoid using the RS-485 that would be desirable, otherwise any suggestions about connecting the pi via RS-485 and not changing the inverter settings would also be welcome.
M