Passive ethernet monitoring

[madether]

Is it at all possible to passively monitor the inverter via Ethernet and not having the inverter connected directly to a raspberry pi. Strangely wireshark does not detect any traffic from the inverter either. The monitoring site is collecting data so I know the connection works. My setup consists of a se5000, a tesla battery and some solaredge meters connect to CT's. The se5000 is connected via Ethernet to an un-managed switch. The RS-485 connector in the se5000 is connected to the solaredge meters as well as a unit that connects to the tesla battery. If I can avoid using the RS-485 that would be desirable, otherwise any suggestions about connecting the pi via RS-485 and not changing the inverter settings would also be welcome.

M

[jbuehl]

You can monitor passively via ethernet, but not if it is connected to the network with a switch. You would either need to connect the inverter to an ethernet hub (they are getting hard to find) or connect it through a raspberry pi or other device with 2 ethernet interfaces. This is due to the fact that a switch won't route traffic to ports that it is not addressed to, so that's why you aren't seeing anything with wireshark.

Once you have the hardware in place, you can use tcpdump, seextract.py, and semonitor.py to monitor in real time. Somethng like this:

tcpdump -i eth0 -U -w - | python seextract.py | python semonitor.py -k keyfile -o outfile

You will have to temporarily connect to the inverter using RS232 to get the inverter key in order to decrypt the data.

[madether]

Many thanks. I had already moved the connecting to the other hub, however I think the systems still think it might be connected to the switch. My sma inverter is connected to the same hub and I can see it. Further investigation is required at my end.

Many thanks for the help.

[madether]

This is what I am receiving with:

tcpdump -U -w - | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -o outfile -t n

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes Traceback (most recent call last): File "/root/solaredge/seextract.py", line 227, in readPcapRec(pcapFile) File "/root/solaredge/seextract.py", line 75, in readPcapRec sys.stdout.flush() IOError: [Errno 32] Broken pipe 6 packets captured 6 packets received by filter 0 packets dropped by kernel

[jbuehl]

I don't know if this will fix it, but I forgot to tell you to filter tcp packets in tcpdump. Also you do not want the -t n option on semonitor.py. Try just displaying the debug data to start with:

tcpdump -U -w - tcp | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -d stdout -vvv

As I mentioned before, you will eventually need to get the inverter key via RS232 and reference it with the -k option, however you will be able to see if you are capturing data without it.

[madether]

I have reconfigured here and I am not seeing any traffic from the inverter using Ethernet. I have just used tcpdump with the basic settings and nothing. I have put a pi and the inverter on a hub . I can see other data from other devices but nothing from the inverter. Any ideas.

[jbuehl]

If you are getting data reported to the monitoring server then it obviously is sending it across the network so tcpdump and wireshark should be able to see it through the hub. You might consider getting a second ethernet interface for the RPi and configuring a bridge. I used this configuration in the past.

[madether]

Thought I had a hub and spent days looking for it, found it. getting stuff but this is one of the errors. Traceback (most recent call last): File "/home/pi/solaredge/seextract.py", line 238, in readPcapRec(pcapFile) File "/home/pi/solaredge/seextract.py", line 74, in readPcapRec sys.stdout.write(pcapFile.read(dataLen)) OverflowError: Python int too large to convert to C long

[jbuehl]

@madether - can you save the captured data into a file and post it so I can look at it?

tcpdump -U -w - tcp | tee capture.pcap | python /root/solaredge/seextract.py | python /root/solaredge/semonitor.py -d stdout -vvv

[madether]

Here you go. capture.zip

[jbuehl]

There is no SolarEdge data in that pcap file. All I see is SSH traffic between 192.168.0.2 and 192.168.0.85.

[madether]

The inverter is connected to a hub and so is the Rpi. I have an RS485 connected and i don't get a whole lot out of that either.

[madether]

Here is a new capture , i know there is solaredge traffic in there, checked it with wireshark. capture2.zip

[madether]

Jul 12 07:07:41 data: 00 00 3c 00 00 00 b8 27 eb df 1f fe f4 6d 04 09 Jul 12 07:07:41 data: 83 25 08 00 45 00 00 28 6f e1 40 00 80 06 09 47 Jul 12 07:07:41 data: c0 a8 00 02 c0 a8 00 55 ea b7 00 16 4f d9 d9 67 Jul 12 07:07:41 data: f9 22 91 7a 50 10 00 fe 8e 82 00 00 00 00 00 00 Jul 12 07:07:41 data: 00 00 bf 09 84 57 fe 3c 0e 00 78 00 00 00 78 00 Jul 12 07:07:41 data: 00 00 34 81 c4 0d ff bb 00 27 02 13 e4 a6 08 00 Jul 12 07:07:41 data: 45 00 00 6a 02 b5 00 00 40 06 4c a4 c0 a8 00 b2 Jul 12 07:07:41 data: d9 44 90 96 0d be 56 ce 00 02 47 97 dd 31 fd 8b Jul 12 07:07:41 data: 50 10 08 68 0d 7e 00 00 Jul 12 07:07:41 Ignoring this message

Seeing a lot of Jul 12 07:07:41 Ignoring this message for some reason

Jul 12 07:20:06 --> message: 5 length: 66 Jul 12 07:20:06 dataLen: 002c Jul 12 07:20:06 dataLenInv: ffd3 Jul 12 07:20:06 sequence: 0335 Jul 12 07:20:06 source: fffffffd Jul 12 07:20:06 dest: ffffffff Jul 12 07:20:06 function: 003d Jul 12 07:20:06 Ignoring this message

[madether]

root@raspberrypi:/home/pi/solaredge# tcpdump -U -w - tcp | tee /mnt/nfs/capture3.pcap | python seextract.py | python semonitor.py -d stdout -vvv tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes Jul 12 07:40:44 debugEnable: True Jul 12 07:40:44 debugFiles: True Jul 12 07:40:44 debugMsgs: True Jul 12 07:40:44 debugData: True Jul 12 07:40:44 debugRaw: False Jul 12 07:40:44 debugFileName: stdout Jul 12 07:40:44 haltOnException: False Jul 12 07:40:44 inFileName: stdin Jul 12 07:40:44 serialDevice: False Jul 12 07:40:44 networkDevice: False Jul 12 07:40:44 networkSvcs: False Jul 12 07:40:44 following: True Jul 12 07:40:44 passiveMode: True Jul 12 07:40:44 commandAction: False Jul 12 07:40:44 masterMode: False Jul 12 07:40:44 outFileName: stdout Jul 12 07:40:44 append: w Jul 12 07:40:45 opening stdin Jul 12 07:41:59 Jul 12 07:41:59 --> message: 1 length: 64536 Jul 12 07:42:00 Jul 12 07:42:00 --> message: 2 length: 404 Jul 12 07:42:00 dataLen: 002c Jul 12 07:42:00 dataLenInv: ffd3 Jul 12 07:42:00 sequence: 01ab Jul 12 07:42:00 source: 7f13e4a6 Jul 12 07:42:00 dest: fffffffe Jul 12 07:42:00 function: 003d Jul 12 07:42:00 Discarding 338 bytes Jul 12 07:42:00 data: c8 61 f8 4e f8 96 6f e4 4e 47 94 f5 c8 4a b2 0a Jul 12 07:42:00 data: 0c f2 68 4d c4 3e 61 88 69 a7 f1 cf c5 3a 1c 2c Jul 12 07:42:00 data: 60 69 76 5f b0 41 bd 06 08 24 8d 53 33 68 c3 51 Jul 12 07:42:00 data: 0b c4 39 e7 bf 0f dc 02 ca d4 75 03 55 51 a0 ac Jul 12 07:42:00 data: 20 0e 31 18 79 0b ca 5c 93 50 fb 03 da 93 12 12 Jul 12 07:42:00 data: 13 14 1a 9b c8 cd 63 2c 4e bb 48 aa 5a 08 44 6a Jul 12 07:42:00 data: 9c 84 4e ba 94 0e aa fe 02 b1 61 93 f5 13 52 b6 Jul 12 07:42:00 data: 96 e2 f8 9a 6a af c9 45 8f a8 ff 3b 66 cf 32 7e Jul 12 07:42:00 data: 01 1e 96 a9 3d ce 06 06 b7 d8 50 d6 34 2b ca 12 Jul 12 07:42:00 data: 00 00 00 00 00 00 69 d5 35 73 17 70 b0 d6 f8 8d Jul 12 07:42:00 data: 46 3d 14 15 94 9b 94 67 57 97 7c f0 25 51 de 06 Jul 12 07:42:00 data: 92 5f 4f a8 56 5d 87 ac 75 25 32 c6 93 6b cc cf Jul 12 07:42:00 data: ed 07 70 65 47 cb b9 c9 a1 de e4 ce e7 78 30 cb Jul 12 07:42:00 data: fc 26 f0 49 bc 2e 31 30 3e 64 7f bd 38 d3 b4 df Jul 12 07:42:00 data: bc a0 c9 dd eb f2 22 fc c1 df 1c 58 ea d6 56 59 Jul 12 07:42:00 data: e9 ed 26 3f e2 ef 3a ca 02 29 f7 01 d9 93 90 0a Jul 12 07:42:00 data: 30 c6 99 fd 7f 20 a0 1b 03 d2 5e f1 ed c3 1c d1 Jul 12 07:42:00 data: 94 2e ca d6 c5 4a a4 a4 f2 d8 49 e6 89 0c cd 55 Jul 12 07:42:00 data: f3 54 fc 72 46 34 3a a6 76 31 e1 3f 9a d4 ba 1c Jul 12 07:42:00 data: f3 7a f5 6b 26 14 b9 56 ab 1a 85 80 0a 31 d1 39 Jul 12 07:42:00 data: ff 98 cb 6c 81 75 00 00 00 00 00 00 00 00 00 00 Jul 12 07:42:00 data: 00 00 Jul 12 07:42:00 Ignoring this message

[jbuehl]

Now I see messages going between your inverter 192.168.0.178 and the SolarEdge monitoring server. The 003d message is an encrypted data packet which it ignores until it can decrypt them. You have to wait a short time for the inverter to send a 0503 message which will enable semonitor.py to begin decrypting the 003d messages if you have specified the correct key with the -k option.

This appears to be functioning properly. Let it run for a while and see if you get some data in your output file.

[madether]

So i ran the monitor for the day and no decernable output.

[jbuehl]

First of all, make sure you have the latest code from the project. Also you must first extract the inverter key using RS232 into a file and reference it when you run semonitor.py with the -k option. Refer to the description of sekey.py in the README. I don't see where you are doing that in the example that you posted above.

Run semonitor.py with -vvv and watch for a 0503 message. You should see "creating decryption object with key" in the debug output, and after that when you see a 003d message it should be followed by "decrypting message" and then a 0500 message that produces data in the output.

[madether]

executing the following: root@raspberrypi:/home/pi/solaredge# tcpdump -i eth0 -U -w - | python seextract.py | python semonitor.py -k 07f16e4a6.key -d stdout -vvv tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes Jul 13 20:33:03 debugEnable: True Jul 13 20:33:03 debugFiles: True Jul 13 20:33:03 debugMsgs: True Jul 13 20:33:03 debugData: True Jul 13 20:33:03 debugRaw: False Jul 13 20:33:03 debugFileName: stdout Jul 13 20:33:03 haltOnException: False Jul 13 20:33:03 inFileName: stdin Jul 13 20:33:03 serialDevice: False Jul 13 20:33:03 networkDevice: False Jul 13 20:33:03 networkSvcs: False Jul 13 20:33:03 following: True Jul 13 20:33:03 passiveMode: True Jul 13 20:33:03 commandAction: False Jul 13 20:33:03 masterMode: False Jul 13 20:33:03 outFileName: stdout Jul 13 20:33:03 append: w Jul 13 20:33:03 keyFileName: 07f16e4a6.key Jul 13 20:33:03 key: 80bxxxxxxxxxxxxxxxe59 Jul 13 20:33:04 opening stdin

[madether]

Jul 13 20:34:01 Jul 13 20:34:01 --> message: 1 length: 18852 Jul 13 20:34:02 Jul 13 20:34:02 --> message: 2 length: 847 Jul 13 20:34:02 dataLen: 0000 Jul 13 20:34:02 dataLenInv: ffff Jul 13 20:34:02 sequence: 0421 Jul 13 20:34:02 source: 7f13e4a6 Jul 13 20:34:02 dest: fffffffe Jul 13 20:34:02 function: 0500 Jul 13 20:34:02 Discarding 825 bytes Jul 13 20:34:02 data: 90 be b5 aa ee b8 b8 62 14 3b ed 2b eb 32 74 da Jul 13 20:34:02 data: 0a 85 7a 4c e9 9b 31 75 5a 81 73 7e da d9 e4 c8 Jul 13 20:34:02 data: b9 18 b4 04 15 a6 31 fd b3 55 66 56 fb 09 db ea Jul 13 20:34:02 data: 78 e9 fb 43 d6 80 a1 be a7 c8 14 fc 6d 17 13 65 Jul 13 20:34:02 data: 0c 84 f2 9d b2 07 5a 20 a2 c6 a3 09 11 d4 e4 e5 Jul 13 20:34:02 data: 47 4f e1 36 12 e1 7c 59 08 17 75 cf fc 79 a9 06 Jul 13 20:34:02 data: aa 42 51 d3 62 52 8f 43 c8 60 d2 1d 2a 89 35 b3 Jul 13 20:34:02 data: c4 0a 42 bd 12 e8 24 4a a2 f8 d1 16 a0 9d 72 f4 Jul 13 20:34:02 data: 1a 41 61 11 26 5b 80 7b b6 26 0f 67 0a 82 d5 03 Jul 13 20:34:02 data: 00 00 00 00 00 00 bd 26 8a 7b 5d 2f 5c bd 32 7e Jul 13 20:34:02 data: ff c1 ba ee 2d 9b c2 d1 11 a3 8d 47 b5 75 c6 09 Jul 13 20:34:02 data: 58 2c 17 09 f6 da d7 42 3a 1f e5 9f c4 75 a0 5c Jul 13 20:34:02 data: 80 db 4b eb 49 27 ee dd 07 64 9c ce ef 0c e5 62 Jul 13 20:34:02 data: c3 ad 56 1c 65 a9 0a 59 68 7f 14 6e b9 89 30 15 Jul 13 20:34:02 data: fc d8 75 3b 5d 6b 31 d9 3f 72 4d c8 24 65 b3 49 Jul 13 20:34:02 data: 1f d9 26 c5 e0 78 06 ec 8a b5 fe b9 33 63 9d 49 Jul 13 20:34:02 data: 30 be 99 11 e8 61 31 41 ef 8b 57 e0 a6 cb a1 46 Jul 13 20:34:02 data: 29 b5 d8 db 54 b7 29 56 4c ab 1e 50 ad 13 26 c7 Jul 13 20:34:02 data: 7a 98 94 78 00 8c 8f 11 c5 06 3c a0 0d ac bc 8e Jul 13 20:34:02 data: 74 00 ab 3f c1 44 05 7d 40 49 f8 49 dd 2a 4e e3 Jul 13 20:34:02 data: e9 32 86 a8 d4 75 00 00 00 00 00 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 33 27 00 00 93 01 00 00 a0 dd Jul 13 20:34:02 data: 4b 9f 93 01 00 00 40 05 00 00 00 00 00 00 dd 2f Jul 13 20:34:02 data: f3 63 00 00 00 00 c0 eb 53 9f 93 01 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 d0 56 48 9f 93 01 00 00 7c 6a Jul 13 20:34:02 data: 73 63 00 00 00 00 e0 08 91 9e 93 01 00 00 b0 a2 Jul 13 20:34:02 data: 21 64 00 00 00 00 49 bd ff 91 74 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 94 b9 ff 91 74 00 00 00 b0 b9 Jul 13 20:34:02 data: ff 91 74 00 00 00 98 45 06 7b 37 62 65 65 38 61 Jul 13 20:34:02 data: 32 65 2d 63 36 30 66 2d 34 30 64 39 2d 38 64 32 Jul 13 20:34:02 data: 32 2d 32 31 65 37 32 38 31 38 62 64 37 62 7d 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 01 00 00 00 00 00 00 00 90 b9 Jul 13 20:34:02 data: ff 91 74 00 00 00 00 00 00 00 00 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Jul 13 20:34:02 data: 00 00 00 00 00 00 00 00 00 6a ef 1c 78 54 54 50 Jul 13 20:34:02 data: 2f 31 2e 31 0d 0a 48 4f 53 54 3a 20 32 33 39 2e Jul 13 20:34:02 data: 32 35 35 2e 32 35 35 2e 32 35 30 3a 31 39 30 30 Jul 13 20:34:02 data: 0d 0a 4d 41 4e 3a 20 22 73 73 64 70 3a 64 69 73 Jul 13 20:34:02 data: 63 6f 76 65 72 22 0d 0a 4d 58 3a 20 35 0d 0a 53 Jul 13 20:34:02 data: 54 3a 20 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75 Jul 13 20:34:02 data: 70 6e 70 2d 6f 72 67 3a 64 65 76 69 63 65 3a 61 Jul 13 20:34:02 data: 76 6d 2d 61 68 61 3a 31 0d 0a 0d 0a e1 4f 07 6c Jul 13 20:34:02 data: 00 7d 2c ef 4d 2d 53 45 41 52 43 48 20 2a 20 48 Jul 13 20:34:02 data: 54 54 50 2f 31 2e 31 0d 0a 48 4f 53 54 3a 20 5b Jul 13 20:34:02 data: 66 66 30 32 3a 3a 63 5d 3a 31 39 30 30 0d 0a 4d Jul 13 20:34:02 data: 41 4e 3a 20 22 73 73 64 70 3a 64 69 73 63 6f 76 Jul 13 20:34:02 data: 65 72 22 0d 0a 4d 58 3a 20 35 0d 0a 53 54 3a 20 Jul 13 20:34:02 data: 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75 70 6e 70 Jul 13 20:34:02 data: 2d 6f 72 67 3a 64 65 76 69 63 65 3a 61 76 6d 2d Jul 13 20:34:02 data: 61 68 61 3a 31 0d 0a 0d 0a Jul 13 20:34:02 Jul 13 20:34:02 --> message: 3 length: 22 Jul 13 20:34:02 dataLen: 0000 Jul 13 20:34:02 dataLenInv: ffff Jul 13 20:34:02 sequence: 0421 Jul 13 20:34:02 source: fffffffe Jul 13 20:34:02 dest: 7f13e4a6 Jul 13 20:34:02 function: 0080 Traceback (most recent call last): File "seextract.py", line 227, in readPcapRec(pcapFile) File "seextract.py", line 74, in readPcapRec sys.stdout.write(pcapFile.read(dataLen)) OverflowError: Python int too large to convert to C long 421 packets captured 421 packets received by filter 0 packets dropped by kernel

Nothing is logged after this point. (21:04)

[jbuehl]

There seems to be a lot of extraneous data that was captured. Try filtering out only the traffic going between the inverter and the SolarEdge server by IP address. You can do it either in tcpdump or seextract.py and by specifying either the address of prod.solaredge.com or your inverter.

tcpdump -i eth0 -U -w - host 217.68.144.150
tcpdump -i eth0 -U -w - host 192.168.0.178
python seextract.py -s 217.68.144.150 
python seextract.py -s 192.168.0.178

Be aware that either of these addresses can change in the future.

[madether]

Using tcpdump -i eth0 host 192.168.0.178 or host 217.68.144.150 -U -w - | python seextract.py | python semonitor.py -k 07f13e4a6.key -d stdout -vvv. Getting Jul 14 20:29:59 --> message: 2 length: 552 Jul 14 20:29:59 dataLen: 020c Jul 14 20:29:59 dataLenInv: fdf3 Jul 14 20:29:59 sequence: 0140 Jul 14 20:29:59 source: 7f13e4a6 Jul 14 20:29:59 dest: fffffffe Jul 14 20:29:59 function: 003d Jul 14 20:29:59 Discarding 6 bytes Jul 14 20:29:59 data: 00 00 00 00 00 00 Jul 14 20:29:59 Ignoring this message Jul 14 20:29:59 Jul 14 20:29:59 --> message: 3 length: 66 Jul 14 20:29:59 dataLen: 002c Jul 14 20:29:59 dataLenInv: ffd3 Jul 14 20:29:59 sequence: 039f Jul 14 20:29:59 source: fffffffd Jul 14 20:29:59 dest: ffffffff Jul 14 20:29:59 function: 003d Jul 14 20:29:59 Ignoring this message Jul 14 20:30:19 Jul 14 20:30:19 --> message: 4 length: 286 Jul 14 20:30:19 dataLen: 00f2 Jul 14 20:30:19 dataLenInv: ff0d Jul 14 20:30:19 sequence: 0141 Jul 14 20:30:19 source: 7f13e4a6 Jul 14 20:30:19 dest: fffffffe Jul 14 20:30:19 function: 003d Jul 14 20:30:19 Discarding 22 bytes Jul 14 20:30:19 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Jul 14 20:30:19 data: 00 00 fc bc e5 96 Jul 14 20:30:19 Ignoring this message Jul 14 20:30:21 Jul 14 20:30:21 --> message: 5 length: 270 Jul 14 20:30:21 dataLen: 00f2 Jul 14 20:30:21 dataLenInv: ff0d Jul 14 20:30:21 sequence: 0142 Jul 14 20:30:21 source: 7f13e4a6 Jul 14 20:30:21 dest: fffffffe Jul 14 20:30:21 function: 003d Jul 14 20:30:21 Discarding 6 bytes Jul 14 20:30:21 data: 00 00 00 00 00 00 Jul 14 20:30:21 Ignoring this message

[madether]

This is different Jul 14 20:37:00 Jul 14 20:37:00 --> message: 15 length: 22 Jul 14 20:37:00 dataLen: 002c Jul 14 20:37:00 dataLenInv: 0000 Jul 14 20:37:00 sequence: 0000 Jul 14 20:37:00 source: 00000000 Jul 14 20:37:00 dest: 00000000 Jul 14 20:37:00 function: bcfc Jul 14 20:37:00 Discarding -44 bytes Jul 14 20:37:00 Exception: unpack requires a string argument of length 2 Jul 14 20:37:00

[madether]

Jul 14 23:22:39 dataLenInv: 7956 Jul 14 23:22:39 sequence: 00c6 Jul 14 23:22:39 source: 0074ff39 Jul 14 23:22:39 dest: 7f13e4a6 Jul 14 23:22:39 function: fffe Jul 14 23:22:39 Discarding -13128 bytes Jul 14 23:22:39 Exception: unpack requires a string argument of length 2 Jul 14 23:22:39 Jul 14 23:22:39 --> message: 9 length: 66 Jul 14 23:22:39 data: 12 34 56 79 2c 00 d3 ff 6b 0c fd ff ff ff ff ff Jul 14 23:22:39 data: ff ff 3d 00 06 13 7e 09 83 a5 dc 4c c1 03 2e 90 Jul 14 23:22:39 data: e7 6e 72 60 dd 01 c7 25 b9 72 29 04 cf 83 38 93 Jul 14 23:22:39 data: 5e 77 70 56 65 73 6e 1d 3c 77 7d e2 06 6d dd 18 Jul 14 23:22:39 data: 7d 33 Jul 14 23:22:39 dataLen: 002c Jul 14 23:22:39 dataLenInv: ffd3 Jul 14 23:22:39 sequence: 0c6b Jul 14 23:22:39 source: fffffffd Jul 14 23:22:39 dest: ffffffff Jul 14 23:22:39 function: 003d Jul 14 23:22:39 decrypting message Jul 14 23:22:39 data: 12 34 56 79 00 00 ff ff 74 00 fe ff ff ff a6 e4 Jul 14 23:22:39 data: 13 7f 80 00 3e 43 Jul 14 23:22:39 dataLen: 3412 Jul 14 23:22:39 dataLenInv: 7956 Jul 14 23:22:39 sequence: 0000 Jul 14 23:22:39 source: 0074ffff Jul 14 23:22:39 dest: fffffffe Jul 14 23:22:39 function: e4a6 Jul 14 23:22:39 Discarding -13326 bytes Jul 14 23:22:39 Exception: unpack requires a string argument of length 2 Jul 14 23:22:39

[jbuehl]

Try this and post the debug output and also the pcap file

tcpdump -i eth0 -U -w - tcp and host 217.68.144.150 | tee capture.pcap | python seextract.py | python semonitor.py -k 07f13e4a6.key -d stdout -vvv

[madether]

There are heaps of 003d in the logs from both days. Even after a powercycle of the inverter I did not see a 0500 . Some from yesterday and some from today 21:45, 0745 AU EST capturex2.zip

[madether]

Ok, this capture has a 503 which I think is the key exchange and now it is decrypting data but nothing usable as far as I can see. Took it most of the day before I saw the 503.

Jul 16 05:22:52 --> message: 91 length: 24 Jul 16 05:22:52 dataLen: 002c Jul 16 05:22:52 dataLenInv: 0402 Jul 16 05:22:52 sequence: 1a02 Jul 16 05:22:52 source: 04020001 Jul 16 05:22:52 dest: 00006405 Jul 16 05:22:52 function: 0402 Jul 16 05:22:52 Discarding -42 bytes Jul 16 05:22:52 Exception: unpack requires a string argument of length 2 Jul 16 05:22:53 Jul 16 05:22:53 --> message: 92 length: 62 Jul 16 05:22:53 dataLen: 0022 Jul 16 05:22:53 dataLenInv: ffdd Jul 16 05:22:53 sequence: 03cb Jul 16 05:22:53 source: 7f13e4a6 Jul 16 05:22:53 dest: fffffffe Jul 16 05:22:53 function: 0503 Jul 16 05:22:53 Discarding 6 bytes Jul 16 05:22:53 data: 00 00 00 00 00 00 Jul 16 05:22:53 creating decryption object

capture4a.zip

[madether]

This is a more complete copy of captur4a in my previous post. This one there is an sudo>ssh>tcp from the inverter to SE and then a port change to 22221. Someone might be interested in this. capture4.zip

[jbuehl]

It looks like you may not have the latest version of the project. Line 130 of seMsg.py should be

(msgSeq, fromAddr, toAddr, function, data) = validateMsg(dataMsg[4:])

I made a change on 25 Mar that should have fixed the issue that I see in the log that you posted 2 days ago so if your copy of the source was before that date it would explain the problem.

I just made some additional changes to improve the message validation and error messages.

The port change to 22221 is something I have not seen before, but it should not affect passive monitoring.

[madether]

have you been able to extract any useful information from the pcaps I have uploaded yet? Is there anyway of forcing the inverter to resend a 0503?

[jbuehl]

See my last post. It looks like you may have an outdated version of the software.

I will not be able to decrypt the data unless I have your inverter key.

[jbuehl]

Yes, I am seeing decoded inverter data in capture4.pcap using the latest version of the project and your key.

Unfortunately, it looks like the optimizer data is not being decoded. The device type is 0x0022 which is one that nobody has encountered before.

[madether]

I don't have any optimizers on my system. I have to meters that are connected via rs485. ARe you able post some of the data please.

[madether]

trying this an it creates an empty .json file python seextract.py /mnt/nfs/capture4.pcap | python semonitor.py -k 07f13e4a6.key -o /mnt/nfs/capture4.json

[jbuehl]

Do you have the latest version of the software? Specifically seMsg.py?

You should see this capture4.json.txt

[madether]

I made the changes but not getting that at all. just a zero byte file. see my previous post.

[jbuehl]

That's exactly the command I am using as well. Here is the debug output capture4.debug.txt

[madether]

I wonder if there is a way of getting the data from the other meters?

[madether]

my system has 2 of these connected to the inverter http://www.all-energy.com.au/en/Exhibitors/918833/SolarEdge-Technologies-Australia-Pty-Ltd/Products/821546/SolarEdge-Modbus-Meter as well as one of these http://sol-distribution.com.au/solaredge-storedge-interface. They are all connected to the RS485 interface. would it be possible to access them via passive monitoring?

[jbuehl]

If the SolarEdge monitoring server displays the data from those devices then it would have to be somewhere in the stream you are monitoring. It's possible that is what the 0022 devices are. In order to decode the format you would have to correlate the data in the message to what you are seeing displayed. Since I don't have those devices I can't help other than to guess at a few things:

2 bytes: device type = 0x0022 4 bytes: inverter ID = 0x7f13e4a6 2 bytes: data length = 0x003a 4 bytes: timestamp 54 bytes: data

The data probably contains a device identifier towards the beginning. It appears to have some 4 byte floating point fields. You could try posting in issue #8 and see if someone who was in that discussion could help.

So were you able to see the data?

[madether]

I have been able see the data from capture 4, however since I don't have a capture key for the new session, I am not able to decrypt anything since.

I am not a programmer so I don't think i will be able to extract the 0022 stuff . :(

[jbuehl]

If you have semonitor.py running and then restart the inverter it should send the 0503 message when it wakes up and you would be able to start decrypting the messages right away.

[madether]

is there a way of forcing it without a powercycle,. I have to discharge the battery levels first, thats why .

[jbuehl]

You could try disconnecting it from the network for a while.

[madether]

disconnected the ethernet and boom. :)

Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 06 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 ff ff 7f ff 00 00 Jul 17 01:23:07 data: 00 00 Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 07 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 01:23:07 data: 7f ff Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 08 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 01:23:07 data: 7f ff Jul 17 01:23:07 Unknown device 0x0022 Jul 17 01:23:07 data: 22 00 a6 e4 13 7f 3a 00 b6 dd 8a 57 09 01 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 01:23:07 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 78 00 Jul 17 01:23:07 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff

[jbuehl]

Excellent!

I just had a thought that since your meter devices are made by a different company, the format of the data may be documented somewhere, possibly by the manufacturer http://www.ccontrolsys.com/w/Home.

[madether]

Found this http://www.opensystemsproducts.com/MM/CCS/Manual_WNC_Modbus.pdf

[madether]

is there a problem with seextract not working with -f python ./seextract.py -f -o /mnt/nfs/capture8.dat /mnt/nfs/capture8.pcap The dat file does not grow.

[madether]

I have a 0300 device too. Jul 17 02:25:00 Unknown device 0x0030 Jul 17 02:25:00 data: 30 00 a6 e4 13 7f 56 00 7a ec 8a 57 54 31 36 43 Jul 17 02:25:00 data: 30 30 30 35 33 39 31 00 cd 8c d1 43 7a d8 d8 40 Jul 17 02:25:00 data: 00 00 c8 45 00 18 cd 45 00 10 09 45 95 20 01 00 Jul 17 02:25:00 data: 00 00 00 00 22 15 01 00 00 00 00 00 ff ff 7f ff Jul 17 02:25:00 data: ff ff 7f ff 00 00 60 41 03 00 00 00 00 00 00 00 Jul 17 02:25:00 data: 00 00 2c 01 00 00 56 00 00 00 01 00 00 00 Jul 17 02:25:00 Unknown device 0x0022 Jul 17 02:25:00 data: 22 00 a6 e4 13 7f 3a 00 7a ec 8a 57 03 01 00 00 Jul 17 02:25:00 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 02:25:00 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01 Jul 17 02:25:00 data: 00 00 00 00 00 00 05 00 00 00 ff ff 7f ff 00 00 Jul 17 02:25:00 data: 00 00 Jul 17 02:25:00 <-- message: 25 length: 364

[madether]

Jul 17 02:55:01 Unknown device 0x0022 Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 05 00 70 f8 Jul 17 02:55:01 data: 03 00 00 00 00 00 e5 78 09 00 00 00 00 00 e7 6e Jul 17 02:55:01 data: fe ff ff ff ff ff 6c 5b 0e 00 00 00 00 00 2c 01 Jul 17 02:55:01 data: 00 00 00 00 00 00 02 00 00 00 1d 23 fe 40 00 00 Jul 17 02:55:01 data: 00 00 Jul 17 02:55:01 Unknown device 0x0022 Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 06 01 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01 Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 ff ff 7f ff 00 00 Jul 17 02:55:01 data: 00 00 Jul 17 02:55:01 Unknown device 0x0022 Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 07 01 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01 Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 02:55:01 data: 7f ff Jul 17 02:55:01 Unknown device 0x0022 Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 08 01 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01 Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 02:55:01 data: 7f ff Jul 17 02:55:01 Unknown device 0x0022 Jul 17 02:55:01 data: 22 00 a6 e4 13 7f 3a 00 82 f3 8a 57 09 01 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 00 Jul 17 02:55:01 data: 00 00 00 00 00 80 00 00 00 00 00 00 00 80 2c 01 Jul 17 02:55:01 data: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff Jul 17 02:55:01 data: 7f ff Jul 17 02:55:01 <-- message: 45 length: 49

[jbuehl]

When writing to a file, seextract.py does not continually flush the buffer as it does when writing to stdout. It should grow the output file if you do it like this

python ./seextract.py -f /mnt/nfs/capture8.pcap > /mnt/nfs/capture8.dat