Koenkk / zigbee2mqtt

Zigbee 🐝 to MQTT bridge 🌉, get rid of your proprietary Zigbee bridges 🔨
https://www.zigbee2mqtt.io
GNU General Public License v3.0
11.12k stars 1.59k forks source link

pairing APsystem YC600 #4221

Closed petsch9 closed 1 year ago

petsch9 commented 3 years ago

Hi,

Did someone manage to connect/pair with the APsystem YC600 micro omvormer. They should connect with zigbee but it is not pairing at all.

Any sugestions are welcom.

Kind regards, Peter

kadzsol commented 3 years ago

Looking for the same information... There is no reset button, powering off/on does not help either. I suspect we have to send a proprietary signal to trigger pairing. Maybe somene with an official ECU-R gateway can do a test and sniff?

Poquaruse commented 3 years ago

Hi all!

Here is what I've found out (with my limited knowledge of Zigbee).

Next steps I'll try:

kadzsol commented 3 years ago

Nice stuff! Now we are one little step further and seem to understand why some people cannot see any packets. I have read on internet that the YC600 has a zigbee communication range of 10-20m. Also, AP systems has confirmed to someone posting on a forum, that they use a proprietary zigbee protocol. This also seems to be in line with your finding. We have to crack this stuff! I am going to order my own 2531 now :-)

petsch9 commented 3 years ago

Great i cant wait

Poquaruse commented 3 years ago

Unfortunately, I had just a few minutes of sunlight left today... But here is what I did. In the config, I set the following parameters:

advanced: pan_id: 0xae14 # sniffed from the link status packets channel: 16 log_level: debug

The I started zigbee2mqtt and sniffed the traffic using another 2531.

44 185.546362 0x0000 Broadcast ZigBee ZDP 108 Permit Join Request

45 198.038446 Broadcast IEEE 802.15.4 70 Beacon Request

46 198.040590 0x26c7 ZigBee 88 Beacon, Src: 0x26c7, EPID: 03:00:12:11:10:90:ff:ff

And that's about it... Then, the "link status" packets continued. No device joined. :-( What puzzles me a bit is that 0x26c7 responds with a Beacon?! I thought that the device trying to join sends a beacon request and then the coordinator answers with a beacon?!

Any ideas where to go from here?

Tomorrow, I'll try and enable zigbee-herdsman debugging, too.

Poquaruse commented 3 years ago

Apparently, the ECU-R (APSystem device to connect to the inverters) needs the inverters' UID. So perhaps there is some kind of authentication going on there?! See https://youtu.be/3UGMGZRTJQI?t=868

kadzsol commented 3 years ago

My zigbee knowledge is very limited, so I was reading some documentation to jump start my knowledge :-) What I understand is that it is always a zigbee device which is sending beacon requests and coordinator is responding with a beacon frame. If coordinator enables permitjoin and is shown in beacon frame, the device can decide to send association request to coordinator to join the network. So it is always the device who decides which network to join from the available/permitted networks. It seems, that the YC600 decides not to join your network, despite the fact that you coordinator permits it. Feeding in the ID of the inverter in the ECU APP (as shown in the video you have linked) is probably the key, I guess the ID somehow needs to be included in the beacon response frame of the coordinator. When de device recognizes the own ID in the frame, it will join.

Poquaruse commented 3 years ago

With herdsman logging enabled: still no luck. Nothing to see...

What I understand is that it is always a zigbee device which is sending beacon requests and coordinator is responding with a beacon frame.

Yes, that is my understanding, too. However, here we see that the inverter (e.g. NOT the coordinator) seems to respond with a beacon! From the sniffing logs you cannot see which device sent the beacon request, but you can see that the inverter answers immediately.

I too assume that you need to tell the inverter a "secret" (probably related to the UID). I don't have any idea how to accomplish this... A sniff from a successful pairing with a physical ECU-R would be very helpful, I guess...

kadzsol commented 3 years ago

Yes, we have to find someone owning an ECU-R and able to make a sniff. Other option is we do some crowd-funding to buy an ECU-R for testing purposes. It costs around the 200 euro, I would not mind to donate some money if we can crack this thing.

kadzsol commented 3 years ago

There is a German forum out there with people that are also interested in having home brewed solution for the YC600 so I have posted a message there to join our forces: https://knx-user-forum.de/forum/%C3%B6ffentlicher-bereich/knx-eib-forum/diy-do-it-yourself/1409938-pv-micro-inverter-diy-auswertung-m%C3%B6glich

manoficons commented 3 years ago

'franck102' in this discussion https://community.openenergymonitor.org/t/zigbee-inputs/11166/10 seems to have an ECU device. Perhaps he is willing to help with some sniffing.

kadzsol commented 3 years ago

Thanks for the thread. He seems to have an ECU-C and not an ECU-R. Nevertheless, I have registered there and i am going to post a question about sniffing. An ECU-C sniff might also give some hints for us how to proceed.

Poquaruse commented 3 years ago

That would be great! I guess there isn't much of a difference between the ECU-C and -R when it comes to the zigbee connection.

manoficons commented 3 years ago

ECU-C seems to be the "bigger" device. With swiching output options and more. https://global.apsystems.com/wp-content/uploads/2018/04/4271801031_APsystems-Energy-Communication-Unit-ECU-C-User-manual_Rev1.5_2018-1-16.pdf

Zigbee section should work in the same way as in ECU-R.

Poquaruse commented 3 years ago

In case somebody who owns an ECU-R (or ECU-C) stumbles across this thread and is willing to help, I'll try to summarize the steps to obtain a successful sniff of the connection between the YC600 and ECU-R...

To anybody who is willing to help: Thanks a lot in advance!

samr037 commented 3 years ago

I'd like to order it from some trustworthy website like Amazon, opening it without leaving any marks, sniff the trafic, and then ship it back ;). Amazon doesn't have it, only PV retailers, but there's shipping costs and they're not very clear about return policies...

manoficons commented 3 years ago

Some statement from the manufacturer (not very useful) https://community.smartthings.com/t/apsystems-yc600-microinverter-monitored-by-the-samsung-hub/166100

manoficons commented 3 years ago

One more guy having the ECU-R and wants to use zigbee connection https://domoticz.com/forum/viewtopic.php?f=28&t=22368&start=140

kadzsol commented 3 years ago

One more guy having the ECU-R and wants to use zigbee connection https://domoticz.com/forum/viewtopic.php?f=28&t=22368&start=140

Thanks for the info. Registered there and posted a message :-)

kadzsol commented 3 years ago

I'd like to order it from some trustworthy website like Amazon, opening it without leaving any marks, sniff the trafic, and then ship it back ;). Amazon doesn't have it, only PV retailers, but there's shipping costs and they're not very clear about return policies...

Creative :-) Let us hope someone out there with an ECU will voluntair to make a sniff. If no, I do nnot mind to help with the shipping costs.

iboot700 commented 3 years ago

One more guy having the ECU-R and wants to use zigbee connection https://domoticz.com/forum/viewtopic.php?f=28&t=22368&start=140

Thanks for the info. Registered there and posted a message :-)

And there I am...

Hope I can help out, I have a CC2531 available and the ECU-R paired with a YC600 and QS1. I’ll try the procedure above to sniff the traffic and post the results here. Might take a few days.

kadzsol commented 3 years ago

Great, thx!

Poquaruse commented 3 years ago

I’ll try the procedure above to sniff the traffic and post the results here. Might take a few days.

Great, thank you very much! Let us know if any problems arise during logging where we might be able to help!

boons605 commented 3 years ago

One of my friends has a bunch of YC600's and en ECU-R. I've only got 4 YC600's, but I do have a CC2531 stick with sniffer firmware. We're planning to sniff next weekend (2020-10-03, CEST time zone, looking at how much of a morning person I am, I'd say in the afternoon).

kadzsol commented 3 years ago

Great, I cannot wait!

petsch9 commented 3 years ago

Great, I can not wait either.

iboot700 commented 3 years ago

In case somebody who owns an ECU-R (or ECU-C) stumbles across this thread and is willing to help, I'll try to summarize the steps to obtain a successful sniff of the connection between the YC600 and ECU-R...

* Set up a CC2531 for sniffing. See https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html for details. In case the CC2531 has the TI sniffing firmware (instead of the ZBOSS one): TI offers a software to stream the sniffed packets to WireShark.

* Use the APSystems app and remove one inverter from the ECU-C. This should disconnect the inverter from the ZigBee network so that as a next step we can observe the joining process.

* Start sniffing on channel 16. (At least for me, channel 16 seems to be the only channel that the YC600 is sending Link Status packets on...)

* Re-add the inverter to the ECU-C using its UID.

* Check that the inverter has successfully been added in the app.

* Wait until values show up in the app. (So now we know for sure that the inverter is back in the ZigBee network.)

* Stop the sniffing process and export/save the PCAP file.

* Upload the PCAP file here for dissecting. :-) (Please be aware that the ZigBee key is -- hopefully -- included in the PCAP file...) It would also be very helpful to know the entered UID so that we can check how it is used to authenticate.

To anybody who is willing to help: Thanks a lot in advance!

I gave it a try tonight but I'm not sure I've succeeded. I've not been able to find the Zigbee key. Removing the inverter is easy but to add it again I had to power cycle the ECU and inverters to get the process to 100% but still the automatic system check in the app fails. As it is dark right now I'm not able to check if the inverter is back in the network, I'll try tomorrow if it is light.

Anyhow, I've attached the file so you can have a look. If there is anything interesting in there it is probably around line 327, here the connection process was finished according to the app.

YC600_zigbee.zip

kadzsol commented 3 years ago

Thanks a lot! Let us see what we can learn from this sniff.

krikk commented 3 years ago

please be aware that the yc600 is powered from dc, so it is only working when you have enough sun... (no sniffing at night )

kadzsol commented 3 years ago

I went through the sniff but could not find anything from the YC600 (similar like "46 198.040590 0x26c7 ZigBee 88 Beacon, Src: 0x26c7, EPID: 03:00:12:11:10:90:ff:ff"). Maybe it was not powered any more (no sun)?

Poquaruse commented 3 years ago

Thank you very much for the sniff! As the others said: there needs to be (enough) sun for the inverteres to be powered.

However, from the sniff we still can see a few things!

My first guess would be that the ECU-R is trying to make the inverters send data (which are now connected because of no sun). Would be great to see what happens when all inverters are powered!

Thanks again!

iboot700 commented 3 years ago

Thanks for your comments. I gave it another try during daytime and at least the pairing procedure was a lot faster and less complicated. The sniffing is started before adding the YC600 to the ECU-R together with 2 QS1 inverters which I did not remove. I stopped as soon as all inverters showed data. I also included the UIDS and short addresses. Hope it helps.

YC600_zigbee_2.zip

kadzsol commented 3 years ago

Many many thanks! I see interesting information in this trace, like UID's! I hope we can understand what is going on and can create our own fake ECU :-)

Poquaruse commented 3 years ago

Thank you very much! I had a quick look at the sniff and it's quite interesting. For better reading I filtered the route packets and the link status packets: ((!(zbee_nwk.cmd.id == 0x08)) && !(zbee_nwk.cmd.id == 0x01)) && !(zbee_nwk.cmd.id == 0x02).

This could be the "non-standard" pairing process. E.g. not really pairing in a network, but authenticating the ECU using some kind of challenge/response-scheme which incorporates the UIDs (and probably some magic).

I guess there is some good news: there doesn't seem to be a fancy type of encryption going on. However, unless I'm mistaken, there is a non-standard authentication using "unknown commands" to make the inverters send data to the ECU using "read attributes" packets. Now we need to figure out how these work and try to understand it. And then we need to emulate it -- a thing I'm not even sure zigbee2mqtt is able to do...

kadzsol commented 3 years ago

Nice analysis! It will not be an easy task to understand this protocol. Coming weekend I will separate the trace of one device and make a flow-chart out of it. Maybe someone here can comment/help how to create/emulate the packets sent by the ECU using zigbee2mqtt?

kadzsol commented 3 years ago

I have also checked the previous sniff when there was no communication with the inverters. The mailformed packages are also present in that sniff. Would it mean a reception problem?

iboot700 commented 3 years ago

I followed this guide (https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html) and stopped after I added the Trust center link key in Wireshark. Is this correct or might this cause issues?

The reception should not be an issue I would say, the panels are on the garage roof and the measurement is done inside the garage directly below the panels en 2m from the ECU-R.

iboot700 commented 3 years ago

Sniffed some more... This time 3 minutes with the ECU-R switched off, after this I switched on the ECU-R (paired with the converters). Maybe it helps to understand the process.

ECU-R_switch_on_paired.zip

iboot700 commented 3 years ago

And even more. This time the readings should be as noise free as they can be.

1 - Measured normal operation for about 10 minutes where you can see the status update of the inverters every 5 minutes (for 3 inverters) 2 - I've requested the grid profile for the connected inverters (around 17 parameters per inverter). I suspect these will explain a lot of the attributes you see. 3 - Removed all inverters from ECU-R 4 - Paired only YC600 5 - Updated the grip profile for the YC600 (takes a long time!) 6 - Removed YC600 and only paired QS1

Hope it helps, if you have any questions just let me know

Zigbee_snif.zip

boons605 commented 3 years ago

I'm currently sniffing with an ECU-R. We've got 5 inverters online here, the ECU-R had been offline for a while before starting the first sniff.

Afterwards, we removed all the inverters from the ECU-R and paired only the YC600 with UID 406000008105

The ECU-R is not connected to the internet. At the time of pairing, the app showed 18W and 19W for the connected panels.

WhatsApp Image 2020-10-03 at 15 27 57

ECUR_Pairing_And_Turnon.zip

kadzsol commented 3 years ago

Lot of stuff to analyse. Thanks guys!

kadzsol commented 3 years ago

Sniffed some more... This time 3 minutes with the ECU-R switched off, after this I switched on the ECU-R (paired with the converters). Maybe it helps to understand the process.

ECU-R_switch_on_paired.zip I have looked into this sniff and I seem to understand (sort of :-):

Poquaruse commented 3 years ago

Thank you guys so much for the sniffs! 👍 That's great! As soon as possible, I'll try to have a thorough look at them.

kadzsol commented 3 years ago

One more observation:

The packets with "Unknown Command: 0x00 & Unknown Command: 0xa5" are only present in sniffs when pairing is going on.

In the sniff I can also see that this is a Manufacturer specific "something" (ID 0xa5a5 = AP systems?)

kadzsol commented 3 years ago

Google-ing on "0xa5a5 & zigbee" gave me the attached reference from Jennic (now NXP):

zigbee device profile reference manual.pdf

Page 29 shows a code snippet with profile ID 0xa5a5. Could it be that the ECU/YC600 is based on the API/microcontroller of Jennic/NXP?

manoficons commented 3 years ago

Could it be that the ECU/YC600 is based on the API/microcontroller of Jennic/NXP?

If the YC600 was easier to open, without breaking it, I would already had a deep look inside. But everything under the black cover seems to be "glued" with a white silicone mixture.

kadzsol commented 3 years ago

Some instructions how to send/receive zigbee commands using a PC: https://e2e.ti.com/support/wireless-connectivity/zigbee-and-thread/f/158/t/434159?CC2531-Send-and-Receive. Could not find something similar after scanning the docs of zigbee2mqtt. Unfortunately, my CC2531 is still on its way from China, so cannot try for myself

kadzsol commented 3 years ago

My CC2531 arrived today! I will start experimenting this weekend.

kadzsol commented 3 years ago

I have managed to get my sniffer up and running using zboss framework. I can see the link state packages of my YC600 on channel 16. So far so good.

I have spent the rest of my free time to find out how to transmit custom zigbee frames back into the network to see if I can trigger the YC600 to send more information.

This does not seem an easy task with the CC2531.

  1. https://github.com/Tropicao/zigbridge/blob/master/doc/firmware_instructions.md This source says that in order to communicate with other devices on the network, one has to build its on framework based on TI' Z-Stack 3.0. It includes a GenericApplication which can be reused and compiled with IAR Embedded Toolchain for 8051. This tool is not free, altough they have a 30-days trial available. Programing language is C++.

  2. I was also searching for some low-level tools which would enable us to inject frames into the network. Sort of hacktool one can use to experiment with feeding back some (modified) commands from our traces. So I found https://github.com/riverloopsec/killerbee. It does support the CC2531, but only for sniffing... Other (fully) supported devices do not seem very cheap but I will dig into this topic further to see if there is a reasonably priced alternative.

Comments/ideas/feedback is welcome!

Poquaruse commented 3 years ago

Hi all!

I finally had the chance to take a closer look at the provided sniffs. Thank you very much for providing them! Here is a short summary of what we can learn from them.

To be honest: I haven't found an (easy) way to send custom commands. Perhaps we could take a look at zigbee-herdsman's core to find out how to do that. But that is probably way (!) beyond my skill-level...