baudneo commented 9 months ago

Hi, I have 3 bt bulbs and 1 wifi/bt smart plug. I've been using cync2mqtt which connects via bluetooth but, I can't get connected to the smart plug.

My bulbs are controlled by a dumb switch so they loose mains power and drop off the network. This means I keep loosing control of the bulbs when they go offline and is quite annoying.

The smart plug is always mains powered which makes it the best candidate to be the proxy to the bt bulbs.

My question is, can your implementation control the bt mesh lights via the plug? I saw on the other issue that it is possible to send commands to a "hub" as long as you send it with the correct device id?

And 2 final questions:

after getting the device info from cync cloud, can all devices be local only?
is there anyway to pair devices local only? Like completely bypass cync. Buy device, pair locally and control locally without ever registering with cync?

Thank you for your work!

baudneo commented 8 months ago

I have a python conversion of cync lan I am working on. I have the server working. I am just working on decoding packets and control commands.

I've got a pretty good handle on received packets. Haven't explored controlling yet. The first 4-5 bytes usually defines what the packet will hold. In packets that start with 0x7e or 0x83. The data is wrapped on 0x7e but, there can be multiple data structures in 1 packet. So, I am working on packet processing right now.

It's kind of tough because when the cync phone app connects, it's totally different and adds logic to reading packets (there are queue ids and msg ids).

Anyways, if anyone wants to help, please see my branch.

https://github.com/baudneo/cync-lan

Python branch.

iburistu commented 8 months ago

Hey! Apologies for the delay on this. Let me answer your questions from your first comment:

BT mesh could be controlled by the plug, as long as the plug is labelled as a "direct connect" plug. I don't have any code right now to actually perform this, but during my initial testing when I was discovering the protocol I would accidentally set the device ID bit when sending TCP to a specific device. This would cause the TCP targeted device to send the command to the correct device ID device. The other issue has additional details from Matthew's findings. You'll also need to figure out the device IDs of your registered devices - use this.
Once you provision the devices in the app the control can be exclusively local. I have to update docs - some of the new firmware for the direct connect bulbs is smart enough to not accept DNS records that exist in private IP space (10.x). I've found a workaround with iptables SNAT + DNAT rules on a router to hack this, but it's pretty janky.
It may be possible to provision devices locally only, but probably not. I think most of the provisioning is done via BT first, and then just synced up to the cloud. If you have a BT sniffer you may be able to figure out the provisioning workflow, but for now the best you can do is temporarily proxy the provisioning to Cync via socat and then just cut the proxy out when it's provisioned. I'd be curious if any details during the provisioning process between the device and Cync cloud could be sus'ed out, but I haven't tried it.

On your Python branch - nice work! One big tip - socat is your best friend. You'll need to run the command on your mobile device that you want to emulate multiple times and then diff the outputs to see what's actually required. When you're at the point where you kinda understand the protocol, try zeroing out as many bytes as possible in messages to identify what's actually needed for the communication to work at a bare minimum - you'll get overwhelmed if you try to just send similar-ish commands and things don't work. Turn off BT too to force it to come from the cloud.

Good luck!

baudneo commented 8 months ago

Hi, thanks for the insights. Funny enough, before I started down CyncLAN port to python, I did a DEEP dive on BT only using cync2mqtt and rewritting a lot of it. There are several issues with BT only command and control due to Linux BT stack and lack of options for userland to kernel BT control.

I have been working on device status for now and leaving control but I have noted the other issues command packets to try when I do move onto control. Since I started with cync2mqtt I have ussed nikshriv's cync_data.json puller script and know device ID's. I was trying to figure a way to not query the cloud at all for id's and ascertain them from the HTTP calls. Some sort of state machine that keeps track of what device ID's it sees (and hopefully there is a 'device type' byte in device status packets, I see in older firmware structs there were but I havent noticed anything in newer firmware structs)
I have 4x DC bulbs with latest firmware and the DNS redirect is working on them so far. I do run OPNsense and can DNAT/SNAT if needed. I can write up docs after I perform it.
I have experience with cync BT and am gaining experience in their HTTP stuff. It would probably take a lot more research to get a device to onboard without the cloud. It would be sweet to be a drop in replacement for cync cloud server like vaultwarden is for bitwarden.

I think I am deep diving the HTTP stuff too hard. I am not just focusing on getting status and control, I am trying to figure out what each packet is for and how it relates. Maybe I should just focus on accomplishing the task rather than playing nice and trying to be an exact replica of cloud. When the phone app connects to the cloud it adds a whole level of complexity. 0x73 (with 0x78 replies and 0x7b replies) shows up when the phone app connects and I dont have my head wrapped around what each reply does.

Ive got status updates figured out and I seem to find patterns in other calls.

Examples

The header seems to be 1-4, a "header ID" is 5, "queue ID" is 6-9 and message ID is 10-13.

# FIRST PACKET - Device info, sends a starting queue id and an ascii ID for cloud?
> 2024/02/24 12:38:59.000212267  length=31 from=0 to=30
 23 00 00 00 1a 03 39 87 c8 57 00 10 31 65 30 37 64 32 63 33 36 34 35 38 32 63 35 39 00 00 3c
 23 00 00 00 1a 03 [39 87 c8 57] 00 10 [31 65 30 37 64 32 63 33 36 34 35 38 32 63 35 39] 00 00 3c
[39 87 c8 57] seems to be opening queue_id
[31 65 30 37 64 32 63 33 36 34 35 38 32 63 35 39] is some sort of cync cloud related id in ascii

# First device status, status for itself and possibly other devices connected to it?
> 2024/02/24 12:39:02.000324757  length=128 from=31 to=158
 83 00 00 00 25 39 87 c8 57 00 01 00 7e 00 00 00 00 fa db 13 00 33 22 11 07 00 07 00 db 11 02 01 01 64 3e ff ff ff 00 00 f1 7e 43 00 00 00 1a 39 87 c8 57 01 01 06 05 00 10 07 01 64 3e ff ff ff 01 00 07 00 00 00 00 00 00 83 00 00 00 32 39 87 c8 57 00 02 00 00 00 00 00 00 fa 00 20 00 00 00 00 00 00 00 00 ea 00 00 00 86 01 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c1 7e
 There are 2 packets. 7e is a boundary packet but is used differently in different commands. i.e. some combined packets end in 7e and some dont.
 [83 00 00 00 25] [39 87 c8 57] [00 01 00] [7e] {00 00 00 00 fa db 13 00 33 22 11 07 00 07 00 db 11 02 01 01 64 3e ff ff ff 00 00 f1} [7e] 
 [83 00 00 00]  header
 [25] header id
 [39 87 c8 57] queue_id (sent in device info)
 [00 01 00] packet number or msg id?
 [7e] boundary
 # There is a device status struct in here, whats the meaning of other data?
 {00 00 00 00 fa    db  13 00  33  22  11 07 00 07 00   db  11 02 01 01   64  3e   ff   ff   ff 00 00   f1}     data: hex
 [0, 0, 0, 0, 250, 219, 19, 0, 51, 34, 17, 7,    0,  7, 0,    219, 17, 2, 1, [1, 100, 62, 255, 255, 255], 0, 0, 241] data: int
                             *     ?   ?    ?    ?    ?   ?    ?   ?  ?       *                    s,   b,    t,    r,      g,      b                     legend
 # I see maybe further structs. 0xdb wrapped? *
 [7e] boundary
 # next packet appeneded to end is a device status
  [43 00 00 00 1a] [39 87 c8 57] [01 01 06] {05 00 10 07 01 64 3e ff ff ff 01 00 07 00 00 00 00 00 00 83 00 00 00 32 39 87 c8 57 00 02 00 00 00 00 00 00 fa 00 20 00 00 00 00 00 00 00 00 ea 00 00 00 86 01 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c1} [7e]
    [43 00 00 00] header
     [1a] header id (sent in device info?)
    [39 87 c8 57] queue_id (from device info)
    [01 01 06] packet number or msg id?
    {05 00 10 07 01 64 3e ff ff ff 01 00 07 00 00 00 00 00 00 83 00 00 00 32 39 87 c8 57 00 02 00 00 00 00 00 00 fa 00 20 00 00 00 00 00 00 00 00 ea 00 00 00 86 01 00 30}
byte 4-10 is the status struct, ID: 7, STATE: 1, BRI: 100 TEMP: 62 R: 255 G: 255 B: 255
byte 13 seems to be the device ID of the device that reported that status packet.
No idea on the rest of the data.
    [7e] closing boundary

0x83 and 0x73 packets have data wrapped in 0x7e boundarys. IDK what 0x83 is but it seems 0x73 are control packets. 0x43 are device status and have different structs. 1 struct seems to be for its own status and the other struct is device status for another device that also has data on which device it is proxying the status report through.

My packet/data struct notes are a mess but I am slowly figuring it out after combing socat logs and breaking down/commenting each packet with what I think it does. I started noticing patterns and stuff while breaking down socat logs. I just need to get the notes into a digestible state before I push them to repo.

Sorry for the wall of text, I'm knee deep in it and have been for a week.

baudneo commented 8 months ago

Got it figured. The newer firmware calculates checksums differently.

I'd say I should have everything buttoned up by weekend. I've got async mqtt and Hass mqtt auto discovery figured out. I am just working on a few data structures. In 0x73 packets you can ask the device for it's BT mesh info.

I've got control and status worked out. The only issue is if the device isn't freshly powered on, it won't send its own status until you ask it or a command comes through.

So just a few smaller things left to fix but the PoC is working. I have local control of cync devices via Hass and there is no lag.

Thanks for your work @iburistu . I've been trying to control cync devices via Hass for awhile now and we're almost there.

iburistu commented 8 months ago

Glad to hear you're making strong progress! I'll be keeping an eye on your repo to see what I can learn regarding device IDs and meshing. I feel there's a lot more headroom here for some really cool stuff, just with the info you discovered, and I hope the community can really take this and run with it.

I didn't even consider MQTT as control method for these devices. I use MQTT quite extensively professionally but never really thought to use it here because I have only ever had less than 5 Cync devices and just wanted things to turn on and off lol. Good work!

baudneo commented 7 months ago

I think it's ready for beta testing. I've had it running for 3 days straight and it is still responsive through Hass.

I've only tested with newer firmware devices so, there may be some issues on older firmware but hopefully not. I just got one of the cync battery powered dimmers and one of their dynamic 2 color lights. If I can find a switch with the motion sensor or ambient light sensor I can work on adding those.

I haven't tackled rooms/groups or light shows yet but, it works for the basic stuff which is a heck of a lot better than what I had before.

Thanks for your work! I wouldn't of gone down this path if it weren't for your repo here.

iburistu / cync-lan

Would this work to control bt mesh lights via wifi/bt smart plug? #2

Examples