FuzzyMistborn / python-eufy-security

Python library for Eufy Security cameras
MIT License
203 stars 43 forks source link

Finish API documentation #3

Open FuzzyMistborn opened 5 years ago

FuzzyMistborn commented 5 years ago

We've got the basics regarding camera stream/images. Most important to me is the ability to control certain aspects of the cameras, like the schedule (ie disable motion sensor), and maybe even turn the chime on/off.

I've tried doing a MITM to try to uncover more but can't get it to work. Other ideas welcome.

bachya commented 5 years ago

I've tried a MITM proxy for these actions, too, with no luck.

I did see something interesting while setting the mode: while on the same network as the Eufy hub, I couldn't see any obvious API calls, but the mode was successfully set; while on a cellular network (and, again, with the proxy in the middle), the mode changes never succeeded. No clue what that means.

Any Android users out there who'd be willing to decompile the app and see what is going on?

FuzzyMistborn commented 5 years ago

The APK is available online on APK Mirror. https://www.apkmirror.com/apk/anker/eufy-security/

bachya commented 5 years ago

I was able to decompile. On the hunt.

EDIT: Well, after several hours, I'm reminded how difficult Java is. 😆I don't see any particular API that gets called when the mode (Away/Home/etc.) buttons are tapped. Interestingly enough, I also see no reference to /api/v1/web/equipment/start_stream or /api/v1/web/equipment/stop_stream; this, combined with the fact that the mobile app's API calls start with https://security-app.eufylife.com/v1 and the web app's start with `https://mysecurity.eufylife.com/api/v1, makes me think we're dealing with different things.

Also worth noting that the web app doesn't appear to support modes currently.

joepadmiraal commented 5 years ago

I had a quick look at the APK and it seems to have some MQTT classes. Maybe it's using MQTT instead of HTTP for the mode buttons.

FuzzyMistborn commented 5 years ago

That would be fascinating if true. Would also likely be true for some of the settings/configuration then too. Still would think the calls would show up in HTTP(S) traffic.

bachya commented 5 years ago

Fascinating!

Most proxies aren't configured to handle raw TCP over TLS – they only look at HTTP traffic. Perhaps using a SOCKS proxy would be better, since that would redirect all traffic. Unfortunately, the Charles iOS app doesn't handle this yet.

FuzzyMistborn commented 5 years ago

If you know of an android option I'm happy to give it a go.

bachya commented 5 years ago

Maybe try this? https://play.google.com/store/apps/details?id=org.sandroproxy.drony&hl=en_US

FuzzyMistborn commented 5 years ago

Will give it a go tonight if I have time.

FuzzyMistborn commented 5 years ago

Still digging but looks like @joepadmiraal is on to something. Seeing traffic going to "security-mqtt.eufylife.com:8789". Trying to get details.

Drony doesn't give out more details and I can't find a better working option that I can set up. Hit my limit for the evening, hope someone else can take the url at do some more sniffing.

joepadmiraal commented 5 years ago

Maybe Mallet or Mallory can do the trick? I would love to do some tests myself but the doorbell is not available in the Netherlands yet.

mjhram commented 5 years ago

Hi I am trying to use the API's. The login is successful, but the get device list returned no data! { "code": 0, "msg": "Succeed." } what am I doing wrong?

gingermike commented 4 years ago

I was able to decompile. On the hunt.

EDIT: Well, after several hours, I'm reminded how difficult Java is. 😆I don't see any particular API that gets called when the mode (Away/Home/etc.) buttons are tapped. Interestingly enough, I also see no reference to /api/v1/web/equipment/start_stream or /api/v1/web/equipment/stop_stream; this, combined with the fact that the mobile app's API calls start with https://security-app.eufylife.com/v1 and the web app's start with `[https://mysecurity.eufylife.com/api/v1](https://mysecurity.eufylife.com/api/v1%60), makes me think we're dealing with different things.

Also worth noting that the web app doesn't appear to support modes currently.

I had a quick look at the decompiled APK. I think that the mode changes are handled by the com.oceanwing.battery.cam.guard.logic.ModeManager class.

The setMode method calls through to the ZMediaCom class, which appears to use some classes labelled as using P2P connectivity. This connection is then handled through a native interface - so no idea how it's initiated or where it's going to! I need to fiddle with the REST API - I think some clues might exist there...?

Also, in regards to MQTT it looks like Anker has hardcoded the passwords to their broker 🤦‍♂. Looks like there are thousands of topics, one per mobile device. Messages being sent over the wire appear to be connect/disconnect notifications with IP addresses embedded.

nonsleepr commented 4 years ago

I've looked into decompiled Android app and my understanding is that the doorbell communicates with the cloud via MQTT while the phone gets notifications via FCM with the directions on how to connect to cloud's MQTT broker. Static analysis wouldn't help here. I will try to MITM the app some time later.

keshavdv commented 4 years ago

While I only have the Eufy Floodlight and not the doorbell to test with, I've been able to identify enough parts of the UDP-based P2P protocol to be able to successfully toggle the light outside of the app. The approach seems to use the same control plane that I think the app uses to talk to the Eufy HomeBase, but I don't have one to confirm. So far, the floodlight seems entirely independent of the MQTT pipeline that seems to exist for the doorbell only.

nonsleepr commented 4 years ago

@keshavdv This is great, do you have any snippets of code you can share, a gist would be enough.

keshavdv commented 4 years ago

I’ll put something up in a bit, but it’s mostly based on a custom version of https://github.com/fbertone/lib32100 for the initial handshake and custom data packets. Interestingly, I tried the HTTP API you exposed in the other PR to set device params and while the property does seem to exist for the manual light state (1400) and is reported, flipping it only caused the app UI state to update but didn’t actually turn the light on or off.

nonsleepr commented 4 years ago

Have you checked what (other things) changes in the params when you flip the switch in the app?

nonsleepr commented 4 years ago

I was able to capture the contents of the app chatter with https://security-app.eufylife.com. It's pretty much the same is of the WebApp. It obtains a private key for the p2p communication and registers FCM token though. It then talks to the doorbell directly using 32100 protocol (thanks @keshavdv for pointing to it), it uses UDP port 10125 though (maybe just in my case).

The app also talks with security-mqtt.eufylife.com:8789, I wasn't able to decrypt that yet though.

nonsleepr commented 4 years ago

Alright! I have everything I need to start working on p2p part. Just added an endpoint here used to obtain private keys for p2p communication.

nonsleepr commented 4 years ago

@keshavdv What were you able to achieve so far? It seems that p2p communication after the handshake is different from the protocol implemented be the library.

keshavdv commented 4 years ago

After the initial handshake, the protocol is indeed different, but replaying packet payloads that were generated from the app seems to work pretty reliably to do certain things (change light/detection settings).

I've made progress on reversing the actual protocol itself and I think with the packet types I've decoded so far, we can change most of the boolean/string based parameters that are available in the app. So far, and a bit worryingly, it seems like most control actions are not encrypted/authenticated apart from the actual video stream itself (I think this is where the key returned by the endpoint you linked comes into play).

nonsleepr commented 4 years ago

Oh, you're right! I thought the settings are set via the API, but there's a P2P communication.

Tried to disable the motion detection and that's what I got:

0000  f1 d0 00 3c d1 00 00 01 58 5a 59 48 a4 06 28 00   ...<....XZYH..(.
0010  00 00 01 00 ff 00 00 00 7b 22 63 6f 6d 6d 61 6e   ........{"comman
0020  64 54 79 70 65 22 3a 31 30 31 36 2c 22 64 61 74   dType":1016,"dat
0030  61 22 3a 7b 22 65 6e 61 62 6c 65 22 3a 30 7d 7d   a":{"enable":0}}

Do you also see those XZYH all over the place?

Not encrypting the data above is indeed concerning.

nonsleepr commented 4 years ago

I saw the option to open telnet in the app disassembly. It is probably meant for the base station though. Anyway, would be cool to get telnet access.

keshavdv commented 4 years ago

Sweet, thanks for sharing! Since I only have one device, I couldn't tell what was user/device specific versus a constant, but it looks like the preamble (XYZH) is the same. Nice find on telnet! I'll see if I can enable it on my device since I think the doorbell and floodlights also act as independent "hubs/stations".

nonsleepr commented 4 years ago

The telnet command is 1247. I tried it via API but that didn't work. I think the API's params endpoint is just for the app and the actual command is sent via p2p. That explains why you weren't available to switch the floodlight.

I'll implement the p2p option setting in the next few days.

nonsleepr commented 4 years ago

@keshavdv have you figured out where's the list of IPs for the initial discovery comes from? I bet it's hardcoded somewhere.

nonsleepr commented 4 years ago

That hardcoded MQTT username/passwords are disturbing as well. No other permissions required to get doorbell events. On the other hand, we can simply subscribe to doorbell button events.

keshavdv commented 4 years ago

I didn't have any luck with turning on telnet on the floodlight -- I do think that's something only available on the homebase. In other news, I've been able to figure out enough to decode a live video stream via P2P but I'm pretty shocked by the pretty shoddy level of encryption and lack of authentication that is used to protect these.

@nonsleepr, it seems like the only authentication that is actually done is by the firebase API which I believe uses some combination of the p2p_conn and p2p_did from the HTTP API. So far, getting motion event seems like the big last milestone left.

nonsleepr commented 4 years ago

Awesome. I actually also managed to pull video steam from the camera. I decoded the frame, next step would be to integrate it with HASS. Speaking of encryption, if you leave encryptkey empty, you'll get unencrypted steam. And that bothers me when more.

I wasn't able to enable telnet either, I patched Android app to enable telnet instead of doing other actions with no success.

Haven't looked at Firebase yet.

I'm hesitant to release any code right now because of the level of security implemented by the doorbell.

nonsleepr commented 4 years ago

Oh, and FCM isn't required, it's all in MQTT.

mrjasonjordan commented 4 years ago

I'm just desperate for the motion notifications so I can switch the lights on around the house when motion is detected by any of the cameras.

Why this functionality isn't already built-in amazes me. Especially considering MQTT is already in there!

FuzzyMistborn commented 4 years ago

Why this functionality isn't already built-in amazes me. Especially considering MQTT is already in there!

Uhh...because this is an alpha integration/library that is far from being feature complete and is only maybe a month old? Give it time.

mrjasonjordan commented 4 years ago

My apologies @FuzzyMistborn - it wasn't meant as a criticism of you or your work - I think you've done a brilliant job.

I was trying to say that I thought Anker should have included the functionality.

Brickbats to them. Bouquets to you!

basriram commented 4 years ago

@keshavdv have you figured out where's the list of IPs for the initial discovery comes from? I bet it's hardcoded somewhere.

It looks like the app_conn string is encoded/encrypted with the server information of the initial discovery servers but I have not been able to figure out what type of encryption the native library is using though.

joepadmiraal commented 4 years ago

Also, in regards to MQTT it looks like Anker has hardcoded the passwords to their broker man_facepalming. Looks like there are thousands of topics, one per mobile device. Messages being sent over the wire appear to be connect/disconnect notifications with IP addresses embedded.

Were you (or anyone else) able to connect to their MQTT server? I tried with several tools (MQTT explorer, paho_cs_sub, etc.) but none of them were able to connect. I seem to get ssl errors.

nonsleepr commented 4 years ago

That issue is (being) fixed.

FuzzyMistborn commented 4 years ago

Any ETA? Not pressuring but are we close/medium/far? Just so expectations can be set.

joepadmiraal commented 4 years ago

That issue is (being) fixed.

Was that a response to my question? If so, is there a known issue on the eufy side with their MQTT service?

I'm posting here again as I now have bought an Eufy doorbell so I can help figuring out how it work.

nonsleepr commented 4 years ago

There are no issues with their MQTT, but the authorization part is flaky. I would say, the ETA for working doorbell event integration is 2-4 weeks.

FuzzyMistborn commented 4 years ago

Thanks!

Passing this one along: http://community.anker.com/t/open-api/70397/17

csrui commented 4 years ago

Unfortunately no contribution on my part but I want to let you know that this thread was a delight to read. Awesome effort that I see here. You guys are amazing.

FuzzyMistborn commented 4 years ago

Looks like Eufy has changed things up according to this OpenHAB thread: https://community.openhab.org/t/new-binding-eufy-doorbell/89513/26

Here's the github for it: https://github.com/basriram/openhab2-addons/tree/eufysecurity/bundles/org.openhab.binding.eufysecurity Looks to be in Java but might be useful.

nonsleepr commented 4 years ago

That's what I was waiting for :) (I mean, the changes).

The OpenHAB binding is 3 months old.

I will attempt to find some time this week to take a look at their changes.

FuzzyMistborn commented 4 years ago

Right, figured it was outdated but looks like it definitely had more functionality built in. I might reach out to the dev there and point him here to pool resources a bit.

basriram commented 4 years ago

@FuzzyMistborn I am the developer of that openhab binding. It was working fine until Eufy released a new firmware version for the doorbell as well as a mandatory upgrade of the mobile app. With this update Eufy has switched to using FCM (firebase cloud messaging) for the most part and doorbell, motion detection etc., are delivered as push notification to the app and their mqtt topics have no messages. Hence it makes it harder to implement outside a mobile app.

FuzzyMistborn commented 4 years ago

Thanks for the information @basriram much appreciated. That sounds good for security but obviously more annoying for our purposes. Really wish they'd do an open API or even IFTTT at this point.

asiridol commented 4 years ago

Hey everyone, I am trying to reverse engineer the Eufy security app too, not the complete app just enough to arm/disarm using an ESP32. I have enabled debugging on their app and was able to capture most of the API logs from device logs. I have found out that there's a UDP port 39855 opened on the device too. Everything looks promising so far. Except when I send the same arm/disarm request to https://security-app.eufylife.com/v1/ the results are very different. If there's an MQTT client do you guys know the port it is using and the authorization?

asiridol commented 4 years ago

I can share the debuggable apk file if anyone's interested. I have added the ability to use custom certificates to use MITM to intercept traffic.

Sillium commented 4 years ago

Hey everyone, I am trying to reverse engineer the Eufy security app too, not the complete app just enough to arm/disarm using an ESP32. I have enabled debugging on their app and was able to capture most of the API logs from device logs. I have found out that there's a UDP port 39855 opened on the device too. Everything looks promising so far. Except when I send the same arm/disarm request to https://security-app.eufylife.com/v1/ the results are very different. If there's an MQTT client do you guys know the port it is using and the authorization?

Nice! Good luck with that, I'd be really interested in arming/disarming the eufys with IFTTT, Webhook, iOS-Shortcut, flic-Button, whatever...

I cannot help you here but hope you you'll be successful.