search

0bmay / py-canary

MIT License
3 stars 1 forks source link

troubleshooting HA's py-canary thread #1

Open 0bmay opened 2 years ago

0bmay commented 2 years ago

moving the conversation here to not flood HA's issue

nielsaust commented 2 years ago

but I need the sensors back as well

@CBOFred: would be nice. I think that's an issue with Canary itself. You also did not see them in the Canary app right? I experience it more often lately that they don't show. No idea why however. Think I'll try to contact support at some point.

@0bmay Would getting back 2FA be at all possible? Does make me feel a bit safer.

0bmay commented 2 years ago

running the run_api,py file should result in the following output:

[EDIT] see message below for corrected output

0bmay commented 2 years ago

@nielsaust I will work on getting 2FA working after this next update that better handles the camera image and adds a few more entities in HA..

which currently looks like this: screen shot of HA entity list

nielsaust commented 2 years ago

I have no idea how to run custom code however. If you point me in the right direction I could also run it, although like said, I think the sensors issue is Canary's issue, not HA's or py-canary's.

nielsaust commented 2 years ago

I will work on getting 2FA working after this next update that better handles the camera image and adds a few more entities in HA..

which currently looks like this:

Wow, you rock! Gets me pretty interested in getting more knowledgeable about python programming.

0bmay commented 2 years ago

pushed a new version of the script that will auto-redact the locationid and hash value from the live stream URL..

Output now looks like: 2022-05-10 14:35:19,544 - main - INFO - device Office is a Canary Pro and is online 2022-05-10 14:35:21,613 - main - INFO - device Office live stream session url = https://my. canary .is/api/watchlive/--loc_id--/--hash--/stream.m3u8 2022-05-10 14:35:21,613 - main - INFO - Latest Readings by device... 2022-05-10 14:35:21,613 - main - INFO - device Office - sensor: AIR_QUALITY value: 0.8130775315964476 2022-05-10 14:35:21,613 - main - INFO - device Office - sensor: HUMIDITY value: 53.73205497417044 2022-05-10 14:35:21,614 - main - INFO - device Office - sensor: TEMPERATURE value: 21.145213066263402 2022-05-10 14:35:21,614 - main - INFO - device Office - sensor: WIFI value: -40

0bmay commented 2 years ago

@nielsaust I never knew Canary HAD 2FA, so thanks for brining that up! At least there are a few examples on how to code for 2FA.. but that's a big change, for both the HA integration and py-canary library.

0bmay commented 2 years ago

@CBOFred I know Canary had an issue with its SSL certificate over the weekend (or yesterday?) that messed me up for a while... but no data since the 7th seems odd to me..

nielsaust commented 2 years ago

I was just chatting with Canary support and: "Our team recently investigated reports from a subset of users that are experiencing some difficulties with their Canary Pro device related to the Home Health feature.

The team is investigating the issue and we will keep you informed."

nielsaust commented 2 years ago

@CBOFred: just wanted to let you know that it would not hurt contacting support. Like said above does (or COULD depending on your situation) this confirm that it's Canary's issue but more importantly, I've been generously compensated for my troubles (50% off for the next 12 months!). Hope you read this.

0bmay commented 2 years ago

@xImportExportx I have an updated build of py-canary and home assistant that might work, but it would involve running HA from source... as I don't know if it will actually fix the issue or not, since I have a subscription to Canary.

But it adds the masquerading headers to the ffmpeg calls.. Doesn't seen to affect me with a subscription one bit. 🤷

xImportExportx commented 2 years ago

@0bmay I'm running HA as a docker container mainly as bridge to accessories not support yet by Homebridge (or better support in Home assistant). If there's a way I can test in the container happy to try. In terms of other data I am unable to tell you as I have none in the app either, I don't particularly rely on that data myself. I am testing all this native in HA as well.

Someone mentioned a shell script to pull up a video feed, could I somehow try that to see if the issue is fixed.

I also DO NOT have 2fa enabled.

doomrt commented 2 years ago

I, first time poster but long time reader.

I have the same kind of situation.

HA setup running on VM on a MAC, 3 X canary pro without subscription. Up to april 27th I never had video but all sensors worked flawlessly. I kept an eye out for anyone investigating the issue, since Canary support wouldn't comment on anything HA related. Can't say how happy I am to see this thread (followed here from the temperature issue and video issue) and I have to say thank you to @0bmay for digging in this.

Now, my HA is updated to the latest version but I realized my other sensor readings have actually been down since may 6th. I have notified Canary as HomeHealth infos do not appear anymore neither on the app or website. I still see cameras online and live streaming is working fine.

I will keep you posted so you can know if the solution works for non-subscription setups once Canary fix the Homehealth issue on their side.

Edit: no 2FA here.

Thanks!

0bmay commented 2 years ago

@doomrt Interesting! Just to make sure I understand, live streaming of video and camera image is working fine, but your other home health sensors are not working?

@nielsaust found out that the settings are down for some users.. I guess I'm one of the lucky ones that the sensors still work.

doomrt commented 2 years ago

@0bmay I could have been a little clearer; english is not my first language.

I could never get any image on HA since I started using it this fall. I can access live streaming on the app, but it's paywalled on the website. I'm pretty certain it is related to the subscription, and everyone able to get the camera feed on HA seems to have an active subscription.

Now for my other sensors, temperature stopped showing up around the same time as everyone +- april 27th, and my last air quality and humidity readings are on may 6th.

For now, nothing works.

0bmay commented 2 years ago

@doomrt thank you for the followup and clarification. It does appear that the camera is behind a paywall, adding headers to the ffmpeg call might fix this, but I can't test this. The next version of the canary integration will have the headers added.

The sensors, however, should be working wether or not you have a subscription. But as we have found out through support, some users HomeHealth data is not working, so for now, that is not an issue with the HA integration.

doomrt commented 2 years ago

@0bmay Correct, we are on the same page.

I will report back when the HomeHealth issue is resolved on Canary's side; i'll know when I can see the sensor readings on the app.

Then, we will see if the other fixes are working. Fingers crossed, thanks again!

CBOFred commented 2 years ago

I noticed an interesting thing. When I opened the Canary app on my iPad (which I hadn't used for several weeks) instead of my phone, I got sensor data... or so I thought. It appeared that it was showing "frozen" values of historical data.

I wonder if the problem with the sensor data could be related to a bad firmware update. I remember that I got a notification about the camera losing connection and then reconnected some days ago, which might correlate roughly with the point in time when I lost sensor data. I was thinking that maybe there was a firmware update which triggered the disconnect notification and now there is some kind of problem with that version of the firmware.

According to the app i am on firmware version 5.6.0 (Canary Pro)

0bmay commented 2 years ago

@doomrt, @xImportExportx and @nielsaust what firmware are you on?

I'm still on 5.5.0 and might be why I still have HomeHealth data.

xImportExportx commented 2 years ago

@doomrt i am on 5.6.0 interestingly when I go to devices and tap a device in the preview there is accurate sensor data below. That kinda implies it’s the cloud service not recording the data as opposed to the sensors not working….

doomrt commented 2 years ago

@CBOFred I think you might be right. All three of my cameras are on 5.6.0.

Also, probably unrelated, but I want to check which version of the integration i'm running, and can't seem to find it. Is there an easy way to make sure it upgraded to 0.5.2? I tried to look it up on google but it wasn't conclusive.

CBOFred commented 2 years ago

@doomrt i am on 5.6.0 interestingly when I go to devices and tap a device in the preview there is accurate sensor data below. That kinda implies it’s the cloud service not recording the data as opposed to the sensors not working….

I also get sensor data in the device preview, but they seem to be old data, just as on my iPad.

0bmay commented 2 years ago

@doomrt if you are running HA 2022.5.3 you should be running py-canary=0.5.2 as HA attempts to install/upgrade changed python libraries upon first start. I can't find a way from HA's front end to see what is installed.

If you know where your HA virtual environment is in the vm, you could look in /lib/python3.9/site-packages/py_canary-0.5.2.dist-info

If that directory exists, you are on 0.5.2 if it's anything else, you are not. Let me see if I can start up a macOS VM to see where it's located..

xImportExportx commented 2 years ago

I just ran the following in my container to see if there was an update or if I was running the latest version

pip install py-canary==0.5.2 --upgrade

0bmay commented 2 years ago

Does anyone here have a Canary View? I think it's a Canary Pro without the HomeHealth sensors, but would like to verify..

doomrt commented 2 years ago

I just ran the following in my container to see if there was an update or if I was running the latest version

pip install py-canary==0.5.2 --upgrade

I tried it and it doesn’t seem to work.

doomrt commented 2 years ago

Quick follow up, canary support emailed back about the home health issue. It seems that have identified the problem and a fix is coming soon.

@nielsaust Our team recently investigated reports from a subset of users that are experiencing some difficulties with their Canary Pro device related to the Home Health feature.

We have identified the issue and expect that this will be resolved soon

0bmay commented 2 years ago

@doomrt I loaded up the Home Assistant VM and was unable to find out how to show if a python module is installed or not through the home assistant os. I tried a google search and nothing came up either...

xImportExportx commented 2 years ago

@0bmay I spun up a HASSOS vm to test, these instructions are confirmed as working

docker exec -it homeassistant /bin/bash once you are in to the docker container shell you should be able to run pip list | grep py-canary

that will give you the version installed something like below

bash-5.1# pip list | grep py-canary
py-canary                        0.5.2

fairly handy it seems to run the home assistant container just like my installation but they just add a few extra containers for various things in the OS.

xImportExportx commented 2 years ago

@doomrt have a look at the above to get your py-canary version from HASSVM or Home Assistant as a container

0bmay commented 2 years ago

Thank you @xImportExportx

0bmay commented 2 years ago

py-canary v0.5.3 has been released. Entry API works again, code refactoring and other tweaks have been made. At least 1 more night of testing updates to Home Assistant to merge the changes there, which include:

Then 2FA work will begin. Looks like it uses text messaging only, which I think I will have to make the UI similar to iCloud support (done via Settings -> Devices & Services -> Canary -> Config and store the token (which I think lasts 30 days?)

xImportExportx commented 2 years ago

@0bmay so have a look at how the blink integration handles 2fa, there's no going in to the config file or anything. Blink mandates 2fa via text message as well, there's no option to turn it off

0bmay commented 2 years ago

Thank you @xImportExportx I will look there too.. always nice to have a few examples.

Also, I'm not sure if I will be able to fix non-subscription video feeds as trying to snoop on the iOS app's web calls completely breaks the app to where no calls work. hah.. secure...

xImportExportx commented 2 years ago

@0bmay have you thought about running a proxy for the app. I remember back in the days I used to jailbreak and wanted Siri on my iPhone 4 I created a small reverse proxy with a self signed certificate. I simply imported the certificate to the device using a profile that marked my own created SSL cert valid and it allowed Siri from an iphone4s to communicate via my machine and I could grab the tokens from the calls there. It depends if there is a hard dependance on the signed cert or if they are offloading trust to the phones store. Likely the latter with the closed nature of the iOS. It would also be odd to write your own SSL handling in your app when the chain of trust is present in the OS.

A good example of this in use more widespread is Plex Connect for Apple TV 3, you simply create a self signed cert, import to the Apple TV and you can successfully take over an app.

Let me know if you need any help setting this up? Also sorry if I am teaching you things you already know, just wanted to add my 2 cents incase you weren't aware.

0bmay commented 2 years ago

@xImportExportx yea.. I have tried using Proxyman, which is like a man-in-the-middle app, as a on device proxy and from my mac. Canary has SSL-Pinning enabled. I even allowed untrusted connections and still no dice.

xImportExportx commented 2 years ago

@0bmay before I dig out notes and get my head in to an SSL habit hole, are you sure its got pinning enabled. Just seems like a potential problem in the instance there is a reverse proxy or something in the middle resigning SSL (lots of ISPs in less developed countries and corpo networks do this). Would break it in the majority of these situations. Real quick suggestion would be to use this project https://mitmproxy.org and proxy all connections from the Phone via the MITM proxy.

0bmay commented 2 years ago

I was able to successfully MinM other sites.. I will give this proxy a go and see what happens..

0bmay commented 2 years ago

192.168.xxx.xxx:59532: server connect api.canaryis.com:443 (54.208.24.248:443) 192.168.xxx.xxx:59532: Client TLS handshake failed. The client disconnected during the handshake. If this happens consistently for api.canaryis.com, this may indicate that the client does not trust the proxy's certificate. 192.168.xxx.xxx:59532: client disconnect

xImportExportx commented 2 years ago

I'm just trying to figure out the MITMPROXY myself, ill report back if I get it working

xImportExportx commented 2 years ago

Ok @0bmay I got it working. Little bit finickerty though. Get the MITMPROXY setup so you can browse to google (install certs to phone and set proxy). Then once you are happy thats working then you need to disable the proxy, launch the app, re enable the proxy then hit live view. Looks like pinning is enabled on the API BUT not on subsequent connections to aws backends.

0bmay commented 2 years ago

Yea.. the connections to the AWS backend is usually the entry (motion detection) images and the video.. BUT in order to get the stream data it needs to call https://my.canary.is/api/watchlive///stream.m3u8

What I don't know is, what the headers and data transmitted by the app are... i can only guess it's similar to the website. I wasn't able to to view the video in the app, even with your trick...

If you can log the ttps://my.canary.is/api/watchlive/ urls (please remove your id) and the request and response data, I can see if they match what py-canary is already doing... I have a contact me form on my github bio that accepts files.. if you can send the logs...

xImportExportx commented 2 years ago

Ok buddy, I’ll see what I can do.

nielsaust commented 2 years ago

I'm loving this conversation. You guys ROCK.

doomrt commented 2 years ago

Hi! Quick update:

Two of my three cameras have updated to firmware 5.6.2. Temperature readings resumed immediately, so all sensors are now working as expected. Great work !

As for the cameras, still no image or preview; same as before.

Thanks !

0bmay commented 2 years ago

@doomrt I'm glad the sensors have returned!

As for the image preview and capture, the next release should help, I hope. It's hard to troubleshoot a scenario that I can't do. If I had unlimited loot, I'd get a Canary Flex to test in a free user mode, that way I could test battery level and free mode too.

0bmay commented 2 years ago

I've updated the run_api.py testing script. If you can run that and take the live stream URL that's printed out and load it up in VLC, if the camera plays, we will be in a good place...

The output will be something like: 2022-05-14 11:47:33,345 - main - INFO - device Office is a Canary Pro and is online 2022-05-14 11:47:34,624 - main - INFO - device Office live stream session url = https://my. canary .is/api/watchlive/######/--(uuid)--/stream.m3u8 .. ..

Let me know...

0bmay commented 2 years ago

The good news is, I have shell script level 2FA working. Now to plug it into HA's config... [edit] in the UI, not yaml files.

nielsaust commented 2 years ago

Hey guys. The image had stopped working a few versions (of Home Assistant) back. Do you experience the same. I think I'll drop the use of Canary in Home Assistant. Or is there still hope? Cheers.

0bmay commented 2 years ago

Mine is still working. Sometimes it takes a few tries (up to 30 seconds) to actually display an image.

The latest version of HA has a version of the canary integration that will cache the image for 15 minutes. There are 2 more pull requests in the queue to add more entities and then add Canary View support.

How canary shows video, even on canary.is is not very optimal. Videos play too fast, and every time you view video it creates a new "motion capture" or entry event. I don't understand why they made some of the choices they made, but it made sense to someone. Hah.

I'm also working on 2fa support, i have it working in 0.60 of py-canary, just need to finish the implementation in HA.