home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
69.8k stars 28.94k forks source link

surepetcare integration is not working anymore #93814

Closed PhiHar closed 1 year ago

PhiHar commented 1 year ago

The problem

The Petcare Integration is not working anymore. Also deleting, restarting und trying to reinstall does not solve the problem. It is always saying connection error. The Surepetcare App itself is working normally.

What version of Home Assistant Core has the issue?

core-2023.5.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

surepetcare

Link to integration documentation on our website

https://www.home-assistant.io/integrations/surepetcare

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @benleb, @danielhiversen, mind taking a look at this issue as it has been labeled with an integration (surepetcare) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `surepetcare` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign surepetcare` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


surepetcare documentation surepetcare source (message by IssueLinks)

bscheutjens commented 1 year ago

I have the same issue. I was using the custom component (SureHA; same author), but because that one seems to be completely dead, I tried this integration and I'm having the same issue when trying to install it. But looking at the reported issues and lack of communication from the author, this one seems to be dead as well.

Ranney1 commented 1 year ago

For what’s worth, here the application is also not working anymore. It looks like that my homey with the sure petcare app is also not updating since 6 hours. Maybe an server error?

hweigel commented 1 year ago

Around 3pm today the integration stopped working for me as well.

Deleted and restarted the integration on HA 2023.5, when trying to re-add the integration, I get the error "Failed to connect".

I also tried rolling back to older HA versions up to 2022.11 without success. Tried to login with 2 different users, both of which can log in manually to surepetcare.io without issue. Both account credentials also work with the Android Sure Petcare app.

When increasing the surepy loglevel in HA to debug, I see the following errors:

`2023-05-30 20:03:54.914 DEBUG (MainThread) [surepy.client] initialization completed | vars(): {'self': <surepy.client.SureAPIClient object at 0x7f02b1a951e0>, 'email': 'REMOVED, 'password': 'REMOVED', 'auth_token': None, 'api_timeout': 60, 'session': <aiohttp.client.ClientSession object at 0x7f02e2f1cf10>, 'surepy_version': '0.8.0', 'token': None}

2023-05-30 20:03:54.914 DEBUG (MainThread) [surepy] initialization completed | vars(): {'self': <surepy.Surepy object at 0x7f02b1a96dd0>, 'email': 'REMOVED', 'password': 'REMOVED', 'auth_token': None, 'api_timeout': 60, 'session': <aiohttp.client.ClientSession object at 0x7f02e2f1cf10>}

2023-05-30 20:03:55.368 DEBUG (MainThread) [surepy.client] Response from https://app.api.surehub.io/api/auth/login: <ClientResponse(https://app.api.surehub.io/api/auth/login) [403 Forbidden]> `

PhiHar commented 1 year ago

@hweigel Got exactly the same behavior

Laho812 commented 1 year ago

Having the same issue. The sure petcare app and surepetcare.io Web version is working well, but the integration has no connection. I just created another account and added it to the household. No connection as well.

Herriaan commented 1 year ago

Having the same issue here as well. Thought first that it was because of updating HA to the newest version. Also, the SP native app is working fine.

benleb commented 1 year ago

Hm seems they changed something... I will have a look later or tomorrow ✌️

aristeu commented 1 year ago

When I logged in the website it forced me to agree to new terms, maybe that's related.

benchristian88 commented 1 year ago

mine just came back online. reloaded the integration, and its all working again now. must have been an issue at Surepet

PhiHar commented 1 year ago

@benchristian88 I am also now able to reinstall and to use the integration again.

Ranney1 commented 1 year ago

Hmm, my account is working in the app, but the mailadres does not exist when trying to login to the web page. Maybe an server issue?

Laho812 commented 1 year ago

Hmm, my account is working in the app, but the mailadres does not exist when trying to login to the web page. Maybe an server issue?

Have you tried https://surepetcare.io ?

My home assistant integration is still not working...

dirkjanvanvliet commented 1 year ago

Mine is working again. It stopped working around the same time as the opening post mentioned. I reloaded the integration this morning through the homeassistant app: settings > devices & services > sure petcare > hit the options button and select reload. And then it worked again.

hweigel commented 1 year ago

So it worked again in the morning - until 13:15 CET:

` 2023-05-31 13:12:21.641 DEBUG (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start | 6

2023-05-31 13:12:22.159 DEBUG (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/report/household/ | 2

2023-05-31 13:15:22.969 INFO (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start: 403 | <ClientResponse(https://app.api.surehub.io/api/me/start) [403 Forbidden]>

2023-05-31 13:15:22.970 DEBUG (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start | 0

2023-05-31 13:15:22.970 ERROR (MainThread) [surepy] could not fetch data ¯_(ツ)_/¯

2023-05-31 13:18:22.917 INFO (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start: 403 | <ClientResponse(https://app.api.surehub.io/api/me/start) [403 Forbidden]>

2023-05-31 13:18:22.918 DEBUG (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start | 0

2023-05-31 13:18:22.918 ERROR (MainThread) [surepy] could not fetch data ¯_(ツ)_/¯

2023-05-31 13:21:22.921 INFO (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start: 403 | <ClientResponse(https://app.api.surehub.io/api/me/start) [403 Forbidden]>

2023-05-31 13:21:22.922 DEBUG (MainThread) [surepy.client] 🐾 · GET app.api.surehub.io/api/me/start | 0

2023-05-31 13:21:22.922 ERROR (MainThread) [surepy] could not fetch data ¯_(ツ)_/¯ `

Reloading the integration now throws the same error as yesterday.

bjoernmayer commented 1 year ago

@benleb Are you in contact with surepetcare?

Either they keep breaking their API or they've setup some rate limiting

benleb commented 1 year ago

@benleb Are you in contact with surepetcare?

Either they keep breaking their API or they've setup some rate limiting

nope, I tried several times when I started to work on surepy but they never showed any interest in contact with me or surepy at all. So I gave up a long time ago 😢 🤷‍♀️ (if anyone from sure petcare reads this, feel free to contact me :D) But, on the positive side... they also never blocked anyone of us for using surepy/the home assistant integration :) 👍 So I guess they are also not interested to harm us this time and just do some updates or whatever on their API/infrastructure ✊

My guess is that the integration will start working as usual again by itself in the next 1-2d :) Otherwise I will have a look as soon as they have finished whatever they do :D

PhiHar commented 1 year ago

@benleb Thank you!

flyize commented 1 year ago

@benleb Are you in contact with surepetcare?

Either they keep breaking their API or they've setup some rate limiting

I don't think it could be rate limiting, as the website still works from the same IP.

Ranney1 commented 1 year ago

Here the same. Homey and home assistant app not working.

chrisannen commented 1 year ago

I have the same problem than yesterday. Surepetcare integration is not working since some hours. Is the problem again the connection to surepetcare or has my homeassistant any issues?

radischem commented 1 year ago

Can confirm this. Had both integrations running and they worked until a few days ago. I had issues connecting to the hub during the weekend, but that may be due to lan issues which is can’t confirm. I reloaded the integrations which didn’t work. Then I decided to redownload which didn’t change anything. Removing and trying to reinstall made it actually worse as I can’t even log in anymore and neither integration finishes the install process. Already dislike that they require you to contact them in order to know which code to chose for a metal door or rechargeable batteries. I sure hope they didn’t do something stupid :(

mknimmsta commented 1 year ago

It's not working for me either. I also get a "Failed to connect" if I try to reinstall it. I guess they changed something minor because I can still access the API from Postman after getting a Token with the auth request.

Steps to reproduce:

  1. URL: https://app.api.surehub.io/api/auth/login METHOD: POST BODY: { "email_address": "REMOVED", "password": "REMOVED", "device_id": "RANDOM_UUIDv1" }
  2. URL: https://app.api.surehub.io/api/pet METHOD: GET AUTH: Bearer with token from Step 1
  3. URL: https://app.api.surehub.io/api/pet/ID_FROM_STEP_2/position METHOD: GET AUTH: Bearer with token from Step 1

In CURL:

  1. curl --location 'https://app.api.surehub.io/api/auth/login' \ --header 'Content-Type: application/json' \ --data-raw '{ "email_address": "REMOVED", "password": "REMOVED", "device_id": "RANDOM_UUIDv1" }'
  2. curl --location 'https://app.api.surehub.io/api/pet'\ --header 'Authorization: Bearer TOKEN_STEP_1'
  3. curl --location 'https://app.api.surehub.io/api/pet/ID_FROM_STEP_2/position' \ --header 'Authorization: Bearer TOKEN_STEP_1'

Hope this helps to find a fix sooner. If I can help more just contact me.

zumheulen commented 1 year ago

Greetings from the Schwobeländle, I too have the same problem, felt after the core update.

happyclayton commented 1 year ago

Thanks for looking at (and maintaining) this @benleb Sorry to say it stopped working for me some time during yesterday, though. When I try to run the close automation I get:

Stopped because an error was encountered at 1 June 2023 at 07:44:46 (runtime: 0.01 seconds)

value must be one of [] for dictionary value @ data['flap_id']

Hopefully either Sure will "un-break" it, or you'll find a workaround.

Thanks again for your efforts - our automation timed to let the cat out at sunrise and lock him in an hour before sunset (so that he can't get run over at night - much more common than in daylight) is one of our favourite uses for homeassistant.

flashi81 commented 1 year ago

Same for me. Unable to connect via HA. Connect via app or direct URL is possible.

hweigel commented 1 year ago

Unlike yesterday, the problem didn't go away overnight. Reloading the integration continues throwing the errors encountered during the last two afternoons, so chances are the changes may be permanent.

Since @mknimmsta was able to retrieve a token: @benleb maybe you could have another look?

Alex-ala commented 1 year ago

They don't like us anymore: Using no user agent (or the one curl automatically sends) works. The uuid is random:

curl -s -XPOST https://app.api.surehub.io/api/auth/login -H "Content-Type: application/json" -d'{"email_address": "valid-email", "password": "valid-pw", "device_id": "e445bcc0-004a-11ee-be56-0242ac120002"}' | jq
{
  "data": {
    "user": {
      "id": valid id,
      "email_address": "valid-email",
      "first_name": "valid user",
      "last_name": "valid user",
      "country_id": 43,
      "language_id": 37,
      "marketing_opt_in": false,
      "terms_accepted": "2022-11-08T11:24:50.000000Z",
      "weight_units": 0,
      "time_format": 0,
      "version": "Ng==",
      "created_at": "2021-09-14T15:59:37+00:00",
      "updated_at": "2022-11-08T11:24:50+00:00",
      "notifications": {
        "device_status": true,
        "animal_movement": false,
        "intruder_movements": false,
        "new_device_pet": true,
        "household_management": true,
        "photos": true,
        "low_battery": true,
        "curfew": false,
        "feeding_activity": false,
        "drinking_activity": false,
        "feeding_topup": false,
        "drinking_topup": false
      }
    },
    "token": "valid token"
  }
}

Doing the very same with using surepy's user agent yields a 403:

curl -XPOST https://app.api.surehub.io/api/auth/login -H "Content-Type: application/json" -H "User-Agent: surepy dev - https://github.com/benleb/surepy" -d'{"email_address": "valid-email", "password": "valid pw", "device_id": "e445bcc0-004a-11ee-be56-0242ac120002"}'
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
</body>
</html>

(edit): Fixed the second request with a correct header. Described behavior still is correct.

That said, when I change SUREPY_USER_AGENT to something random, it works. I havent tested that workaround within HA yet. Just with surecli.

Alex-ala commented 1 year ago

I hacked a different user agent into deps/lib/python3.10/site-packages/surepy/const.py (Im using sureha via hacs, so surepy is located here), restarted HA and it works again... We will see for how long. I assume they implemented one of these scenarios:

  1. Rate limiting per user agent (as lots of people use surepy {version} - https://github.com/benleb/surepy with either dev or a small number of different versions`
  2. Blocked anything that includes some parts of surepy {version} - https://github.com/benleb/surepy. I haven't tested using some subtsrings from this user agent.
  3. Rate limiting base on a substring of surepys user agent.
radischem commented 1 year ago

If limiting is the case they should just be open about it. Basically I’d need a handful of calls per day. If someone needs a lot more, give options like subscription plans or so. just finished my irrigation project away from Gardena as I hate cloud services and this is the perfect example why I do. Unfortunately I can’t program or anything so all I can do is provide info or provide time for testing and rely on all the helpful people here to find a solution.

PhiHar commented 1 year ago

I also hate the cloud, so maybe this is a solution for all users with the Hub V1: https://pethublocal.github.io/

Sadly I got myself an Hub V2, which should not work with pethublocal. I will try to implement this in the next month.

Alex-ala commented 1 year ago

The unreliability of SurePets cloud and the lack of alternatives is why I currently build my own catflap... Within the last months the API wasn't properly working for almost every second morning. When its working the api aswell as the app are tediously slow (who wants to wait 10+ seconds for an api response?).

With surepy it's been ok-ish but if they really start implementing counter measures against foreign api usages, we'd have to see what we can do. I've been reaching out for them asking for an open API, a documentation, official modules or even firmware sources, with only negative responses so far.

I'd wait for tomorrow to see if a per-user-useragent still works. Then have every HA instance could use a more or less unique user agent.

mknimmsta commented 1 year ago

@Alex-ala I tried to change that aswell but my deps directory is empty. Any guess where I can find the dependencies on my installation on a Raspberry Pi with HAOS?

bobjoylove commented 1 year ago

+1 cannot connect to the hub with HA.

blair287 commented 1 year ago

I hacked a different user agent into deps/lib/python3.10/site-packages/surepy/const.py (Im using sureha via hacs, so surepy is located here), restarted HA and it works again... We will see for how long. I assume they implemented one of these scenarios:

  1. Rate limiting per user agent (as lots of people use surepy {version} - https://github.com/benleb/surepy with either dev or a small number of different versions`
  2. Blocked anything that includes some parts of surepy {version} - https://github.com/benleb/surepy. I haven't tested using some subtsrings from this user agent.
  3. Rate limiting base on a substring of surepys user agent.

What's the full file path if I go to config and deps that folder is empty.

Thanks

Alex-ala commented 1 year ago

My example was for a manual install of HA with using HACS for the integration. With this setup the dependencies for HACS libraries are within the configuration directory deps/lib/...

If you are using HA with a Python venv and use surepy without HACS, it will be in the venv's directory under lib/..

Docker and HASSOS deployments might be a bit more tricky. I have none around to test, but I think they are stored within a Docker volume. You'd have to bash into the Docker container to modify those.

blair287 commented 1 year ago

My example was for a manual install of HA with using HACS for the integration. With this setup the dependencies for HACS libraries are within the configuration directory deps/lib/...

If you are using HA with a Python venv and use surepy without HACS, it will be in the venv's directory under lib/..

Docker and HASSOS deployments might be a bit more tricky. I have none around to test, but I think they are stored within a Docker volume. You'd have to bash into the Docker container to modify those.

I'm running Hass os so guess I'm out of luck surely this file must be in the install files somewhere how else does the addon know what to do?

mknimmsta commented 1 year ago

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

Edit: Changed User Agent to a real one

mknimmsta commented 1 year ago

I also verified that they block user agents that start with surepy. What a move....

hweigel commented 1 year ago

Would somebody explain to me why they have an API when they don't want 3rd parties to use them?

flyize commented 1 year ago

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Chrome" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

I'd suggest we use an actual valid user agent now that they're watching.

flyize commented 1 year ago

Would somebody explain to me why they have an API when they don't want 3rd parties to use them?

They don't have an open API.

mknimmsta commented 1 year ago

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Chrome" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error. Hope that helps some of you

I'd suggest we use an actual valid user agent now that they're watching.

Yeah I think that too. I looked at the Web App that also uses this API and it just uses a normal User Agent and no custom one. In my case Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 So I'm using this for now

bscheutjens commented 1 year ago

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Chrome" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

Thanks, those steps work perfectly. For those who are interested: I run Home Assistant OS in a virtual machine on ESXI, which is created with the official ova file.

radischem commented 1 year ago

I also verified that they block user agents that start with surepy. What a move....

Meaning they confirmed that they don’t want users to integrate their product into one of the most used Smart Home Software?

I’ll contact them as well to make sure they know that this really is their intention, but if so, I guess I’m hopping from website to forum to customer review to ensure no one buys the product, thinking they can integrate it. :/

mknimmsta commented 1 year ago

I also verified that they block user agents that start with surepy. What a move....

Meaning they confirmed that they don’t want users to integrate their product into one of the most used Smart Home Software?

I’ll contact them as well to make sure they know that this really is their intention, but if so, I guess I’m hopping from website to forum to customer review to ensure no one buys the product, thinking they can integrate it. :/

If we use a valid User Agent they can't block it that way anymore or their web app won't work too.

blair287 commented 1 year ago

I hate any smart home product that tried to lock stuff down was going to buy one of their cat flaps rethinking that now.

christianschneider89 commented 1 year ago

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

Edit: Changed User Agent to a real one

I can confirm this workaround. Changed to my browsers user agent and it works.

blair287 commented 1 year ago

Changed my user agent and also working thanks

radischem commented 1 year ago

Thanks to all of those trying to find a solution. I have a raspberrypi HAOS install and happy to help if I somehow can :(