surepetcare integration is not working anymore

[PhiHar]

The problem

The Petcare Integration is not working anymore. Also deleting, restarting und trying to reinstall does not solve the problem. It is always saying connection error. The Surepetcare App itself is working normally.

What version of Home Assistant Core has the issue?

core-2023.5.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

surepetcare

Link to integration documentation on our website

https://www.home-assistant.io/integrations/surepetcare

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[mknimmsta]

Thanks to all of those trying to find a solution. I have a raspberrypi HAOS install and happy to help if I somehow can :(

Just try my solution https://github.com/home-assistant/core/issues/93814#issuecomment-1572009788 That makes it work for now. But i guess it has to be repeated with every Core update. But it's a fix until surepy is updated to use a different user agent

[bjoernmayer]

@benleb Changing the User-Agent fixes it. What to you think of making the user agent configurable in the UI + some fancy placeholders to put in a uuid?

[flyize]

@benleb Changing the User-Agent fixes it. What to you think of making the user agent configurable in the UI + some fancy placeholders to put in a uuid?

If they want to ratchet this up, the user agent isn't going to matter. Rate limiting would be super easy, as 99% of their user base isn't going to be pinging their servers every 2 minutes.

[srk23]

Having the same issue - tried the fix above but struggling to find the files to edit. Running HAOS on a VM under Proxmox. I have the SSH/terminal addon, but the docker command is not found?

[benleb]

Hm now it is just the question if they use some product that just blocks unusual user-agents or if they explicitly blocked surepy... 🧐

I intentionally set the user-agent so something custom that they can see whats from "us" and maybe even talk to us if they see something which they do not like or whatever... little bit sad if this would now used to block us without any notice 👿 I always tried to be as friendly as possible...

But, until confirmed, I still hope its just accidentally :/ If anyone gets a reply from Sure regarding this, please let us know 👍

[bscheutjens]

Having the same issue - tried the fix above but struggling to find the files to edit. Running HAOS on a VM under Proxmox. I have the SSH/terminal addon, but the docker command is not found?

Do you have the correct addon? There are two SSH / terminal addons. You need "Advanced SSH & Web Terminal" and you'll have to turn OFF Protection mode

[Herriaan]

Having the same issue - tried the fix above but struggling to find the files to edit. Running HAOS on a VM under Proxmox. I have the SSH/terminal addon, but the docker command is not found?

I have the exact same setup as you have, and for me the steps that @mknimmsta described worked perfectly fine for me.

[Efekent]

@mknimmsta Thanks for the detailed workaround, works perfectly here.

[Laho812]

Having the same issue - tried the fix above but struggling to find the files to edit. Running HAOS on a VM under Proxmox. I have the SSH/terminal addon, but the docker command is not found?

I made the same mistake... You have to use the Advanced SSH and Web Terminal Add-on. This one has the docker command.

[benleb]

reached out to them on twitter too btw... https://twitter.com/ben_leb/status/1664285009883328515

[Laho812]

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

Edit: Changed User Agent to a real one

Thank you so much for the workaround! Used my own user agent and it worked immediately.

[SamKr]

Unfortunately @mknimmsta's steps don't work on hassos (as predicted). Any idea when this'll be implemented in the integration? Or a workaround for hassos?

[srk23]

@bscheutjens @Laho812 - that was it - doh! Now working.

[Efekent]

I intentionally set the user-agent so something custom that they can see whats from "us" and maybe even talk to us if they see something which they do not like or whatever... little bit sad if this would now used to block us without any notice 👿 I always tried to be as friendly as possible...

@benleb You did it correct that way, and they should be happy about that, since they can see where the connections coming from, I can't imagine why they will block it, there are only 640 active installations. Or some users modified your script and set the refresh rate way too short, which SurePetCare didn't like, but this just my assumption, could be something else.

[Laho812]

Unfortunately @mknimmsta's steps don't work on hassos (as predicted). Any idea when this'll be implemented in the integration? Or a workaround for hassos?

@SamKr Have you tried this solution? It works on hassos. See below:

Ok I got it working. If you are running HAOS the easiest way for me was to install this addon: Advanced SSH & Web Terminal After install disable protection mode, set a password in the config and start it

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

Hope that helps some of you

Edit: Changed User Agent to a real one

[LAB02-Admin]

Hm maybe it's something different then - I use the OVA (VM) deployment method. When logging in through SSH, I get docker: command not found":

[bscheutjens]

Hm maybe it's something different then - I use the OVA (VM) deployment method. When logging in through SSH, I get docker: command not found":

You are using the wrong addon. You have to use "Advanced SSH & Web Terminal"

[JGD78540]

where to find my SUREPY_USER_AGENT

[ghost]

When you have accessed the new SSH Terminal you can copy the file we need to change out of the container with this command: docker cp homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py . then edit it with nano nano const.py and change the line with "SUREPY_USER_AGENT" to something else like SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" save the file and exit nano to copy the file back to the docker container you need to run following command: docker cp const.py homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py after that you have to restart your Home Assistant and then you should be able to readd the integration withoout the error.

I've tried this but I think there's an issue on my side; nano does save the file (confirms with: wrote 38 lines), but after restarting the server/the Addon, it reverts back to default... (HomeAssistantOS on a VM, using the Advanced SSH & Web Terminal Addon)

If anybody knows a quick fix I'd appreciate it, but if it is more complicated that just waiting for an update I'm also ok with that.

[flyize]

You don't mention that you copied it back into the container...

[ghost]

Oh wow, my bad. Works now. Thanks! (shame on me)

[mknimmsta]

Hm now it is just the question if they use some product that just blocks unusual user-agents or if they explicitly blocked surepy... 🧐

I intentionally set the user-agent so something custom that they can see whats from "us" and maybe even talk to us if they see something which they do not like or whatever... little bit sad if this would now used to block us without any notice 👿 I always tried to be as friendly as possible...

But, until confirmed, I still hope its just accidentally :/ If anyone gets a reply from Sure regarding this, please let us know 👍

@benleb I tried "testsurepy" that works And "surepy test" that does not work. So I think it's just an If startsWith("surepy") return 403

[chrisdrackett]

can confirm that it seems to be based on surepy. I went ahead and used a browser user agent for now so hopefully my automation keep running for a longer term ;)

[thewishy]

pip3 show surepy for anybody with custom installs. Easy fix, now they get to play wack-a-mole with random user agents. Definitely better than constructive discussion with the module writer.

[mknimmsta]

pip3 show surepy for anybody with custom installs. Easy fix, now they get to play wack-a-mole with random user agents. Definitely better than constructive discussion with the module writer.

What module writer do you mean exactly?

[hweigel]

pip3 show surepy for anybody with custom installs. Easy fix, now they get to play wack-a-mole with random user agents. Definitely better than constructive discussion with the module writer.

What module writer do you mean exactly?

He meant SurePetcare playing whack-a-mole with randomized surepy user agent strings rather than talking to Ben

[thewishy]

To be clear, I mean that all of the users will update the const file to their chosen user agents, and it'll be a difficult task to manage blocking. I'm assuming they're currently doing it on the forward proxy,. It would have been super easy to speak to Ben and manage any issues with the behaviour, the URL is right in the user agent. Instead they chose to intentionally break things.

[mknimmsta]

To be clear, I mean that all of the users will update the const file to their chosen user agents, and it'll be a difficult task to manage blocking. I'm assuming they're currently doing it on the forward proxy,. It would have been super easy to speak to Ben and manage any issues with the behaviour, the URL is right in the user agent. Instead they chose to intentionally break things.

Okay than. Didn't mean to be offensive or anything but I already encountered to much "strong" opinions against people that gift their time to make such things happen

[blair287]

We should organise a day of repeatedly calling sure pet care says we would like them to work with us and try and get the message across if they want to block other smart home stuff from working with it then I'll consider it a dead product and either make my own or convert my existing stuff to esp based and get rid of surepet care.

[hweigel]

Per @Laho812's advice, I shelled into the docker container, edited

homeassistant:/usr/local/lib/python3.10/site-packages/surepy/const.py .

and changed the user agent string to

SUREPY_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"

Restarted HA, now everything is working again.

Guess all that's left to do is for @benleb to update the user agent string in surepy and we're good to go.

Last but not least, shoutout to all you guys who dug into the issue and identified a fix.

[Ryanmt2]

User agent fix works, I wonder if they have blocked surepy user agent specifically or just an upgrade to their infrastructure to filter non standard user agents.

I had to report a security issue to them once and it was hard work, took multiple emails, linked in messages etc before someone actually replied and fixed it.. I don't expect they will ever engage with this community.

Inching closer to using the local version of control and forgetting about their slow and clunky web app

[Endolf]

I'm guessing specifically surepy, I set my user agent to "Random agent" and it works :)

[Ryanmt2]

That is disappointing 👎🏻

[thewishy]

We should organise a day of repeatedly calling sure pet care says we would like them to work with us and try and get the message across if they want to block other smart home stuff from working with it then I'll consider it a dead product and either make my own or convert my existing stuff to esp based and get rid of surepet care.

I'm going with leaving honest reviews on amazon for the various products associated with the hub making it clear that they don't support local integration, and that they've maliciously broken integrations rather than discuss any issues they might have.

[JoshuaRedmond]

I've reached out to their support team (https://www.surepetcare.com/en-gb/support), but also changed my user agent by adding "Please talk to surepy" to the front of it - hopefully someone in their team will reach out to benleb if enough people reach out politely. Who knows, if server load is their concern then maybe they would be open to exposing the API locally...

[BadgerLoaf]

Whilst it’s disappointing to see it being blocked, I agree with @JoshuaRedmond that we need to ensure we stay polite - we don’t know if there are business reasons for the change, nor what their plans are.

As with many HA integrations, this isn’t official and they have no duty to support nor indeed even allow them.

Unfortunately my support queries with Sure have been very poor in terms of customer support, and given that is for BAU support, my expectations for them to engage with something like this is low.

In the short term, the main thing is we have a workaround that can be applied to the package and rolled out. Longer term we can hope for local API’s and/or more engagement from the developers; and whilst my hopes aren’t high for either, it will only be possible if we engage in civil conversations with Sure.

[slug56]

I sent a (polite) note to Sure Petcare the other day. I've got a second catflap on order, with the intention of also integrating it into Home Assistant. I spoke of the value of the ability to integrate and how it heavily influenced my decision to purchase their products. The response I've received is unfortunately pretty generic, if not borderline irrelevant:

Thank you for your interest in the Sure Pet care products!

We are currently in the process of updating our product range to include variations on our cat flaps as well as improvements on current products.

However, this is an ongoing project with new product releases scheduled over the next few years.

[chrisannen]

I sent a (polite) note to Sure Petcare the other day. I've got a second catflap on order, with the intention of also integrating it into Home Assistant. I spoke of the value of the ability to integrate and how it heavily influenced my decision to purchase their products. The response I've received is unfortunately pretty generic, if not borderline irrelevant:

Thank you for your interest in the Sure Pet care products! We are currently in the process of updating our product range to include variations on our cat flaps as well as improvements on current products. However, this is an ongoing project with new product releases scheduled over the next few years.

I think this answer might come from a bot 😏

[dirkjanvanvliet]

I'm afraid this is a real human. https://twitter.com/SurePetcare/status/1664647893637013504

[benleb]

for convenience... the notice sure petcare gave on twitter...

[BadgerLoaf]

Hopefully maybe Sure will respond or reach out to this community. None of us want their systems to be impacted by the integration, and we would much prefer to work with them to create something that enhances their products in the marketplace.

[hweigel]

for convenience... the notice sure petcare gave on twitter...

I guess they did not bother responding to you directly @benleb?

[thewishy]

Thing is, none of us really want cloud polling anyway. We know from sure local that the hub is mqtt based, the problem being it's a pain to get out talking to your own server. If they could just give us the option to define a local mqtt server (really shouldn't be rocket science), we can get off their servers.

[bruring]

Their tweet is gone now. I hope it gets resolved soon :-)

[flyize]

Their tweet is gone now. I hope it gets resolved soon :-)

No, its still there...

[glennwalsh]

Glad I found this, I thought I had screwed something up when I was playing earlier with other config.

Are we hopeful a solution will be found?

[mknimmsta]

Glad I found this, I thought I had screwed something up when I was playing earlier with other config.

Are we hopeful a solution will be found?

Current fix that works is https://github.com/home-assistant/core/issues/93814#issuecomment-1571254875 But that has to be done after every core update

[glennwalsh]

Glad I found this, I thought I had screwed something up when I was playing earlier with other config. Are we hopeful a solution will be found?

Current fix that works is #93814 (comment) But that has to be done after every core update

I'm not going to lie, that's a little bit foreign for me :(

[radischem]

Honestly, as friendly as their responses have been in the past, I never understood how they work. The app and flap have so much unused potential imho.

And for those reading this wondering why having this integrated into a smart home environment is such a big deal, here’s one example:

My cats can leave in the morning and have to stay inside at night. For this one could set times, but say your cat comes home 10 minutes before curfew and you’d like to lock the flap already, you’d have to do this manually while disabling curfew mode. This means either opening the door in the morning manually or restoring the curfew mode manually.

Unless I missed something, this by itself and without the ability to use software like home assistant makes it the dumbest smart home appliance in my house while other automations just work thanks to many conditions without me even noticing. Looking at Nuki is a great example how companies understand their customers.

Sorry for the rant and again thanks to all who helped finding a solution. That said, I think the integration should have a warning on Home Assistant website so potential customers are made aware of this issue.

[robchandhok]

So very disappointing. As noted, the proper solution is to allow local MQTT events....or just rate limit. We want them to stay in business!