gaggle331 / home-assistant-kevo

Kevo Custom Component for Home Assistant
12 stars 14 forks source link

Getting "500" error in logs #12

Open hdprice opened 2 years ago

hdprice commented 2 years ago

Hi.

I installed the integration via HACS and added the suggested config into configuration.yaml. After a restart I'm seeing this in the logs and the integration isn't able to get the status of the locks or control them.

2021-11-14 22:21:45 ERROR (MainThread) [homeassistant.helpers.entity] Update for lock.kitchen_door fails Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 438, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 625, in async_device_update raise exc File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run result = self.fn(*self.args, self.kwargs) File "/config/custom_components/kevo_plus/lock.py", line 98, in update File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 108, in _wrapped return method(self, *args, *kwargs) File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 272, in GetBoltState self.Refresh() File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 108, in _wrapped return method(self, args, kwargs) File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 223, in Refresh raise KevoError("Error getting lock info: {}".format(info_result.text)) pykevoplus.KevoError: Error getting lock info: {"status":"500","error":"Internal Server Error"}

Any ideas what could be causing this?

Thanks!

rkaramandi commented 2 years ago

I just got this and I'm seeing the same. Were you able to get anywhere with it @hdprice?

hdprice commented 2 years ago

Unfortunately, no. I gave up and removed the integration. I switched to using Alexa to automatically lock the doors.

On Mar 24, 2022, at 2:06 PM, Rohan Karamandi @.***> wrote:

I just got this and I'm seeing the same. Were you able to get anywhere with it @hdprice https://github.com/hdprice?

— Reply to this email directly, view it on GitHub https://github.com/gaggle331/home-assistant-kevo/issues/12#issuecomment-1078301322, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADH4775R66L4MPQEB2KXRWDVBTKNTANCNFSM5IA4TUSA. You are receiving this because you were mentioned.

-- All content is personal opinion. All content is copyright. All rights are reserved. No reproduction in any form is allowed without prior written permission 

rkaramandi commented 2 years ago

Dang, ok. Thanks!

dcmeglio commented 2 years ago

Spent a lot of time troubleshooting this. It appears Kevo has some throttling around the Login API. I'm still learning python and home assistant, but frequent calls to the Login method start generating the 500 errors. Seems like something needs to be done in the code to better cache the session cookie. Like I said though, I don't yet know enough about HA programming to know exactly how we do that. Basically while reproducing the error I then went to the Kevo website and I logged in. I get :

{
    "HttpStatusCode": 429,
    "Message": "Too Many Requests",
    "DetailedMessage": "You have made too many requests. Wait a little while and try again."
}

So it's pretty clear it's a rate limit issue. Just trying to see how we fix it!

More updates, if you have multiple locks this issue is almost guaranteed to occur. Each lock instance logs into the api separately so it's just going to increase how fast you get rate limited. Same with the retry count. Each time it does a retry it re-logs in. There is a lot of room for improvement in this code.

dcmeglio commented 2 years ago

I'm making progress. I've got it working a lot better but not fully working yet!

dcmeglio commented 2 years ago

Got it generally working now. The root of the problem is both the way this integration, and the underlying kevo library are written cause a lot of login API calls which Kevo rate limits. I've worked around that. Last thing is to make sure things still work when the cookie expires. My code is super hacky since I'm still learning Home Assistant coding, but it's at least getting there!

hdprice commented 2 years ago

Got it generally working now. The root of the problem is both the way this integration, and the underlying kevo library are written cause a lot of login API calls which Kevo rate limits. I've worked around that. Last thing is to make sure things still work when the cookie expires. My code is super hacky since I'm still learning Home Assistant coding, but it's at least getting there!

I really appreciate you doing all this work. It's so great of you to do this. I'd like to support you with a donation, you know, some beer money.. :) Let me know if you have a Patreon account or some other way for me to show my appreciation.

dcmeglio commented 2 years ago

Well first let's see if it actually works for folks other than me! If you wanted to test, I think if you install from this repo https://github.com/dcmeglio/home-assistant-kevo it should work. I've been running it for a few days now and it seems to be working fine. Logs show occasional errors getting lock status but this so far seems to be flakiness in their API. I'm still investigating. But so far it's working a lot better than the original. The code is still super ugly so I'm trying to use this as an opportunity to learn some Home Assistant programming. Oh other thing I added was support for uniqueids so now the lock entities can be assigned to Areas via the UI

RXM307 commented 2 years ago

Thanks trying this out now!

RXM307 commented 2 years ago

Thanks trying this out now!

getting this error

2022-06-20 20:18:56 ERROR (MainThread) [homeassistant.components.lock] Error while setting up kevo_plus platform for lock Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 249, in _async_setup_platform await asyncio.shield(task) File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/kevo_plus/lock.py", line 53, in setup_platform cookie = Kevo.Login(email, password) TypeError: Login() missing 1 required positional argument: 'password'

is there any change to the config?

lock:

dcmeglio commented 2 years ago

Ah I know what the problem is. This is my inexperience with Python. The problem is I had to update and override the pykevoplus library. Any chance you're running HAOS? If so, do you have the SSH addon installed? You'll need to do an ha core rebuild after installing the component so it clears all of the python caches. If you can't do that, rebooting the host device should fix it.

RXM307 commented 2 years ago

Sorry not running HAOS, running in Docker (2022.6) I have done a full restart (docker-compose down && docker-compose up -d) I cant see any caches in the exposed file systems...

dcmeglio commented 2 years ago

Sorry not running HAOS, running in Docker (2022.6) I have done a full restart (docker-compose down && docker-compose up -d) I cant see any caches in the exposed file systems...

Sorry for the delay, I was learning how to publish a package to pypi. I just published a new version to HACS that should fix the issue you were having. It now should download the package to talk to kevo from pypi which should bypass the issue you had. Can you try again?

RXM307 commented 2 years ago

Sorry not running HAOS, running in Docker (2022.6) I have done a full restart (docker-compose down && docker-compose up -d) I cant see any caches in the exposed file systems...

Sorry for the delay, I was learning how to publish a package to pypi. I just published a new version to HACS that should fix the issue you were having. It now should download the package to talk to kevo from pypi which should bypass the issue you had. Can you try again?

same issue 2022-07-06 22:25:23 ERROR (MainThread) [homeassistant.components.lock] Error while setting up kevo_plus platform for lock Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 249, in _async_setup_platform await asyncio.shield(task) File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/kevo_plus/lock.py", line 53, in setup_platform cookie = Kevo.Login(email, password) TypeError: Login() missing 1 required positional argument: 'password'

dcmeglio commented 2 years ago

Hmm this is really confusing.

Any chance you can SSH in and show me the output to the following:

cat /config/custom_components/kevo_plus/manifest.json

RXM307 commented 2 years ago

Hmm this is really confusing.

Any chance you can SSH in and show me the output to the following:

cat /config/custom_components/kevo_plus/manifest.json

cat /config/custom_components/kevo_plus/manifest.json

{ "domain": "kevo_plus", "name": "Kevo Plus", "documentation": "https://github.com/gauvingingras/kevo_plus", "dependencies": [], "codeowners": ["gauvingingras","bahnburner","gaggle331","plmilord","davidlang42","dcmeglio"], "requirements": ["pykevoplusnew==3.0.0"], "version": "0.1.0" }

RXM307 commented 2 years ago

Hmm this is really confusing. Any chance you can SSH in and show me the output to the following: cat /config/custom_components/kevo_plus/manifest.json

cat /config/custom_components/kevo_plus/manifest.json

{ "domain": "kevo_plus", "name": "Kevo Plus", "documentation": "https://github.com/gauvingingras/kevo_plus", "dependencies": [], "codeowners": ["gauvingingras","bahnburner","gaggle331","plmilord","davidlang42","dcmeglio"], "requirements": ["pykevoplusnew==3.0.0"], "version": "0.1.0" }

this got me thinking, could it be these mounts in my docker-compose causing these issues?

            "/etc/homeassistant/pykevocontrol-2.0.0.dist-info:/usr/local/lib/python3.8/site-packages/pykevocontrol-2.0.0.dist-info:rw",
            "/etc/homeassistant:/config:rw",
            "/etc/localtime:/etc/localtime:ro",
            "/etc/homeassistant/pykevoplus:/usr/local/lib/python3.8/site-packages/pykevoplus:rw"
dcmeglio commented 2 years ago

Yup! That pykevocontrol sound like it's going to cause your issue. You're going to need a mount for pykevoplusnew

RXM307 commented 2 years ago

Yup! That pykevocontrol sound like it's going to cause your issue. You're going to need a mount for pykevoplusnew

I can't recall why I mounted those, I think there was some issue in the past with dependancies being downloaded, I've removed those from my docker-compose.yml now but this getting the same issue.

they were both referencing python 3.8 though so perhaps not used? and now 2022.6 is on python 3.9 (and 2022.7 is on python 3.10 from what I read)

2022-07-10 12:59:38 ERROR (MainThread) [homeassistant.components.lock] Error while setting up kevo_plus platform for lock Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 249, in _async_setup_platform await asyncio.shield(task) File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/kevo_plus/lock.py", line 53, in setup_platform cookie = Kevo.Login(email, password) TypeError: Login() missing 1 required positional argument: 'password'

to me it seems to me the password is not being passed through from the config.... the only odd thing there is my password starts with a !

ie. password: "!aFakePassword"

although this was all working with gaggle331's version, before moving to your fork.

RXM307 commented 2 years ago

nope not ! in the password, I'll rollback and see if the original version is still working...

dcmeglio commented 2 years ago

It's not the password. Still learning python, but this is because the code isn't loading my new version of the kevo package, https://pypi.org/project/pykevoplusnew/

The reason it talks about password is because in my library login expects: def Login(username, password): (which is the component is calling)

Whereas the old version was: def Login(session, username, password):

For some reason, your machine won't download this pypi package and I'm not sure why. HA should automatically download pypi packages as I understand it. So it's either a cache of some kind, or something pointing the component to the old version still.

RXM307 commented 2 years ago

ok thanks for the hint, I can see these in the /config directory which look like they were the issue as they must have been preferenced over the python3.9/site-packages

/usr/local/lib/python3.9/site-packages/pykevoplusnew-3.0.0.dist-info /usr/local/lib/python3.9/site-packages/pykevoplus /config/pykevocontrol-2.0.0.dist-info /config/pykevoplus

I moved the /config ones and all seems to be working correctly now, I'll keep an eye on it, thanks for your work here and the support.

RXM307 commented 2 years ago

Looks like still cant lock/unlock on this new version but atleast these HTTP 500 errors are fixed.

2022-07-13 11:21:52 ERROR (MainThread) [homeassistant.core] Error executing service: <ServiceCall lock.unlock (c:01G7TJ6GVE7AWRQV0AT0DV3Y8G): entity_id=['lock.front_door']> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/core.py", line 1722, in catch_exceptions await coro_or_task File "/usr/src/homeassistant/homeassistant/core.py", line 1741, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service await service.entity_service_call( File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 680, in entity_service_call future.result() # pop exception if have File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 964, in async_request_call await coro File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 717, in _handle_entity_call await result File "/usr/src/homeassistant/homeassistant/components/lock/init.py", line 158, in async_unlock await self.hass.async_add_executor_job(ft.partial(self.unlock, kwargs)) File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "/config/custom_components/kevo_plus/lock.py", line 108, in unlock self._kevo.Unlock() File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 104, in _wrapped return method(self, args, kwargs) File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 284, in Unlock self.WaitForUnlocked() File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 266, in WaitForUnlocked self._WaitForState("unlocked", timeout) File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 104, in _wrapped return method(self, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/pykevoplus/init.py", line 221, in _WaitForState raise KevoError("Timeout waiting for {}".format(state.lower())) pykevoplus.KevoError: Timeout waiting for unlocked

but I guess this is because of the capta?

dcmeglio commented 2 years ago

Hmm no. Locking and unlocking should work fine. I’ll need to dig further. How many locks do you have configured?

RXM307 commented 2 years ago

Just the one Lock

RXM307 commented 1 year ago

@dcmeglio just upgraded and this is working needed to change the ekey to an admin ekey (would not unlock with an anytime ekey)