Unifi Protect fails to re-setup

holdestmade commented 4 months ago

The problem

When trying to install (clicking on configure when it is auto discovered), after inputting username and password, get an unknown error message and the error in logs (see below). If I put an incorrect password in, it says invalid credentials

What version of Home Assistant Core has the issue?

2024.3.0b5

What was the last working version of Home Assistant Core?

2024.2.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

UniFi Protect

Link to integration documentation on our website

https://rc.home-assistant.io/integrations/unifiprotect

Diagnostics information

Not available as won't install

Example YAML snippet

NA

Anything in the logs that might be useful for us?

Logger: aiohttp.server
Source: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:421
First occurred: 9:40:29 am (1 occurrences)
Last logged: 9:40:29 am

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 91, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 25, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 235, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 31, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 68, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 72, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 197, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
    result = await method(view, request, data, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 111, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 405, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 502, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/unifiprotect/config_flow.py", line 177, in async_step_discovery_confirm
    nvr_data, errors = await self._async_get_nvr_data(user_input)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/unifiprotect/config_flow.py", line 259, in _async_get_nvr_data
    bootstrap = await protect.get_bootstrap()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/api.py", line 1009, in get_bootstrap
    return Bootstrap.from_unifi_dict(**data, api=self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 120, in from_unifi_dict
    data = cls.unifi_dict_to_dict(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/bootstrap.py", line 224, in unifi_dict_to_dict
    return super().unifi_dict_to_dict(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 369, in unifi_dict_to_dict
    data[key] = cls._clean_protect_obj_dict(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 311, in _clean_protect_obj_dict
    items[key] = cls._clean_protect_obj(value, klass, api)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 289, in _clean_protect_obj
    return klass.unifi_dict_to_dict(data=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/user.py", line 72, in unifi_dict_to_dict
    return super().unifi_dict_to_dict(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 360, in unifi_dict_to_dict
    data[key] = cls._clean_protect_obj_list(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 300, in _clean_protect_obj_list
    items[index] = cls._clean_protect_obj(item, klass, api)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/base.py", line 289, in _clean_protect_obj
    return klass.unifi_dict_to_dict(data=data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyunifiprotect/data/user.py", line 34, in unifi_dict_to_dict
    data["nodes"] = [PermissionNode(n) for n in parts[1].split(",")]
                     ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 744, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 1158, in __new__
    raise ve_exc
ValueError: 'readlive' is not a valid PermissionNode

Additional information

Running Protect v2.11.21 Official

home-assistant[bot] commented 4 months ago

Hey there @angellusmortis, @bdraco, mind taking a look at this issue as it has been labeled with an integration (unifiprotect) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `unifiprotect` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign unifiprotect` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

unifiprotect documentation unifiprotect source _{^{(message by IssueLinks)}}

wmandra commented 4 months ago

@holdestmade are you sure you're running Protect 2.11.21? The error is showing the new "readlive" permission which was only recently added into the 3.0.10 EA version.

kudos commented 4 months ago

I had upgraded, then downgraded again and I have the same issue. It's already fixed upstream https://github.com/AngellusMortis/pyunifiprotect/commit/077e9c8120016e825d658cbbe1119971765557b0

holdestmade commented 4 months ago

Same here, I did upgrade to v3 but downgraded to 2.11.21 after it failed to start. I'll wait for the fix, thanks

kudos commented 4 months ago

I manually applied the patch on my install and it resolved the issue.

CoreyJ87 commented 4 months ago

I manually applied the patch on my install and it resolved the issue.

How? Im stuck with the same problem. Downgraded and it still shows that error

EDIT: Oh I assume you dont run HAOS? Just pulled their container?

Is there some way to pull those updates into HAOS or no?

kudos commented 4 months ago

I run HAOS, I docker exec'd a shell in the hass container and used vi to edit the file.

MuscleGeek69 commented 4 months ago

I am running HAOS and I have the same issue. I also upgraded and downgraded. Thanks

CoreyJ87 commented 4 months ago

I run HAOS, I docker exec'd a shell in the hass container and used vi to edit the file.

You have core then? In HAOS I dont have those files as its built into hass itself. Or is still on the filesystem somewhere? Where was the file?

holdestmade commented 4 months ago

If we are lucky it might make it into ha 2024.3 tomorrow ? (or a subsequent rev 2024.3.x)

AngellusMortis commented 4 months ago

Upgrading and then downgrading is still not supported. Do not use EA.

AngellusMortis commented 4 months ago

@home-assistant close

AngellusMortis commented 4 months ago

And it will not be in the release tomorrow. The new EA version dropped on a Monday morning right before the beta ended.

HA will not be getting a update PR until at least this weekend because I do not have time to upgrade my Protect instance this week to test everything and add support.

CoreyJ87 commented 4 months ago

Upgrading and then downgrading is still not supported. Do not use EA.

Well we know that now. We didnt know that before. Mine just auto updated because I had changed it to EA to update the network app. Nothing we can do to get around it? Guess this is my sign to stop using HAOS and goto core.

AngellusMortis commented 4 months ago

Restore a 2.x backup. Otherwise no. The docs are very explicit about not using EA versions if you depend on HA and do not want it to break.

This is essentialy my personal open source project. That means I work on it when I want and in my time. UniFi does not have an open API and does not give me any kind of special access so I cannot fix things before others have access to them.

CoreyJ87 commented 4 months ago

Restore a 2.x backup. Otherwise no. The docs are very explicit about not using EA versions if you depend on HA and do not want it to break.

This is essentialy my personal open source project. That means I work on it when I want and in my time. UniFi does not have an open API and does not give me any kind of special access so I cannot fix things before others have access to them.

I did restore a 2 backup. thats what i mean.

Edit: well I guess I downgraded. thats not really the same

I understand that you cant do EA stuff. I dont think thats what anyone was asking here. We were all looking for a temp workaround or fix.

kudos commented 4 months ago

@CoreyJ87 it's not difficult to fix it for yourself. Figure out how to get a shell inside the homeassistant Docker container. You can use something like docker exec -it homeassistant bash on a HAOS shell. Then do something like vi /usr/local/lib/python3.12/site-packages/pyunifiprotect/data/types.py and bang in the fix from https://github.com/AngellusMortis/pyunifiprotect/pull/354.

CoreyJ87 commented 4 months ago

@CoreyJ87 it's not difficult to fix it for yourself. Figure out how to get a shell inside the homeassistant Docker container. You can use something like docker exec -it homeassistant bash on a HAOS shell. Then do something like vi /usr/local/lib/python3.12/site-packages/pyunifiprotect/data/types.py and bang in the fix from AngellusMortis/pyunifiprotect#354.

I know how to exec into a docker container. Thats not the problem. I dont have a docker container. Im running HAOS. Which is not core. Which is what you're running. HAOS standalone(non docker version) does not have the files you have. If it does. Its in some other place that I dont know of.

kudos commented 4 months ago

➜  ~ uname -a
Linux a0d7b954-ssh 6.1.73-haos-raspi #1 SMP PREEMPT Mon Feb 26 10:59:01 UTC 2024 aarch64 Linux

AngellusMortis commented 4 months ago

If you want to discuss doing unsupported shit, I recommend doing it somewhere else. EA is not supported. Period. If you do not want your HA to break, do not use it. Encouraging people to use hacks and workaround to make it work just add more support burden and spam on me. Do it somewhere else.

kudos commented 4 months ago

I was trying to get the guy off your back, but I understand where you're coming from :+1:

CoreyJ87 commented 4 months ago

If you want to discuss doing unsupported shit, I recommend doing it somewhere else. EA is not supported. Period. If you do not want your HA to break, do not use it. Encouraging people to use hacks and workaround to make it work just add more support burden and spam on me. Do it somewhere else.

Sorry, I didnt know it was continuing to ping you. Also, it broke accidentally cus we didnt know that it was unsupported(our own fault not yours). It also auto updated. So it was random. Jesus christ chill. Were not all just running EA against better advice. We ended up here because of outside influences. We tried to downgrade.

and @kudos thats in the docker container version only. Those files dont exist in the standalone one. Or they are placed somewhere else. but w/e im just moving to the docker container version

ngolf commented 4 months ago

I've had the same issue (using HAOS on raspberry pi). Upgraded (automatically) to EA then downgraded (From what I remember, by recovering an old backup of my dream machine - but this was 2 weeks ago). I've since disabled all EA channels / auto-update.

When is the fix referred in https://github.com/AngellusMortis/pyunifiprotect/pull/354 expected to make it into HA Core?