search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70.04k stars 29.12k forks source link

Problems with Skyconnect installation #116545

Open mervyngriffiths opened 2 months ago

mervyngriffiths commented 2 months ago

The problem

I want to reinstall SkyConnect. I connect the SkyConnect Thumb Drive and press Submit. the message: 'Unknown error occurred' is returned.

I have attached the Log file entry.

What version of Home Assistant Core has the issue?

2024.4.4

What was the last working version of Home Assistant Core?

2024.4.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Home Assistant SkyConnect

Link to integration documentation on our website

https://www.home-assistant.io/integrations/homeassistant_sky_connect/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: aiohttp.server
Source: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:421
First occurred: 18:45:28 (7 occurrences)
Last logged: 19:23:56

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 71, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 222, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper
    result = await method(view, request, data, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 356, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 402, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 506, in _async_handle_step
    result: _FlowResultT = await getattr(flow, method)(user_input)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 576, in async_step_confirm
    return await self.async_step_verify_radio()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 326, in async_step_verify_radio
    return await self.async_step_choose_formation_strategy()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/config_flow.py", line 342, in async_step_choose_formation_strategy
    await self._radio_mgr.async_load_network_settings()
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 250, in async_load_network_settings
    async with self.connect_zigpy_app() as app:
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 182, in connect_zigpy_app
    app = await self.radio_type.controller.new(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 252, in new
    await app._load_db()
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 107, in _load_db
    self._dblistener = await zigpy.appdb.PersistingListener.new(database_file, self)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/appdb.py", line 150, in new
    await listener.initialize_tables()
  File "/usr/local/lib/python3.12/site-packages/zigpy/appdb.py", line 121, in initialize_tables
    rows = await cursor.fetchall()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiosqlite/cursor.py", line 76, in fetchall
    return await self._execute(self._cursor.fetchall)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiosqlite/cursor.py", line 40, in _execute
    return await self._conn._execute(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiosqlite/core.py", line 132, in _execute
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiosqlite/core.py", line 115, in run
    result = function()
             ^^^^^^^^^^
sqlite3.DatabaseError: database disk image is malformed

Additional information

No response

home-assistant[bot] commented 2 months ago

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (homeassistant_sky_connect) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `homeassistant_sky_connect` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign homeassistant_sky_connect` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)

homeassistant_sky_connect documentation homeassistant_sky_connect source (message by IssueLinks)

home-assistant[bot] commented 2 months ago

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `zha` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zha` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)

zha documentation zha source (message by IssueLinks)

TheJulianJES commented 2 months ago

sqlite3.DatabaseError: database disk image is malformed

Looks like your zigpy/ZHA database file is corrupted (zigbee.db).

mervyngriffiths commented 2 months ago

Hi there,

So what can I do? The thumb drive is not recognised on a Mac. Do I need a new drive? Where do I get that from?

Regards

Charles Griffiths

On 1 May 2024, at 19:55, TheJulianJES @.***> wrote:

sqlite3.DatabaseError: database disk image is malformed

Looks like your zigpy/ZHA database file is corrupted (zigbee.db).

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/116545#issuecomment-2088842636, or unsubscribe https://github.com/notifications/unsubscribe-auth/A33WXA2GE4ZDHFEEO6KAOVDZAET73AVCNFSM6AAAAABHCKCLPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBYHA2DENRTGY. You are receiving this because you authored the thread.

mervyngriffiths commented 2 months ago

Hi there,

I have attached a copy of my Zigbee.db

Regards

CG

On 1 May 2024, at 19:55, TheJulianJES @.***> wrote:

sqlite3.DatabaseError: database disk image is malformed

Looks like your zigpy/ZHA database file is corrupted (zigbee.db).

— Reply to this email directly, view it on GitHubhttps://github.com/home-assistant/core/issues/116545#issuecomment-2088842636, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A33WXA2GE4ZDHFEEO6KAOVDZAET73AVCNFSM6AAAAABHCKCLPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBYHA2DENRTGY. You are receiving this because you authored the thread.Message ID: @.***>

puddly commented 2 months ago

@mervyngriffiths GitHub doesn't accept file uploads over email. Can you attach it to a comment?

mervyngriffiths commented 2 months ago

I have attached the Zigbee.db. It looks ok structurally as it can be read by a Sqlite app, so I guess there is a problem with the data. Changed the extension to .pdf so GitHub will accept it.

I have got past the problem by deleting and recreating the Zigbee.db as follows

  1. Stop HAOS
  2. 'mv zigbee.db Zigbee-corrupt.db'
  3. Restart HAOS Now everything seems OK.

zigbee.pdf

puddly commented 2 months ago

The database has a few issues:

sqlite> pragma integrity_check;
*** in database main ***
Tree 29 page 29: btreeInitPage() returns error code 11
Tree 28 page 28: btreeInitPage() returns error code 11
Runtime error: database disk image is malformed (11)

You can try to recover it with zigpy-cli, which seems to run without errors:

pip install zigpy-cli
zigpy db recover zigbee.db zigbee.db.recovered
mervyngriffiths commented 2 months ago

The database initialisation seemed to work OK. I don't have many Zigbee devices attached so they were easily recovered. The corruption occurred when I was implementing new stuff on HAOS and was restarting a lot. Could this have caused a possible corruption?

puddly commented 2 months ago

SQLite should be very immune to corruption. Like it shouldn't even be possible under normal conditions.

How are you running Home Assistant OS? In a VM?

mervyngriffiths commented 2 months ago

It's on a Raspberry Pi running HAOS. Don't know enough about the PI to see if there were any possible hardware glitches, or Linux related errors.

TH3xR34P3R commented 2 months ago

I have a VM based install via Virtualbox, it is seeing the skyconnect in the hardware list but the ZHA, Matter and Thread integrations are stuck in a failed to setup loop with the following build installed:

Core 2024.5.0 Supervisor 2024.04.4 Operating System 12.3.rc2 Frontend 20240501.0

Issue only started since Core 2024.5.0 and OS 12.3.rc2 was installed. Database issue may be related but not sure how to access to test the possible fix listed in the above thread.

edit: Reverting back to the following build has restored ZHA and Thread integrations from being stuck in the failed to setup loop, now just waiting for the fix to Matter integration to start working again.

Core 2024.5.0 Supervisor 2024.04.4 Operating System 12.2 Frontend 20240501.0

edit2; So far with the following version instances everything is running as it should be, I had to remove via list without pairing them again and let ZHA auto discover and readd the devices for them to start working again after reverting to an older snapshot.

Core 2024.5.1 Supervisor 2024.04.4 Operating System 12.2 Frontend 20240501.0

mervyngriffiths commented 2 months ago

If we discount the SQLite software, the Linux kernel and the hardware from causing the error then that pretty much leave the HAOS Core or the Zigbee handler as the possible culprit.

As I said earlier the problem occurred during a period of multiple restarts of HAOS. Could this possibly point to deficiencies in any SQL INSERTS, UPDATE, RELEASE implementations possibly running without START TRANSACTION/COMMIT functionality?

TH3xR34P3R commented 2 months ago

Only the developers of the code can speak on what may of been missing or if there was a typo between these releases; but as stated its all working now.

It's akin to needing to go into the hass agent intergration and adding some missing items in the script as per https://github.com/hass-agent/HASS.Agent-Integration/issues/2 to get the media playback control working for the client install again.

puddly commented 2 months ago

There really shouldn't be a way to corrupt an SQLite database without something going seriously wrong: https://www.sqlite.org/howtocorrupt.html. Interrupting, crashing, restarting, and bad SQL should not leave the database in a corrupted state.