scanner / asimap

Pure python based IMAP server with a MH based file store
BSD 3-Clause "New" or "Revised" License
43 stars 7 forks source link

Vacuum of the sqlite db sometimes fails when called from aiosqlite (OperationalError, readonly db) #272

Closed scanner closed 4 months ago

scanner commented 4 months ago

Seeing some strange state where a mailbox has new mail, apple mail shows nothing, rebuild mailbox and mail shows up. We may be missing some notifies or something.

scanner commented 4 months ago

Okay, turns out the subprocess had died due to this exception. Need to check logic: either we need to make this an ignorable operation, or we are somehow opening more than one connection to the sqlite db?

This call seems to work at first, and then fail some time later.. do we open multiple connections to the db? Should not be doing that, right?

scanner commented 4 months ago
Traceback (most recent call last):
  File "/app/asimap/asimapd_user.py", line 129, in <module>
    main()
  File "/app/asimap/asimapd_user.py", line 118, in main
    asyncio.run(create_and_start_user_server(Path.cwd(), debug, trace))
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/app/asimap/asimapd_user.py", line 85, in create_and_start_user_server
    server = await IMAPUserServer.new(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/asimap/user_server.py", line 469, in new
    user_server.db = await Database.new(maildir)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/asimap/db.py", line 113, in new
    await db.execute("vacuum")
  File "/venv/lib/python3.12/site-packages/aioretry/retry.py", line 164, in wrapped
    return await perform(
           ^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/aioretry/retry.py", line 111, in perform
    raise e
  File "/venv/lib/python3.12/site-packages/aioretry/retry.py", line 97, in perform
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/asimap/db.py", line 224, in execute
    await self.conn.execute(sql, *args, **kwargs)
  File "/venv/lib/python3.12/site-packages/aiosqlite/core.py", line 193, in execute
    cursor = await self._execute(self._conn.execute, sql, parameters)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/aiosqlite/core.py", line 132, in _execute
    return await future
           ^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/aiosqlite/core.py", line 115, in run
    result = function()
             ^^^^^^^^^^
scanner commented 4 months ago

Is it trying to start up a new subprocess?

Maybe our code is not properly detecting when the subprocess goes away (due to idle timeout?)

scanner commented 4 months ago

Also note that this exception is appearing on the stderr/stdout of the docker-compose run container. Not in the log files for the server or user_user, so we have an uncaptured exception bubbling up to the top level. That should not be happening.