sqlite3.OperationalError: disk I/O error - Likely due to lack of disk space

[begincalendar]

Describe the bug Noticed messages weren't flowing through this proxy and found this in the log file:

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3144, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.OperationalError: disk I/O error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/python3.8/dist-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/dist-packages/pantalaimon/daemon.py", line 773, in sync
    json_response = await self.decrypt_body(client, json_response)
  File "/usr/local/lib/python3.8/dist-packages/pantalaimon/daemon.py", line 732, in decrypt_body
    return await asyncio.wait_for(
  File "/usr/lib/python3.8/asyncio/tasks.py", line 483, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.8/dist-packages/pantalaimon/daemon.py", line 725, in decrypt_loop
    return decryption_method(body, ignore_failures=False)
  File "/usr/local/lib/python3.8/dist-packages/pantalaimon/client.py", line 906, in decrypt_sync_body
    self.handle_to_device_from_sync_body(body)
  File "/usr/local/lib/python3.8/dist-packages/pantalaimon/client.py", line 893, in handle_to_device_from_sync_body
    self.olm.handle_to_device_event(event)
  File "/usr/local/lib/python3.8/dist-packages/nio/crypto/olm_machine.py", line 404, in handle_to_device_event
    decrypted_event = self.decrypt_event(event)
  File "/usr/local/lib/python3.8/dist-packages/nio/crypto/olm_machine.py", line 1644, in decrypt_event
    return self.decrypt(event.sender, event.sender_key, message)
  File "/usr/local/lib/python3.8/dist-packages/nio/crypto/olm_machine.py", line 1664, in decrypt
    plaintext = self._try_decrypt(sender, sender_key, message)
  File "/usr/local/lib/python3.8/dist-packages/nio/crypto/olm_machine.py", line 1226, in _try_decrypt
    self.save_session(sender_key, session)
  File "/usr/local/lib/python3.8/dist-packages/nio/crypto/olm_machine.py", line 1990, in save_session
    self.store.save_session(curve_key, session)
  File "/usr/local/lib/python3.8/dist-packages/nio/store/database.py", line 40, in inner
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/nio/store/database.py", line 235, in save_session
    OlmSessions.replace(
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1907, in inner
    return method(self, database, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1978, in execute
    return self._execute(database)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 2745, in _execute
    return super(Insert, self)._execute(database)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 2474, in _execute
    cursor = database.execute(self)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3157, in execute
    return self.execute_sql(sql, params, commit=commit)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3151, in execute_sql
    self.commit()
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 2917, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 190, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3144, in execute_sql
    cursor.execute(sql, params or ())
peewee.OperationalError: disk I/O error

To Reproduce Most likely: Running out of disk space.

Expected behavior Exception to be handled more gracefully (if possible).

Additional context Running on a Raspberry Pi 4B with a microSD card (so it could be that this is at fault), but the card is an endurance model (e.g. suitable for dash camera usage).

[begincalendar]

Pretty confident this was caused by running out of disk space. I've reclaimed some free space and it's working now, but I had to delete the SQLite database (presumably because it got corrupted).

I'll leave it up to the maintainers to decide whether there is anything here that is worth actioning.

[poljar]

Usually there's not much you can do if there's an IO error, either the disk is full or it's malfunctioning. Not sure why the database would be corrupted, inspecting it with the sqlite CLI tool might give more insight. Sqlite has a journal which should protect against IO errors.

[begincalendar]

To be more specific, the application would run with the database afterwards, but the application wasn't functioning as it normally does:

• Wasn't processing incoming messages properly.
• Wasn't accepting new logins.

There weren't database errors in the log, there were session encryption errors. I had to delete the database and start the application again for it to become functional again.

I apologise for not keeping the database file and the log files, but I needed to move quickly and was doing too many things at once.

I guess if you think it's worth trying to reproduce with a tight disk quota, then it might uncover

[poljar]

I won't really have time to look into this in any detailed capacity, at least not for now, so I don't want to waste your time.

Let's leave this issue open and if someone else hits it we might get more details.