search

dwavesystems / dwave-cloud-client

A minimal implementation of the REST interface used to communicate with D-Wave Solver API (SAPI) servers.
https://docs.ocean.dwavesys.com/projects/cloud-client/en/stable/
Apache License 2.0
59 stars 40 forks source link

Regions cache (sqlite3) is not thread-safe #621

Closed randomir closed 1 month ago

randomir commented 3 months ago

When accessing regions cache from multiple threads (dwave.cloud.regions.get_regions), the following errors have been reported:

  ...
  File "/.../lib/python3.10/site-packages/dwave/cloud/regions.py", line 37, in <module>
    @cached.ondisk(maxage=REGIONS_CACHE_MAXAGE, key='cache_key', bucket='regions')
  File "/.../lib/python3.10/site-packages/dwave/cloud/utils.py", line 642, in ondisk
    cache = diskcache.Cache(disk=diskcache.JSONDisk, directory=directory,
  File "/.../lib/python3.10/site-packages/diskcache/core.py", line 500, in _init
    self.reset(key, value)
  File "/.../lib/python3.10/site-packages/diskcache/core.py", line 2409, in reset
    sql_retry(statement, (value, key))
  File "/.../lib/python3.10/site-packages/diskcache/core.py", line 666, in _execute_with_retry
    return sql(statement, args, *kwargs)
sqlite3.DatabaseError: database disk image is malformed

or

  ...
  File "/.../lib/python3.10/site-packages/dwave/cloud/regions.py", line 37, in <module>
    @cached.ondisk(maxage=_REGIONS_CACHE_MAXAGE, key='cache_key', bucket='regions')
  File "/.../lib/python3.10/site-packages/dwave/cloud/utils.py", line 642, in ondisk
    cache = diskcache.Cache(disk=diskcache.JSONDisk, directory=directory,
  File "/.../lib/python3.10/site-packages/diskcache/core.py", line 499, in __init__
    sql(query, (key, value))
  File "/.../lib/python3.10/site-packages/diskcache/core.py", line 666, in _execute_with_retry
    return sql(statement, *args, **kwargs)
sqlite3.OperationalError: database is locked
randomir commented 1 month ago

sqlite3.threadsafety in Python 3.9:

“Threads may share the module, but not connections.”

randomir commented 1 month ago

diskcache claims to be thread-safe and process-safe, but there's this.

randomir commented 1 month ago

Starting with Python 3.11, sqlite3.threadsafety attribute has been fixed to dynamically (correctly) reflect tread-safety of the underlying sqlite3 C module (a compile-time setting).

By default sqlite3 module is compiled in "serialized mode", meaning that "SQLite can be safely used by multiple threads with no restriction," and in particular:

Threads may share the module, connections and cursors

Given the above, I'll close this issue, and refer users to:

  1. Upgrade to Python 3.11+
  2. Check that their SQLite distribution has sqlite3.threadsafety set to 3 / serialized mode.

We can re-open upon reproduction in SQLite serialized mode. (Or open issues on sqlite/diskcache repos).