search

piskvorky / sqlitedict

Persistent dict, backed by sqlite3 and pickle, multithread-safe.
Apache License 2.0
1.16k stars 130 forks source link

When using multiprocessing I get database is locked #144

Closed larytet closed 2 years ago

larytet commented 2 years ago

I have a pool of processes reading and writing the same table. Write can be occasionally slow. May be 1-2s. What am I missing?

File "/home/user/.local/lib/python3.8/site-packages/sqlite_object/_sqlite_dict.py",
line 31, in __init__
    super(SqliteDict, self).__init__(self.__schema, self.__index,
filename or str(uuid.uuid4())+".sqlite3", coder, decoder, index=index,
persist=persist, commit_every=commit_every)
  File "/home/user/.local/lib/python3.8/site-packages/sqlite_object/_sqlite_object.py",
line 21, in __init__
    cursor.execute(schema)
sqlite3.OperationalError: database is locked
mpenkov commented 2 years ago

I don't think sqlite3 supports concurrent reads/writes.

larytet commented 2 years ago

If I add my own locks will it solve the problem?

mpenkov commented 2 years ago

Maybe. You could connect to the DB before the critical section, do the IO work, and then disconnect at the end of the section. Sqlite wasn't really built for this sort of use, though.