Deadlock on exception when opening connection

piskvorky / sqlitedict

Persistent dict, backed by sqlite3 and pickle, multithread-safe.

Apache License 2.0

1.16k stars 130 forks source link

Deadlock on exception when opening connection #90

Open arnauorriols opened 5 years ago

arnauorriols commented 5 years ago

When SqliteDict is initialized, a connection to SQLite is opened (in here). This connection might fail.

If an exception happens, the connection does not get opened. However, the initialization still tries to create the table. In the commit of that operation, the execution is deadlocked, because the code keeps waiting infinitely in the req Queue.

menshikh-iv commented 5 years ago

Hi @arnauorriols, can you add a code example that reproduces this error (so we can fix it)?

bluenote10 commented 5 years ago

I think I'm having the same issue. Steps to reproduce:

create write protected folder.
try to open a non-existing DB in the folder sqlitedict.SqliteDict("/tmp/write_protected/non_existing.sqlite", tablename="test", autocommit=True).

This crashes internally (traceback visible on stdout) but never returns, looking like the mentioned deadlocked.

arnauorriols commented 5 years ago

Hi @arnauorriols, can you add a code example that reproduces this error (so we can fix it)?

I've opened #97 proposing a fix.

padelt commented 4 years ago

@arnauorriols realized that #97 has a race condition. #107 is my fresh approach that solves the undetected failure to open the connection (on first command exection) which currently leads to a deadlock in the calling code. It also avoids the race condition when the thread's run() is slower to set self.exception than the calling code executes a command (including a then false negative exception check) and then deadlocking on the answer.