zodb / relstorage

A backend for ZODB that stores pickles in a relational database.
Other
55 stars 46 forks source link

Fix sqlite+gevent deadlocks #384

Closed jamadden closed 4 years ago

jamadden commented 4 years ago

The problem happened if we switched away from a connection holding locks and into one that wanted to take those same locks, and wanted to do so without switching. This could be non-deterministic (or rather, very data specific because of how interval switching is done).

The solution was to move those operations that take locks into gevent's threadpool so they're guaranteed not to block the event loop. Fixes #382.

Add tests for this scenario. Also, CI wasn't properly testing gevent monkey-patching because of a bug in gevent.monkey. Workaround that and fix some small issues that the gevent.monkey bug was hiding (around mixing of real native threads and greenlets) and a couple of logging issues I encountered while debugging.