search

devsnd / cherrymusic

Stream your own music collection to all your devices! The easy to use free and open-source music streaming server.
http://www.fomori.org/cherrymusic
GNU General Public License v3.0
1.03k stars 187 forks source link

PyPy: commits fail when cursors not closed #694

Open tilboerner opened 7 years ago

tilboerner commented 7 years ago

The underlying problem which lead to #676 is actually not related to Python 2 as @MaxLeiter already noted, but to PyPy. Unfortunately, it's a bug that causes database commits to fail, and probably results in 500 errors to HTTP requests that involve persisting data. PyPy support is broken because of this.

It's caused by a combination of implicitly creating db cursors through connection.execute..., and trusting them to be implicitly closed by garbage collection by the time we want to commit. Unlike CPython, PyPy uses a different garbage collection strategy than refcounting, and does not collect objects right when their last pointer is removed. Therefore, cursors remain open, and sqlite sees their statements as not finalized, which means it will refuse to commit, and possibly also to rollback, with the following error:

OperationalError: cannot commit transaction - SQL statements in progress

You can see this happen in the failing sqlitecache unit tests on travis. I could also reproduce it locally.

I made the tests work by explicitly closing every cursor created in sqlitecache.py. To fix the bug, we need to make sure that db cursors are closed in all code that touches the database.

tilboerner commented 7 years ago

Here's the commit that fixes the issue in sqlitecache.py as an example: aa4eea. (The formatting is borked, but in this legacy code that's hardly noticeable. =:0 Ay.)