While fetching feeds and using MySQL with multiprocessing sometimes database returns errors like this:
Exception _mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; you can't run this command now") in <bound method Cursor.__del__ of <MySQLdb.cursors.Cursor object at 0x101826410>> ignored
This seems caused by fetcher processes sharing the same connection thus open cursors. The Typical solution is to let each spawn process should open/close its own Peewee database connection. Incidentally is what Bottle Fever does.
It seems to me that it's the closing phase that matters: after all the connection is already open by the starter process just before spawning the subprocesses. So by explicitly closing the connection helps to close open cursors.
passiomatic
commented
11 years ago
While fetching feeds and using MySQL with multiprocessing sometimes database returns errors like this:
This seems caused by fetcher processes sharing the same connection thus open cursors. The Typical solution is to let each spawn process should open/close its own Peewee database connection. Incidentally is what Bottle Fever does.
It seems to me that it's the closing phase that matters: after all the connection is already open by the starter process just before spawning the subprocesses. So by explicitly closing the connection helps to close open cursors.
This Peewee issue has more details: https://github.com/coleifer/peewee/issues/67