codegooglecom / cjklib

Automatically exported from code.google.com/p/cjklib
Other
0 stars 0 forks source link

dbconnector will crash in a multithreaded environment with sqlite (patch attached) #14

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. Use cjklib in any multithreaded environment (like a threaded webserver, for 
example Django)
2. Access a view that uses cjklib a few times (make sure it's a function call 
that hits the database)

What is the expected output? What do you see instead?

Instead of working, you get a backtrace. This is the relevant part:

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 244, in decompose
   decompositionParts = self.getDecompositionTree(readingString)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 299, in getDecompositionTree
   segmentations = self.segment(part)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 358, in segment
   segmentationTree = self._recursiveSegmentation(readingString)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 388, in _recursiveSegmentation
   readingString[0:substringIndex].lower()):

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 1650, in _hasEntitySubstring
   readingString)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 467, in _hasEntitySubstring
   return readingString in self._substringTable

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/util.py", line 631, in fget_wrapper
   value = fget(self)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 450, in _substringTable
   entities = self.getReadingEntities() | self.getFormattingEntities()

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/util.py", line 658, in oneshot
   result = self.fget(*args, **kwargs)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 1847, in getReadingEntities
   syllables = self.getPlainReadingEntities()

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/util.py", line 658, in oneshot
   result = self.fget(*args, **kwargs)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/reading/operator.py", line 1799, in getPlainReadingEntities
   select([self.db.tables['PinyinSyllables'].c.Pinyin])))

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/dbconnector.py", line 511, in selectScalars
   result = self.execute(request)

 File "/home/shengci/dev/lib/python2.6/site-packages/cjklib-0.3-py2.6.egg/cjklib/dbconnector.py", line 467, in execute
   return self.connection.execute(*options, **keywords)

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1358, in execute
   params)

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1491, in _execute_clauseelement
   compiled_sql, distilled_params

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1558, in _execute_context
   None, None)

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1721, in _handle_dbapi_exception
   self._autorollback()

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1295, in _autorollback
   self._rollback_impl()

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1215, in _rollback_impl
   self._handle_dbapi_exception(e, None, None, None, None)

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/base.py", line 1212, in _rollback_impl
   self.engine.dialect.do_rollback(self.connection)

 File "/home/shengci/dev/lib/python2.6/site-packages/SQLAlchemy-0.7.1-py2.6-linux-i686.egg/sqlalchemy/engine/default.py", line 294, in do_rollback
   connection.rollback()

ProgrammingError: (ProgrammingError) SQLite objects created in a thread can 
only be used in that same thread.The object was created in thread id 
-1218057328 and this is thread id -1234834544 None None

What version of the product are you using? On what operating system?

Version 0.3 on linux.

Please provide any additional information below.

The attached patch fixes the dbconnector to use threadlocal variables for the 
database connection.

Original issue reported on code.google.com by kwo...@gmail.com on 6 Jun 2011 at 12:36

Attachments: