search

I2PC / scipion

Scipion is an image processing framework to obtain 3D models of macromolecular complexes using Electron Microscopy (3DEM)
http://scipion.i2pc.es
Other
76 stars 47 forks source link

concurrency problems mapper #1697

Open rsanchezgarc opened 6 years ago

rsanchezgarc commented 6 years ago

I have checked that, when run with multiple threads, this line, included in Consensus particle picker protocol (when STEPS_PARALLEL is set) is causing problems due to sqlite cursor recurrence.

228: coordArray = np.asarray([x.getPosition() for x in coordinates.get().iterCoordinates( micrograph.getObjId())], dtype=float)

My guess about how to fix it is that the mapper object should have a different cursor for each query.

delarosatrevin commented 6 years ago

Sqlite3 does not support threads. So, all operations on a Set (or anything using a mapper) should be done in the main thread. This imply that you can not iterate over the input sets in the steps of protocols that run in parallel.

pconesa commented 6 years ago

is there a workaround? like copying the sqlite...

pconesa commented 6 years ago

https://www.sqlite.org/threadsafe.html