gbook / nidb

NeuroInformatics Database
GNU General Public License v3.0
26 stars 8 forks source link

File IO module can encounter problem when multithreaded #96

Closed gbook closed 1 year ago

gbook commented 2 years ago

If more than one fileio module is running, they can interfere with each other, especially if the IO operations are run on the same subject or study, or serial operations on the same data object.

For example if a study is moved from subject A to subject B, the study will appear under subject B before all of the data is copied on disk. If the user moves that study to a new enrollment, within subject B, before the file copy is done, the final study will be incomplete.

Because of the range of possible file IO operations, it may not always be possible to ensure that a file io job exclusive access to subject/study/series/enrollment/project. Another possibility is to have a flag within each data object to indicate that is unsafe to operate on.

The simplest fix for now is to limit the fileio module to single threading because high throughput file IO is rarely needed.

gbook commented 1 year ago

fix is for fileio module to be only single threaded