search

andim / easyleed

Software for Low Energy Electron Diffraction I(E)-spectra analysis
http://andim.github.io/easyleed/
Other
12 stars 4 forks source link

Parallellize the tracker #6

Open andim opened 7 years ago

andim commented 7 years ago

From @feranick on October 10, 2014 21:6

Either multiprocessing or multithreading. This should also help in solving issue feranick/easyleed/issues/3

Copied from original issue: feranick/easyleed#39

andim commented 7 years ago

From @feranick on October 10, 2014 21:7

See here: http://stackoverflow.com/questions/26303896/parallellize-for-loops-for-image-analysis-in-qt-python

andim commented 7 years ago

From @feranick on October 10, 2014 22:28

This is still very relevant during acquisition, not just for the slider.

mgrady3 commented 6 years ago

I thought I had chimed in on this earlier.

I went through a number of tests with parallelizing things in my Qt applications. The conclusions I came to we're that the only definite way to do so was to sidestep the Python GIL using one of two methods:

Use multiprocessing (not the Python threading library nor the Qthreads framework from Qt) to create multiple processes to execute tasks. This is a simple framework of you just have one or two tasks that you need to execute on a set of data that are effectively independent of each other, but gets complex if you need communication between the processes.

The other way is simply to leave the Python realm and use another language to do the parallel work. The easiest would be C/C++ of Fortran, however Go and Rust are becoming possibilities these days. This will allow your tasks to run free on the Python GIL, but comes with the downsides of having to use another language as well as complicating the distribution of your code to other users.

Neither of these solutions is necessarily easy, but that's just about the only option. Simple multithreading with Python threads or the Qthread framework won't work for cpu-bound tasks in python because the GIL prevents multiple threads from simultaneously executing code.

feranick commented 4 days ago

Python 3.13 is set to be the first release with no-GIL (at least in the experimental stage). We might want to consider multithreading once Python 3.13+ reaches critical mass adoption.