search

navis-org / pymaid

Python library to interface with CATMAID servers. Fully interoperable with navis.
https://pymaid.readthedocs.io/en/latest/
GNU General Public License v3.0
23 stars 11 forks source link

Make clustering methods use parallel threads #22

Open schlegelp opened 7 years ago

schlegelp commented 7 years ago

Just copy paste code from get_urls_threaded

schlegelp commented 7 years ago

Pool class from multiprocessing seems to be the better choice here (see https://docs.python.org/3.5/library/multiprocessing.html#multiprocessing.pool.Pool) as Threading only really improves performance for non-CPU-heavy tasks (parallelism).

schlegelp commented 7 years ago

http://sebastianraschka.com/Articles/2014_multiprocessing.html#multi-threading-vs-multi-processing

schlegelp commented 7 years ago

https://stackoverflow.com/questions/41920124/multiprocessing-use-tqdm-to-display-a-progress-bar

schlegelp commented 7 years ago

Quick initial tests show 2-10fold increase in speed for e.g. reroot, pruning and clustering

schlegelp commented 7 years ago

Partially implemented with commit 46f933ce7f0e97738820efa37d79b5a144da4af2