added multiprocessing option for prepare subfunctions - Githubissues

frankkramer-lab / MIScnn

A framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning

GNU General Public License v3.0

403 stars 116 forks source link

added multiprocessing option for prepare subfunctions #39

Closed MLRadfys closed 4 years ago

MLRadfys commented 4 years ago

Added an option for parallel preprocessing when prepare_subfunctions is set to True. Multipreprocessing can be activated by initializing the a Preprocessor with use_mp_preprocessing = True. Furthermore a new class variable has been added in the Preprocessor class in order to choose the number of workers used for multiprocessing.

Cheers,

Michael

muellerdo commented 4 years ago

Pull Request: Accepted

Status: Merged in branch Multiprocessing-Subfunctions https://github.com/frankkramer-lab/MIScnn/tree/Multiprocessing-Subfunctions

To-do:

[X] Code Review
[X] Unittesting
[X] Documentation

Update:

Renamed the Preprocessor parameter to "use_multiprocessing" in order to be synchronic to the Keras variable naming.
Renamed the Preprocessor class variable from mp_cpus to mp_threads.
Little code enhancement in run_subfunctions function in Preprocessor class
Updated the Preprocessor wiki page
Added unittesting for multiprocessed subfunction preprocessing
Merged branch Multiprocessing-Subfunctions into the dev branch
Archived branch Multiprocessing-Subfunctions

Will be available in the master branch/PyPI version in the next days.

Comments

Hey @MichaelLempart,

it works perfectly. Thank you for this great contribution!

Cheers, Dominik