aramis-lab / clinica

Software platform for clinical neuroimaging studies
http://www.clinica.run/
Other
218 stars 72 forks source link

Enable debugging for converters using multiprocessing #1183

Closed NicolasGensollen closed 1 month ago

NicolasGensollen commented 1 month ago

Some of our converters like ADNI-to-BIDS use multiprocessing when converting images using a Pool of workers where the number of workers is set to n_procs:

https://github.com/aramis-lab/clinica/blob/7f7a0b412caa4a5166bdded5467cb5aed3eeebbe/clinica/iotools/converters/adni_to_bids/adni_utils.py#L1299-L1306

When setting n_procs to one, this still creates a Pool with one worker which prevent using the classical Python debugger for example.

I believe we should only create the Pool of workers when n_jprocs >= 2, and run the conversion sequentially when n_procs==1:

create_file_ = partial(
    create_file,
    modality=modality,
    bids_dir=bids_dir,
    mod_to_update=mod_to_update,
)
output_file_treated = [create_file_(image) for image in images_list]

This wouldn't change the user experience, but would greatly improve the developer experience when debugging image conversions.