search

mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.74k stars 1.33k forks source link

Ignore `n_jobs` in config if `--subject` flag is passed #12965

Closed berkgercek closed 1 week ago

berkgercek commented 1 week ago

Describe the new feature or enhancement

Currently when n_jobs is specified in the config file and is not 1, processes are restricted to use only 1 core even when the command mne_bids_pipeline is called with the --subject MySub flag, and only one subject is being processed. It would be nice if, should the command-level override be passed, the n_jobs parameter would also be ignored and a single python process run that has access to all available cores.

Describe your proposed implementation

mne_bids_pipeline --config myconf.py should respect the n_jobs: int parameter in myconf.py

mne_bids_pipeline --config myconf.py --subject MySub --session 01 --task MyTask should ignore the n_jobs parameter and implicitly set n_jobs = 1

possibly: mne_bids_pipeline --config myconf.py --subject MySub --task MyTask should respect the n_jobs parameter to process sessions in parallel, or tasks in parallel should there be one session specified and multiple possible tasks. My use case only has one session and task per subject so I'm not sure of the mechanics here.

Describe possible alternatives

The current solution is to also set the flag --n_jobs 1 when calling with a single subject/session/task, which is more explicit but causes possible confusion when omitted as the pipeline will only use a single core despite processing a single datum.

berkgercek commented 1 week ago

Apologies, this was intended for mne-bids-pipeline, not this repo.