meeg-ml-benchmarks / brain-age-benchmark-paper

M/EEG brain age benchmark paper
https://meeg-ml-benchmarks.github.io/brain-age-benchmark-paper/
BSD 3-Clause "New" or "Revised" License
24 stars 10 forks source link

Incorrect handling of n_jobs > 1 in shallow / deep benchmark #19

Closed gemeinl closed 3 years ago

gemeinl commented 3 years ago

I just want to point out that this message (https://github.com/dengemann/meeg-brain-age-benchmark-paper/blob/main/compute_benchmark_age_prediction.py#L257-L259) is not how I intended. It is true that we do not use n_jobs to run several folds of the CV in parallel. However, this does not mean that you should put n_jobs to 1 when calling the script. The n_jobs is handed to the data loaders for train and validation set (https://github.com/dengemann/meeg-brain-age-benchmark-paper/blob/main/compute_benchmark_age_prediction.py#L228, https://github.com/dengemann/meeg-brain-age-benchmark-paper/blob/main/X_y_model.py#L340-L341) and could speed up computations (due to lazily loading the next batch of data for the GPU in the same amount of time that it does computations on the previous batch). See @hubertjb work here https://github.com/braindecode/braindecode/pull/75.