BrokenProcessPool with compute_covariance on loaded epochs

Description of the problem

The cross-validation step to select the best estimator fails in joblib. Setting n_jobs=1 does by-pass the problem, but it would still be interesting to figure out what is going wrong here.

Traceback

``` _RemoteTraceback: """ Traceback (most recent call last): File "/home/scheltie/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/externals/loky/process_executor.py", line 426, in _process_worker call_item = call_queue.get(block=True, timeout=timeout) File "/usr/lib/python3.10/multiprocessing/queues.py", line 122, in get return _ForkingPickler.loads(res) File "/home/scheltie/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/numpy_pickle.py", line 600, in load_temporary_memmap add_maybe_unlink_finalizer(obj) File "/home/scheltie/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/_memmapping_reducer.py", line 72, in add_maybe_unlink_finalizer "".format(type(memmap), id(memmap), os.path.basename(memmap.filename), File "/usr/lib/python3.10/posixpath.py", line 142, in basename p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType """ The above exception was the direct cause of the following exception: Traceback (most recent call last): Cell In[2], line 5 cov = compute_covariance( File :12 in compute_covariance File ~/git/mscheltienne/mne-python/mne/cov.py:1161 in compute_covariance cov_data = _compute_covariance_auto( File ~/git/mscheltienne/mne-python/mne/cov.py:1386 in _compute_covariance_auto loglik = _cross_val(data, estimator, cv, n_jobs) File ~/git/mscheltienne/mne-python/mne/cov.py:1419 in _cross_val cross_val_score( File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/sklearn/model_selection/_validation.py:562 in cross_val_score cv_results = cross_validate( File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:211 in wrapper return func(*args, **kwargs) File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/sklearn/model_selection/_validation.py:309 in cross_validate results = parallel( File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/sklearn/utils/parallel.py:65 in __call__ return super().__call__(iterable_with_config) File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:1952 in __call__ return output if self.return_generator else list(output) File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:1595 in _get_outputs yield from self._retrieve() File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:1699 in _retrieve self._raise_error_fast() File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:1734 in _raise_error_fast error_job.get_result(self.timeout) File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:736 in get_result return self._return_or_raise() File ~/pyvenv/mscheltienne/mne-python/lib/python3.10/site-packages/joblib/parallel.py:754 in _return_or_raise raise self._result BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable. ```

Steps to reproduce

from mne import compute_covariance, read_epochs

epochs = read_epochs("/home/scheltie/Downloads/short-epo.fif")
cov = compute_covariance(
    epochs,
    tmin=None,
    tmax=0,
    method="auto",  # ('shrunk', 'diagonal_fixed', 'empirical', 'factor_analysis')
    cv=3,
    n_jobs=6,
)

Interestingly, I can not reproduce with the sample dataset..

Link to data

short-epo.zip

Additional information

Fresh environment with MNE-main.

mne.sys_info()

``` Platform Linux-6.4.6-76060406-generic-x86_64-with-glibc2.35 Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] Executable /home/scheltie/pyvenv/mscheltienne/mne-python/bin/python CPU x86_64 (12 cores) Memory 31.0 GB Core ├☑ mne 1.6.0.dev126+gf3bb56d1d ├☑ numpy 1.26.0 (OpenBLAS 0.3.23.dev with 12 threads) ├☑ scipy 1.11.3 ├☑ matplotlib 3.8.0 (backend=Qt5Agg) ├☑ pooch 1.7.0 └☑ jinja2 3.1.2 Numerical (optional) ├☑ sklearn 1.3.1 ├☑ nibabel 5.1.0 ├☑ nilearn 0.10.1 ├☑ dipy 1.7.0 ├☑ pandas 2.1.1 └☐ unavailable numba, openmeeg, cupy Visualization (optional) ├☑ pyvista 0.42.2 (OpenGL 4.6 (Core Profile) Mesa 23.1.3-1pop0~1689084530~22.04~0618746 via Mesa Intel(R) UHD Graphics 770 (ADL-S GT1)) ├☑ pyvistaqt 0.11.0 ├☑ vtk 9.2.6 ├☑ qtpy 2.4.0 (PyQt5=5.15.2) └☐ unavailable ipympl, pyqtgraph, mne-qt-browser, ipywidgets, trame_client, trame_server, trame_vtk, trame_vuetify Ecosystem (optional) ├☑ mne-bids 0.14.dev0 ├☑ mne-connectivity 0.6.0dev0 └☐ unavailable mne-nirs, mne-features, mne-icalabel, mne-bids-pipeline ```

mne-tools / mne-python