MNE-BIDS-Pipeline maxwell filter reference run error

[allermat]

https://mne.discourse.group/t/mne-bids-pipeline-maxwell-filter-reference-run-error/7030

Hi,

I am getting the below error whenever I set a run number different than '01' in the maxwell filter reference run parameter (mf_reference_run) in mne-bids-pipeline.

[16:19:22] │ ⏳️ preprocessing/_01_data_quality sub-02 run-02 Finding flat channels and noisy channels using Maxwell filtering.
Opening raw data file /imaging/davis/Projects/SpeechMisperceptionMEEG/MNE-BIDS_data/sub-02/meg/sub-02_task-main_run-02_meg.fif...
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/ma09/.conda/envs/my_mne1.4/lib/python3.11/site-packages/mne_bids_pipeline/_run.py", line 54, in wrapper
    out = memory.cache(func)(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma09/.conda/envs/my_mne1.4/lib/python3.11/site-packages/mne_bids_pipeline/_run.py", line 147, in wrapper
    in_files = kwargs["in_files"] = self.get_input_fnames(**these_kwargs)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma09/.conda/envs/my_mne1.4/lib/python3.11/site-packages/mne_bids_pipeline/steps/preprocessing/_01_data_quality.py", line 46, in get_input_fnames_data_quality
    in_files = _get_raw_paths(
               ^^^^^^^^^^^^^^^
  File "/home/ma09/.conda/envs/my_mne1.4/lib/python3.11/site-packages/mne_bids_pipeline/_import_data.py", line 564, in _get_raw_paths
    raw_fname = _read_json(_empty_room_match_path(bids_path_in, cfg))[
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma09/.conda/envs/my_mne1.4/lib/python3.11/site-packages/mne_bids_pipeline/_io.py", line 17, in _read_json
    with open(fname, "r", encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/imaging/davis/Projects/SpeechMisperceptionMEEG/data/derivatives/mne-bids-pipeline/sub-01/meg/sub-01_task-main_run-01_emptyroommatch.json'

So the pipeline is looking for a file named 'sub-01_task-main_run-01_emptyroommatch.json', whereas the actual file is called sub-01_task-main_run-03_emptyroommatch.json as I set mf_reference_run = '03' as it is recommended that a middle run should be used in this situation.

[welcome[bot]]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[larsoner]

@allermat do you have use_maxwell_filter = True in your config? It seems like this should make it use the ref run:

https://github.com/mne-tools/mne-bids-pipeline/blob/3e8d5fdc528d9dec547b941ed55fa5847ca1b835/mne_bids_pipeline/steps/init/_02_find_empty_room.py#L133-L136

But maybe later we don't have the correct logic. And even if you set use_maxwell_filter = False it shouldn't error like this I think. So definitely some bug to fix. @allermat also feel free to try fixing the code and opening a PR if you're up for it!

[allermat]

Hi All,

Sorry I'm a bit late to this. Yes I do have use_maxwell_filter = True in my config. Thanks @larsoner and @hoechenberger for fixing the bug, I'll test is as soon as I can!