search

mne-tools / mne-bids-pipeline

Automatically process entire electrophysiological datasets using MNE-Python.
https://mne.tools/mne-bids-pipeline/
BSD 3-Clause "New" or "Revised" License
134 stars 65 forks source link

finding empty room fails with multiple sessions but succeeds with one session #964

Closed drammock closed 5 days ago

drammock commented 3 weeks ago

Sorry that this issue is a bit cursory, it's the end of the work week; I'll add more details when I can.

In my config I have sessions = ["a", "b", "c"]. Running the pipeline (in debug mode, so n_jobs=1), the init steps run fine and preprocessing 1 (data quality) runs for session a but chokes when it reaches session b --- it fails to find ...emptyroommatch.json. I've seen #740 but either this issue is different or the fix in #742 didn't actually work, not sure yet.

If I then run the pipeline again but add --session b to the command line, suddenly it runs fine (all the way through data quality and on to head pos, before I quit out of the run).

welcome[bot] commented 3 weeks ago

Hello! πŸ‘‹ Thanks for opening your first issue here! ❀️ We will try to get back to you soon. πŸš΄πŸ½β€β™‚οΈ

drammock commented 3 weeks ago

OK, more details: this is a test run using just 2 of the ~50 subjects I have. In fact for now I've only written 2 of the subjs into the BIDS folder tree. Here's the tree:

``` |bids-data/ |--- README |--- dataset_description.json |--- participants.json |--- participants.tsv |--- sub-116/ |------ ses-a/ |--------- sub-116_ses-a_scans.tsv |--------- meg/ |------------ sub-116_ses-a_acq-calibration_meg.dat |------------ sub-116_ses-a_acq-crosstalk_meg.fif |------------ sub-116_ses-a_coordsystem.json |------------ sub-116_ses-a_task-AmplitudeModulatedTones_channels.tsv |------------ sub-116_ses-a_task-AmplitudeModulatedTones_events.json |------------ sub-116_ses-a_task-AmplitudeModulatedTones_events.tsv |------------ sub-116_ses-a_task-AmplitudeModulatedTones_meg.fif |------------ sub-116_ses-a_task-AmplitudeModulatedTones_meg.json |------------ sub-116_ses-a_task-InfantDirectedSpeech_channels.tsv |------------ sub-116_ses-a_task-InfantDirectedSpeech_events.json |------------ sub-116_ses-a_task-InfantDirectedSpeech_events.tsv |------------ sub-116_ses-a_task-InfantDirectedSpeech_meg.fif |------------ sub-116_ses-a_task-InfantDirectedSpeech_meg.json |------------ sub-116_ses-a_task-SyllableMismatchNegativity_channels.tsv |------------ sub-116_ses-a_task-SyllableMismatchNegativity_events.json |------------ sub-116_ses-a_task-SyllableMismatchNegativity_events.tsv |------------ sub-116_ses-a_task-SyllableMismatchNegativity_meg.fif |------------ sub-116_ses-a_task-SyllableMismatchNegativity_meg.json |------ ses-b/ |--------- sub-116_ses-b_scans.tsv |--------- meg/ |------------ sub-116_ses-b_acq-calibration_meg.dat |------------ sub-116_ses-b_acq-crosstalk_meg.fif |------------ sub-116_ses-b_coordsystem.json |------------ sub-116_ses-b_task-AmplitudeModulatedTones_channels.tsv |------------ sub-116_ses-b_task-AmplitudeModulatedTones_events.json |------------ sub-116_ses-b_task-AmplitudeModulatedTones_events.tsv |------------ sub-116_ses-b_task-AmplitudeModulatedTones_meg.fif |------------ sub-116_ses-b_task-AmplitudeModulatedTones_meg.json |------------ sub-116_ses-b_task-InfantDirectedSpeech_channels.tsv |------------ sub-116_ses-b_task-InfantDirectedSpeech_events.json |------------ sub-116_ses-b_task-InfantDirectedSpeech_events.tsv |------------ sub-116_ses-b_task-InfantDirectedSpeech_meg.fif |------------ sub-116_ses-b_task-InfantDirectedSpeech_meg.json |------------ sub-116_ses-b_task-SyllableMismatchNegativity_channels.tsv |------------ sub-116_ses-b_task-SyllableMismatchNegativity_events.json |------------ sub-116_ses-b_task-SyllableMismatchNegativity_events.tsv |------------ sub-116_ses-b_task-SyllableMismatchNegativity_meg.fif |------------ sub-116_ses-b_task-SyllableMismatchNegativity_meg.json |------ ses-c/ |--------- sub-116_ses-c_scans.tsv |--------- meg/ |------------ sub-116_ses-c_acq-calibration_meg.dat |------------ sub-116_ses-c_acq-crosstalk_meg.fif |------------ sub-116_ses-c_coordsystem.json |------------ sub-116_ses-c_task-AmplitudeModulatedTones_channels.tsv |------------ sub-116_ses-c_task-AmplitudeModulatedTones_events.json |------------ sub-116_ses-c_task-AmplitudeModulatedTones_events.tsv |------------ sub-116_ses-c_task-AmplitudeModulatedTones_meg.fif |------------ sub-116_ses-c_task-AmplitudeModulatedTones_meg.json |------------ sub-116_ses-c_task-InfantDirectedSpeech_channels.tsv |------------ sub-116_ses-c_task-InfantDirectedSpeech_events.json |------------ sub-116_ses-c_task-InfantDirectedSpeech_events.tsv |------------ sub-116_ses-c_task-InfantDirectedSpeech_meg.fif |------------ sub-116_ses-c_task-InfantDirectedSpeech_meg.json |------------ sub-116_ses-c_task-SyllableMismatchNegativity_channels.tsv |------------ sub-116_ses-c_task-SyllableMismatchNegativity_events.json |------------ sub-116_ses-c_task-SyllableMismatchNegativity_events.tsv |------------ sub-116_ses-c_task-SyllableMismatchNegativity_meg.fif |------------ sub-116_ses-c_task-SyllableMismatchNegativity_meg.json |--- sub-215/ |------ ses-a/ |--------- sub-215_ses-a_scans.tsv |--------- meg/ |------------ sub-215_ses-a_acq-calibration_meg.dat |------------ sub-215_ses-a_acq-crosstalk_meg.fif |------------ sub-215_ses-a_coordsystem.json |------------ sub-215_ses-a_task-AmplitudeModulatedTones_channels.tsv |------------ sub-215_ses-a_task-AmplitudeModulatedTones_events.json |------------ sub-215_ses-a_task-AmplitudeModulatedTones_events.tsv |------------ sub-215_ses-a_task-AmplitudeModulatedTones_meg.fif |------------ sub-215_ses-a_task-AmplitudeModulatedTones_meg.json |------------ sub-215_ses-a_task-InfantDirectedSpeech_channels.tsv |------------ sub-215_ses-a_task-InfantDirectedSpeech_events.json |------------ sub-215_ses-a_task-InfantDirectedSpeech_events.tsv |------------ sub-215_ses-a_task-InfantDirectedSpeech_meg.fif |------------ sub-215_ses-a_task-InfantDirectedSpeech_meg.json |------------ sub-215_ses-a_task-SyllableMismatchNegativity_channels.tsv |------------ sub-215_ses-a_task-SyllableMismatchNegativity_events.json |------------ sub-215_ses-a_task-SyllableMismatchNegativity_events.tsv |------------ sub-215_ses-a_task-SyllableMismatchNegativity_meg.fif |------------ sub-215_ses-a_task-SyllableMismatchNegativity_meg.json |------ ses-b/ |--------- sub-215_ses-b_scans.tsv |--------- meg/ |------------ sub-215_ses-b_acq-calibration_meg.dat |------------ sub-215_ses-b_acq-crosstalk_meg.fif |------------ sub-215_ses-b_coordsystem.json |------------ sub-215_ses-b_task-AmplitudeModulatedTones_channels.tsv |------------ sub-215_ses-b_task-AmplitudeModulatedTones_events.json |------------ sub-215_ses-b_task-AmplitudeModulatedTones_events.tsv |------------ sub-215_ses-b_task-AmplitudeModulatedTones_meg.fif |------------ sub-215_ses-b_task-AmplitudeModulatedTones_meg.json |------------ sub-215_ses-b_task-InfantDirectedSpeech_channels.tsv |------------ sub-215_ses-b_task-InfantDirectedSpeech_events.json |------------ sub-215_ses-b_task-InfantDirectedSpeech_events.tsv |------------ sub-215_ses-b_task-InfantDirectedSpeech_meg.fif |------------ sub-215_ses-b_task-InfantDirectedSpeech_meg.json |--- sub-emptyroom/ |------ ses-19130718/ |--------- sub-emptyroom_ses-19130718_scans.tsv |--------- meg/ |------------ sub-emptyroom_ses-19130718_task-noise_channels.tsv |------------ sub-emptyroom_ses-19130718_task-noise_meg.fif |------------ sub-emptyroom_ses-19130718_task-noise_meg.json |------ ses-19130721/ |--------- sub-emptyroom_ses-19130721_scans.tsv |--------- meg/ |------------ sub-emptyroom_ses-19130721_task-noise_channels.tsv |------------ sub-emptyroom_ses-19130721_task-noise_meg.fif |------------ sub-emptyroom_ses-19130721_task-noise_meg.json |------ ses-19131206/ |--------- sub-emptyroom_ses-19131206_scans.tsv |--------- meg/ |------------ sub-emptyroom_ses-19131206_task-noise_channels.tsv |------------ sub-emptyroom_ses-19131206_task-noise_meg.fif |------------ sub-emptyroom_ses-19131206_task-noise_meg.json ```

Here's the part of my preparatory script where the Raw data files (and associated ERMs) are written with MNE-BIDS:

https://github.com/ilabsbrainteam/badbaby-redux/blob/b21bf5379943f47bba725282b15d0f759310f20c/prep-dataset/bidsify.py#L204-L214

As you can see, empty_room is always passed, and just to be sure I (locally) stuck an assert erm is not None into that script and re-ran it, and it succeeds.

I checked the *scans.tsv files to make sure the dates match the sub-emptyroom folders/files (they do):

``` $ grep -R -a3 acq_time bids-data/ bids-data/sub-emptyroom/ses-19131206/sub-emptyroom_ses-19131206_scans.tsv:ο»Ώfilename acq_time bids-data/sub-emptyroom/ses-19131206/sub-emptyroom_ses-19131206_scans.tsv-meg/sub-emptyroom_ses-19131206_task-noise_meg.fif 1913-12-06T15:57:32.801790Z -- bids-data/sub-emptyroom/ses-19130718/sub-emptyroom_ses-19130718_scans.tsv:ο»Ώfilename acq_time bids-data/sub-emptyroom/ses-19130718/sub-emptyroom_ses-19130718_scans.tsv-meg/sub-emptyroom_ses-19130718_task-noise_meg.fif 1913-07-18T15:24:39.330560Z -- bids-data/sub-emptyroom/ses-19130721/sub-emptyroom_ses-19130721_scans.tsv:ο»Ώfilename acq_time bids-data/sub-emptyroom/ses-19130721/sub-emptyroom_ses-19130721_scans.tsv-meg/sub-emptyroom_ses-19130721_task-noise_meg.fif 1913-07-21T15:44:48.232088Z -- bids-data/sub-116/ses-b/sub-116_ses-b_scans.tsv:ο»Ώfilename acq_time bids-data/sub-116/ses-b/sub-116_ses-b_scans.tsv-meg/sub-116_ses-b_task-AmplitudeModulatedTones_meg.fif 1913-12-06T16:24:43.743053Z bids-data/sub-116/ses-b/sub-116_ses-b_scans.tsv-meg/sub-116_ses-b_task-InfantDirectedSpeech_meg.fif 1913-12-06T16:29:51.873795Z bids-data/sub-116/ses-b/sub-116_ses-b_scans.tsv-meg/sub-116_ses-b_task-SyllableMismatchNegativity_meg.fif 1913-12-06T16:35:18.483082Z -- bids-data/sub-116/ses-c/sub-116_ses-c_scans.tsv:ο»Ώfilename acq_time bids-data/sub-116/ses-c/sub-116_ses-c_scans.tsv-meg/sub-116_ses-c_task-AmplitudeModulatedTones_meg.fif 1913-07-18T18:30:53.874570Z bids-data/sub-116/ses-c/sub-116_ses-c_scans.tsv-meg/sub-116_ses-c_task-InfantDirectedSpeech_meg.fif 1913-07-18T18:35:52.872554Z bids-data/sub-116/ses-c/sub-116_ses-c_scans.tsv-meg/sub-116_ses-c_task-SyllableMismatchNegativity_meg.fif 1913-07-18T18:48:49.881742Z -- bids-data/sub-116/ses-a/sub-116_ses-a_scans.tsv:ο»Ώfilename acq_time bids-data/sub-116/ses-a/sub-116_ses-a_scans.tsv-meg/sub-116_ses-a_task-AmplitudeModulatedTones_meg.fif 1913-07-18T18:30:53.874570Z bids-data/sub-116/ses-a/sub-116_ses-a_scans.tsv-meg/sub-116_ses-a_task-InfantDirectedSpeech_meg.fif 1913-07-18T18:35:52.872554Z bids-data/sub-116/ses-a/sub-116_ses-a_scans.tsv-meg/sub-116_ses-a_task-SyllableMismatchNegativity_meg.fif 1913-07-18T18:48:49.881742Z -- bids-data/sub-215/ses-b/sub-215_ses-b_scans.tsv:ο»Ώfilename acq_time bids-data/sub-215/ses-b/sub-215_ses-b_scans.tsv-meg/sub-215_ses-b_task-AmplitudeModulatedTones_meg.fif 1913-12-06T18:11:52.330383Z bids-data/sub-215/ses-b/sub-215_ses-b_scans.tsv-meg/sub-215_ses-b_task-InfantDirectedSpeech_meg.fif 1913-12-06T18:53:07.394415Z -- bids-data/sub-215/ses-a/sub-215_ses-a_scans.tsv:ο»Ώfilename acq_time bids-data/sub-215/ses-a/sub-215_ses-a_scans.tsv-meg/sub-215_ses-a_task-AmplitudeModulatedTones_meg.fif 1913-07-21T22:04:31.843037Z bids-data/sub-215/ses-a/sub-215_ses-a_scans.tsv-meg/sub-215_ses-a_task-InfantDirectedSpeech_meg.fif 1913-07-21T22:09:52.254852Z bids-data/sub-215/ses-a/sub-215_ses-a_scans.tsv-meg/sub-215_ses-a_task-SyllableMismatchNegativity_meg.fif 1913-07-21T22:15:04.662665Z ```

Here is the full pipeline log:

``` $ mne_bids_pipeline --config amtone.py --debug [3/1196] β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬ Welcome aboard MNE-BIDS-Pipeline! πŸ‘‹ ────────────────────────────────────────────────────────────────────────────────────────────────── β”‚11:29:51β”‚ πŸ“ Using configuration: amtone.py β”‚11:29:51β”‚ ❌ Overriding config.on_error = 'debug' β”‚11:29:51β”‚ ❌ Setting config.n_jobs=1 because config.on_error="debug" └────────┴ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬ init/_01_init_derivatives_dir ───────────────────────────────────────────────────────────────────────────────────────────────────────── β”‚11:29:51β”‚ ⏳️ Initializing output directories. └────────┴ done (1s) β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬ init/_02_find_empty_room ────────────────────────────────────────────────────────────────────────────────────────────────────────────── └────────┴ done (1s) β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬ preprocessing/_01_data_quality ──────────────────────────────────────────────────────────────────────────────────────────────────────── β”‚11:29:51β”‚ ⏳️ sub-116 ses-a Reading experimental recording: sub-116_ses-a_task-AmplitudeModulatedTones β”‚11:29:52β”‚ ⏳️ sub-116 ses-a Finding flat channels and noisy channels using Maxwell filtering. β”‚11:30:29β”‚ ⏳️ sub-116 ses-a Found no flat channels. β”‚11:30:29β”‚ ⏳️ sub-116 ses-a Found 3 noisy channels: MEG1413, MEG2432, MEG2622 β”‚11:30:29β”‚ ⏳️ sub-116 ses-a Found 3 bad channels. β”‚11:30:29β”‚ ⏳️ sub-116 ses-a Initializing report HDF5 file β”‚11:30:30β”‚ ⏳️ sub-116 ses-a Adding original raw data to report β”‚11:30:48β”‚ ⏳️ sub-116 ses-a Adding noisy channel detection to report β”‚11:30:51β”‚ ⏳️ sub-116 ses-a Adding config and sys info to report β”‚11:30:55β”‚ ⏳️ sub-116 ses-a Saving report: /storage/badbaby-redux/bids-data/derivatives/mne-bids-pipeline/sub-116/ses-a/meg/sub-116_ses-a_task-AmplitudeModulatedTones_report.html β”‚11:30:55β”‚ ⏳️ sub-116 ses-a run-noise Reading empty-room recording: sub-emptyroom_ses-19130718_task-noise_meg.fif β”‚11:30:57β”‚ ⏳️ sub-116 ses-a run-noise Finding flat channels and noisy channels using Maxwell filtering. β”‚11:32:03β”‚ ⏳️ sub-116 ses-a run-noise Found no flat channels. β”‚11:32:03β”‚ ⏳️ sub-116 ses-a run-noise Found 3 noisy channels: MEG1941, MEG2422, MEG2432 β”‚11:32:03β”‚ ⏳️ sub-116 ses-a run-noise Found 3 bad channels. β”‚11:32:03β”‚ ⏳️ sub-116 ses-a run-noise Adding original raw data to report β”‚11:32:12β”‚ ⏳️ sub-116 ses-a run-noise Adding noisy channel detection to report β”‚11:32:15β”‚ ⏳️ sub-116 ses-a run-noise Saving report: /storage/badbaby-redux/bids-data/derivatives/mne-bids-pipeline/sub-116/ses-a/meg/sub-116_ses-a_task-AmplitudeModulatedTones_report.html Traceback (most recent call last): File "/opt/conda/miniforge3/envs/badbaby/bin/mne_bids_pipeline", line 10, in sys.exit(main()) ^^^^^^ File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/_main.py", line 234, in main step_module.main(config=config_imported) File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/steps/preprocessing/_01_data_quality.py", line 343, in main logs = parallel( ^^^^^^^^^ File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/steps/preprocessing/_01_data_quality.py", line 354, in for run, task in get_runs_tasks( ^^^^^^^^^^^^^^^ File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/_config_utils.py", line 288, in get_runs_tasks if _get_noise_path(mf_reference_run=mf_reference_run, **kwargs): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/_import_data.py", line 643, in _get_noise_path raw_fname = _read_json(_empty_room_match_path(raw_fname, cfg))["fname"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/miniforge3/envs/badbaby/lib/python3.12/site-packages/mne_bids_pipeline/_io.py", line 14, in _read_json with open(fname, encoding="utf-8") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/storage/badbaby-redux/bids-data/derivatives/mne-bids-pipeline/sub-116/ses-b/meg/sub-116_ses-b_task-AmplitudeModulatedTones_emptyroommatch.json' ```

the emptyroommatch files seem to only exist for session a:

$ find bids-data/ -name "*emptyroommatch*"
bids-data/derivatives/mne-bids-pipeline/sub-116/ses-a/meg/sub-116_ses-a_task-AmplitudeModulatedTones_emptyroommatch.json
bids-data/derivatives/mne-bids-pipeline/sub-215/ses-a/meg/sub-215_ses-a_task-AmplitudeModulatedTones_emptyroommatch.json

Here's the config file: https://github.com/ilabsbrainteam/badbaby-redux/blob/main/pipeline/amtone.py

Any ideas what might be going wrong?

drammock commented 3 weeks ago

forgot to mention: this is with version 1.9.0 (stable) installed on the workstation (not current main)

drammock commented 6 days ago

still hitting this, though I've found a fix in #976. Meanwhile, contrary to my comment in https://github.com/mne-tools/mne-bids-pipeline/issues/967#issuecomment-2253524228:

when running with an editable install of MBP, the --debug flag does indeed drop me into a postmortem debugger on error. Not sure why that wasn't working with a standard (conda-forge) install of 1.9.0

in fact today with an editable install I'm once again not getting a debug prompt no matter where I stick in an assert False or raise RuntimeError, even though I was getting one previously when debugging #967. I'm running editable installs of current main of MNE, MNE-BIDS, and MNE-BIDS-Pipeline. I really have no clue what has changed 😞

larsoner commented 6 days ago

I'm running editable installs of current main of MNE, MNE-BIDS, and MNE-BIDS-Pipeline. I really have no clue what has changed 😞

Is this in a plain Linux terminal or VSCode terminal? If VSCode I would verify in a plain Linux terminal just to rule out any VSCode-specific stuff