welcome[bot]
commented
2 weeks ago Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽♂️
Closed juliapottkaemper closed 2 weeks ago
welcome[bot]
commented
2 weeks ago Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽♂️
drammock
commented
2 weeks ago .csv is not one of the supported formats for EEG data. See https://bids-specification.readthedocs.io/en/stable/modality-specific-files/electroencephalography.html#eeg-recording-data
For BIDS, EEG data MUST be stored in one of the following formats:
.edf(European Data Format),.vhdr, .vmrk, .eeg(BrainVision),.set, .fdt(FieldTrip),.bdf(biosemi)
juliapottkaemper
commented
2 weeks ago Hi,
Thanks a lot for your comment. However, I do still get the same error if I use .edf files. So it must come from something else?
Best, Julia
drammock
commented
2 weeks ago Ah ok, I was misled by the question about CSV files. Looking at the screenshot, the error is:
missing in_files["raw_task-rest_run-01"] = /data/jcpottkamper/Desktop/BIDS/sub_01/eeg/sub-01_task-rest_run-01
Which means it can't find the file at that location. Can you confirm that there is in fact data at that location? If so, what filename(s) are present?
(also please in future copy-paste the actual text of the error instead of posting screenshots, so we can (1) zoom in more easily, and (2) copy-paste parts of the error instead of re-typing)
juliapottkaemper
commented
2 weeks ago Hi,
Thanks for your comment. I indeed had two questions in 1 that I could have phrased more clearly. It is clear now that csv files are not supported. I tried out several things and now I know where the error came from. I had the right path to the data, but there was an issue with how I named my data. For example, I had the following names:
I also put task = "rest" into my config.py file, which complicated matters. It works now! Thanks.
I am running into another problem now. If I want to run an ICA for preprocessing, do I need the _electrodes, _channels and *_coordsystem files necessarily? I do not have an MRI scan for the coordsystem file, which is why I also cannot include the _electrodes file. I chose the 'biosemi32' montage in the config.py file, without having the _electrodes, _channels or _coordsystem files in my BIDS path. MNE-BIDS gets stuck saying that two electrodes overslap (Fpz, POz) with the following error:
────────┬ preprocessing/_06a2_find_ica_artifacts ────────────────────────────────────────────────────────── │10:56:05│ ⏳️ sub-01 Loading ICA solution │10:56:05│ ⏳️ sub-01 No ECG or magnetometer channels are present, cannot automate artifact detection for ECG. │10:56:05│ ⏳️ sub-01 Creating EOG epochs … │10:56:05│ ⏳️ sub-01 Performing automated EOG artifact detection … │10:56:06│ ⏳️ sub-01 Detected 1 EOG-related ICs in 50 EOG epochs. │10:56:06│ ⏳️ sub-01 Saving ICA solution and detected artifacts to disk. │10:56:07│ ⏳️ sub-01 Saving ICA report: /data/jcpottkamper/Desktop/mne_sourcedata/data/derivatives/mne-bids-pipeline/sub-01/eeg/sub-01_task-eyesclosed_proc-ica+components_report.html │10:56:07│ ❌ sub-01 A critical error occurred. The error message was: The following electrodes have overlapping positions, which causes problems during visualization: Fpz, POz
Aborting pipeline run. The traceback is:
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne_bids_pipeline/steps/preprocessing/_06a2_find_ica_artifacts.py", line 310, in find_ica_artifacts
report.add_ica(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 2041, in add_ica
self._add_ica(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 1972, in _add_ica
self._add_ica_components(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 1817, in _add_ica_components
figs = ica.plot_components(picks=picks, title="", colorbar=True, show=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/preprocessing/ica.py", line 2473, in plot_components
return plot_ica_components(
^^^^^^^^^^^^^^^^^^^^
File "
Could you help with that or should I open a new issue here first?
hoechenberger
commented
2 weeks ago Hello @juliapottkaemper, MNE-BIDS-Pipeline expects your dataset to be fully confirming to BIDS. Based on what you wrote above, I'm not sure that this is the case. Please run the BIDS validator to check your dataset for BIDS compliance.
juliapottkaemper
commented
2 weeks ago Hi @hoechenberger ,
I checked my data with the BIDS validator, but the error remains. I did include a _channels.tsv file, which does not seem to give any issues (after using mne_bids.write_raw_bids(raw, bids_path, overwrite=True) to create this file). I included the file in the appendix. However, the same error remains: sub-01_task-eyesclosed_channels.csv Note that I only changed the file type from tsv to csv to upload the file here on Github.
┌────────┬ preprocessing/_06a2_find_ica_artifacts ────────────────────────────────────────────────────────── │12:50:08│ ⏳️ sub-01 Loading ICA solution │12:50:08│ ⏳️ sub-01 No ECG or magnetometer channels are present, cannot automate artifact detection for ECG. │12:50:08│ ⏳️ sub-01 Creating EOG epochs … │12:50:08│ ⏳️ sub-01 Performing automated EOG artifact detection … │12:50:09│ ⏳️ sub-01 Detected 1 EOG-related ICs in 50 EOG epochs. │12:50:09│ ⏳️ sub-01 Saving ICA solution and detected artifacts to disk. │12:50:11│ ⏳️ sub-01 Adding config and sys info to report │12:50:22│ ⏳️ sub-01 Saving ICA report: /data/jcpottkamper/Desktop/mne_sourcedata/data/derivatives/mne-bids-pipeline/sub-01/eeg/sub-01_task-eyesclosed_proc-ica+components_report.html │12:50:23│ ❌ sub-01 A critical error occurred. The error message was: The following electrodes have overlapping positions, which causes problems during visualization: Fpz, POz
Aborting pipeline run. The traceback is:
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne_bids_pipeline/steps/preprocessing/_06a2_find_ica_artifacts.py", line 310, in find_ica_artifacts
report.add_ica(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 2041, in add_ica
self._add_ica(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 1972, in _add_ica
self._add_ica_components(
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/report/report.py", line 1817, in _add_ica_components
figs = ica.plot_components(picks=picks, title="", colorbar=True, show=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/preprocessing/ica.py", line 2473, in plot_components
return plot_ica_components(
^^^^^^^^^^^^^^^^^^^^
File "
I understand that this means that somewhere in the montage, two electrodes are overlapping. But, can I manually change that if I do not have a coordsystem and electrodes file?
hoechenberger
commented
2 weeks ago Specifying a template montage in your config file should ensure this won't happen. Could you try that again? If you disable ICA via spatial_filter = None, do the other steps pass?
juliapottkaemper
commented
2 weeks ago Hi,
I originally had the montage template "biosemi32" specified in the configuration file. As you suggested, I disabled the ICA. I also disabled the autoreject feature as this threw an error. I get some output now. However, I cannot open the files or plot the processed data to check how it looks like. It is in .fif format, so I used mne.io.read_raw_fif(). But I can only open this file "*_epo.fif", not the files "_proc-clean_epo.fif" or "proc-filt_raw.fif" for example. If I try to open any other .fif file, the following error occurs:
^
File "/data/jcpottkamper/miniconda3/lib/python3.12/site-packages/mne/io/fiff/raw.py", line 537, in read_raw_fif
return Raw(
^^^^
File "
Is this normal behavior? Shouldn't all files containt information to be opened? I am confused.. Also, of course I would like to implement ICA because I am certain I need this for cleaning my data appropriately. Do you have any suggestions and explantions for the output files? Thanks in advance.
hoechenberger
commented
2 weeks ago Hi,
I originally had the montage template "biosemi32" specified in the configuration file. As you suggested, I disabled the ICA. I also disabled the autoreject feature as this threw an error. I get some output now.
So is the pipeline working without ICA and autoreject?
The problems you're having when opening the output files is a different issue. Please try to stick to one thing at a time.
juliapottkaemper
commented
2 weeks ago Yes the pipeline works without ICA and autoreject
hoechenberger
commented
2 weeks ago @juliapottkaemper Could you kindly try to install the latest development version of the pipeline and see if this fixes anything for you?
pip install -U https://github.com/mne-tools/mne-bids-pipeline/archive/refs/heads/main.zipGood morning @hoechenberger I updated for the last version as you suggested and ran the pipeline again. First without the ICA and then tried enabling it. Unfortunately, I get the same error:
┌────────┬ preprocessing/_06a2_find_ica_artifacts ───────────────────────────────────────────────────────────────────────────────────── │11:43:46│ ⏳️ sub-01 Loading ICA solution │11:43:46│ ⏳️ sub-01 No ECG or magnetometer channels are present, cannot automate artifact detection for ECG. │11:43:46│ ⏳️ sub-01 Creating EOG epochs … │11:43:46│ ⏳️ sub-01 Performing automated EOG artifact detection … │11:43:46│ ⏳️ sub-01 Detected 1 EOG-related ICs in 50 EOG epochs: 17 │11:43:46│ ⏳️ sub-01 Saving ICA solution and detected artifacts to disk. │11:43:47│ ⏳️ sub-01 Saving EOG artifact: /data/jcpottkamper/Desktop/mne_sourcedata/data/derivatives/mne-bids-pipeline/sub-01/eeg/sub-01_task-eyesclosed_proc-ica+eog_ave.fif │11:43:47│ ⏳️ sub-01 Adding "ICA: components" to report. │11:43:48│ ⏳️ sub-01 Saving report: /data/jcpottkamper/Desktop/mne_sourcedata/data/derivatives/mne-bids-pipeline/sub-01/eeg/sub-01_task-eyesclosed_report.html │11:43:48│ ❌ sub-01 A critical error occurred. The error message was: The following electrodes have overlapping positions, which causes problems during visualization: Fpz, POz
hoechenberger
commented
2 weeks ago How do you specify the montage in your config file?
juliapottkaemper
commented
2 weeks ago The only template that matches my electrode setup is biosemi32, so I specified this as follows: eeg_template_montage= "biosemi32"
hoechenberger
commented
2 weeks ago I don't think this montage comes with Fpz and POz channels
can you try a different montage, for example easycap-M1?
juliapottkaemper
commented
2 weeks ago Oh shoot you are correct!! I even plotted it beforehand, but it does not contain these channels.. now the pipeline works with ICA. The autoreject function now also works!! Thank you so much!
If i understand it correctly, the file with the cleaned data is the following: sub-01_task-eyesclosed_proc-clean_raw.fif I can indeed open this file and plot it. The other files (_proc-clean_epo.fif and _proc-clean_cov.fif) I still cannot open, but I guess they are irrelevant as I have access to the preprocessed data?
drammock
commented
2 weeks ago The other files (_proc-clean_epo.fif and _proc-clean_cov.fif) I still cannot open, but I guess they are irrelevant as I have access to the preprocessed data?
Those are opened with different functions (not with read_raw_fif). Search docs for read_epochs and read_cov
hoechenberger
commented
2 weeks ago We just released MNE-BIDS-Pipeline 1.9, which should not silently proceed anymore if a montage doesn't contain locations for all electrodes in your data. You can update via pip install -U mne-bids-pipeline.
For user questions like how to read the output files, please refer to the MNE user forum.
Hi,
I am trying to get MNE-BIDS to work for my eeg data. However, it seems that it is not possible to work with .csv files containing eeg data is that correct? I believe that I have all data in BIDS format. I get an error about missing in_files. Could that be the issue here? Thanks in advance for helping!
Kind regards, julia