mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.71k stars 1.31k forks source link

MEG data problem consulting #12795

Open student123-coder opened 2 months ago

student123-coder commented 2 months ago

Description of the problem

https://github.com/neuroscan/curry-python-reader

001 002

The data collected by neuroscan MEG device can be read through this plug-in, and the data can be read normally, but there will be errors later.

Steps to reproduce

Consistent with the problem description.

Link to data

No response

Expected results

NeuroScan MEG data can be analyzed through MNE,The neuroscan MEG data analysis tutorial using the MNE toolkit can be found on github.

Actual results

There will be an error after reading the data, see Picture 002 for details

Additional information

No other information

welcome[bot] commented 2 months ago

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴

drammock commented 2 months ago

Hello! Please provide a link to the problematic file, and copy-pasteable text (not screenshots) to reproduce the problem. Also please let us know: are you able to successfully load this file using other software besides MNE?

student123-coder commented 2 months ago

Hello, Professor.

I am very glad to receive your reply. Thank you very much for your reply. My initial intention was to analyze this data with MNE tool, and I attached the original data. The mistake in this picture is that I asked someone else to help me, and he was wrong there. I did not successfully read this data here myself.

Attached is the method I found of reading data and the original data. Could you please help me read it and then do two-step preprocessing for the example? If you need any additional information from me, please feel free to contact me. Thanks again.

https://drive.google.com/drive/folders/1FekBu8JQF_12dHuMV5geA6Ilo5-LZWf6?usp=drive_link https://drive.google.com/drive/folders/1jBKJX8o23tYIfUb6MweiDtfC3_wAqh8p?usp=drive_link

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月17日(星期六) 晚上7:52 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

Hello! Please provide a link to the problematic file, and copy-pasteable text (not screenshots) to reproduce the problem. Also please let us know: are you able to successfully load this file using other software besides MNE?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

student123-coder commented 2 months ago

Hello, Professor

 

I converted the data to set format for reading, but the problem still lies in the montage. The data and my reading statement are as follows. Could you please help me take a look? Do you need me to consult the manufacturer for some information? Thank you very much and I look forward to your reply.

https://drive.google.com/drive/folders/1lj_wa1Dofzu8zab4ga2VxgQCOR1pgGOF?usp=drive_link

import mne import matplotlib.pyplot as plt raw = mne.io.read_raw_eeglab("data.set",montage_units='mm',preload=True) plt.show()

C:\Users\jd\study\Scripts\python.exe C:\Users\jd\001\set.py 

C:\Users\jd\001\set.py:3: RuntimeWarning: Estimated head radius (0.0 cm) is below the 3rd percentile for infant head size. Check if the montage_units argument is correct (the default is "mm", but your channel positions may be in different units).

  raw = mne.io.read_raw_eeglab("data.set",montage_units='mm',preload=True)

Reading C:\Users\jd\001\data.fdt

Reading 0 ... 878199  =      0.000 ...   878.199 secs...

进程已结束,退出代码为 0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月17日(星期六) 晚上7:52 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

Hello! Please provide a link to the problematic file, and copy-pasteable text (not screenshots) to reproduce the problem. Also please let us know: are you able to successfully load this file using other software besides MNE?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

mscheltienne commented 2 months ago

We can't download anything from the google drive.

C:\Users\jd\001\set.py:3: RuntimeWarning: Estimated head radius (0.0 cm) is below the 3rd percentile for infant head size. Check if the montage_units argument is correct (the default is "mm", but your channel positions may be in different unit

This is a warning, not an error. The montage in the files is probably wrong, but maybe you don't need it. Hard to tell.

If there is indeed a bug in the mne.io.read_raw_curry function and if we can reproduce it on a file you share, then we can have a look. Else this issue will be closed.

student123-coder commented 2 months ago

Hello, Professor. This is a link to the raw data, where the.cdT file is the raw data. would you download it?

https://www.dropbox.com/scl/fo/hpnz0wdmhvlueiavvhfmu/ABwhmkOqaKgATEMuuljyIeU?rlkey=eczw1codwnbpvwdjijbz5jdml&st=kfylag45&dl=0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月21日(星期三) 晚上11:06 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

We can't download anything from the google drive.

C:\Users\jd\001\set.py:3: RuntimeWarning: Estimated head radius (0.0 cm) is below the 3rd percentile for infant head size. Check if the montage_units argument is correct (the default is "mm", but your channel positions may be in different unit

This is a warning, not an error. The montage in the files is probably wrong, but maybe you don't need it. Hard to tell.

If there is indeed a bug in the mne.io.read_raw_curry function and if we can reproduce it on a file you share, then we can have a look. Else this issue will be closed.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

student123-coder commented 2 months ago

Hello, Professor. This is the file that I found to read the data, but I couldn't read it successfully. If you need any information from me, please feel free to contact me.

https://www.dropbox.com/scl/fo/wodj6dpiw1rsytqzxdxy6/AAKuFEwNUpdFtOzWfV2pP0E?rlkey=1ub0v0r66mv99926x10wpfo2y&st=u0ivdbh9&dl=0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月21日(星期三) 晚上11:06 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

We can't download anything from the google drive.

C:\Users\jd\001\set.py:3: RuntimeWarning: Estimated head radius (0.0 cm) is below the 3rd percentile for infant head size. Check if the montage_units argument is correct (the default is "mm", but your channel positions may be in different unit

This is a warning, not an error. The montage in the files is probably wrong, but maybe you don't need it. Hard to tell.

If there is indeed a bug in the mne.io.read_raw_curry function and if we can reproduce it on a file you share, then we can have a look. Else this issue will be closed.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

drammock commented 2 months ago

dropbox link also does not work :(

dropbox

student123-coder commented 2 months ago

Hello, Professor. would you try these two links?

https://www.dropbox.com/scl/fo/sxt0urbr5hpzx5nqzs6lx/AOpAOktu3wV7-aZYDl66Bk8?rlkey=0zh6duu9iaocas8fq8zvy89ed&st=w526ochw&dl=0 https://www.dropbox.com/scl/fo/sxt0urbr5hpzx5nqzs6lx/AOpAOktu3wV7-aZYDl66Bk8?rlkey=rxazrz5upkhpn6s3k1x7oisl9&st=q08uh5zr&dl=0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 凌晨1:31 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

dropbox link also does not work :(

dropbox.png (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

student123-coder commented 2 months ago

Hello, Professor. would you try these two links?

https://www.dropbox.com/scl/fo/sxt0urbr5hpzx5nqzs6lx/AOpAOktu3wV7-aZYDl66Bk8?rlkey=rxazrz5upkhpn6s3k1x7oisl9&st=w6g70n6f&dl=0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 凌晨1:31 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

dropbox link also does not work :(

dropbox.png (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

student123-coder commented 2 months ago

Hello, Professor

would you send me a shared email address, Icannot add a share to your current email address. Thank you very much and look forward to your reply.

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 凌晨1:31 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

dropbox link also does not work :(

dropbox.png (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

student123-coder commented 2 months ago

Hello, Professor. The 8.8A-B in here is the raw data, this should be fine. https://www.dropbox.com/scl/fo/ugu4olzg670kd3a78bulv/AChsMrhOZy02TwtIRMXutRI?rlkey=zy0rdp7epbs545siedfu2tg6p&st=30b60px8&dl=0

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 凌晨1:31 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

dropbox link also does not work :(

dropbox.png (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

mscheltienne commented 2 months ago

None work.

student123-coder commented 2 months ago

Hello, Professor. I shared it with two email addresses. I failed to share your email. Would you see it? Raw data folder and read data folder. Looking forward to your reply, thank you very much. Hello, Professor. I shared it with two email addresses. I failed to share your email. Can you see it? Raw data folder and read data folder. Looking forward to your reply, thank you very much.

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 晚上7:23 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

None work.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

drammock commented 2 months ago

All of the dropbox links have the same error message as in my previous reply. However I was able to go back to the google drive links, and it seems my request for access was granted, so now they work. I'll try to take a look at the data soon.

student123-coder commented 2 months ago

Ok, thank you very much. .cdt is data in its raw format. If you need any additional information, please feel free to contact us.

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月22日(星期四) 晚上10:46 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

All of the dropbox links have the same error message as in my previous reply. However I was able to go back to the google drive links, and it seems my request for access was granted, so now they work. I'll try to take a look at the data soon.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

drammock commented 2 months ago

OK. I'm not really sure what we would need to do to get MNE-Python to read this file. But I can confirm that https://github.com/neuroscan/curry-python-reader can read the file. From there, I'm able to shoehorn it into an MNE object, but some things are not accounted for (see below):

In [1]: import mne
In [2]: import curryreader as cr
In [3]: curry_data = cr.read("test data/Acq 2024_08_08_0924_Sub1 Data.cdt", plotdata=0)
INFO: Reading file Acq 2024_08_08_0924_Sub1 Data.cdt ...
INFO: Number of samples = 878200, number of channels = 188, number of trials/epochs = 1, sampling frequency = 1000.0 Hz
INFO: Found channel labels
INFO: Found sensor positions
INFO: Found events
In [4]: info = mne.create_info(ch_names=curry_data["labels"], sfreq=curry_data["info"]["samplingfreq"], ch_types="mag")
In [5]: raw = mne.io.RawArray(data=curry_data["data"].T, info=info)
Creating RawArray with float64 data, n_channels=188, n_times=878200
    Range : 0 ... 878199 =      0.000 ...   878.199 secs
Ready.

Notes: According to the Data Parameters file (.cdt.dpa), there are 2 "groups": a MAG group with 177 channels, and an "other" group with 11 channels. Data for mag channels are in fT, so will need to be scaled by 1e-15. Data in "other" group are in uV, so will need to be scaled by 1e-6. As far as I can tell, curryreader doesn't account for / handle the units in any way. Something like scaler = np.repeat([1e-15, 1e-6], (177, 11)) should be a correct scaling factor, assuming the two channel groups aren't interleaved (I have no idea if they are or not). Once you work out the channel order question, then replace the last line above with

raw = mne.io.RawArray(data=(scaler * curry_data["data"]).T, info=info)

Note that the info will still be incorrect, so you'll need to go back and correctly specify the ch_types param in create_info too (since they're not all Magnetometers, 11 of them are something else... not clear if EEG, ECG, EOG, EMG, GSR...)

Hopefully that's enough of a lead that you can work out how to get it into MNE quickly, and don't need to wait for us to get the reader bug corrected.

student123-coder commented 2 months ago

Hello, Professor. I am very glad to receive your reply. I can do that here, but there's still a coordinate system conversion problem. I will send you my solution and you can see if there is any problem. Thank you very much and look forward to your reply.

import mne

Raw=mne.io.read_raw_curry("H:\testdata\Acq2024_08_08_0924_Sub1.cdt", preload=True)preload=True)

print(raw)

 

During data reading, you need to read the location information of the.dpa file, but an error will be reported during the reading, as follows:

 

 

Analysis:The vector after normalization in the.dpa file should be the normal vector, the direction of the coil position. The reason why the error is reported is that when curry exported the data, the data of the normal vector was saved with 6 decimals, and the accuracy can be considered as 5 digits. After calculating the length of the module (which needs to open the radical sign), the accuracy could not reach 4 digits, which could not meet the accuracy requirements.

 

To do this, click on the error link, open the curry.py file, and modify it.

 

 

Check whether the suffix of the hpi file is hpi or hpim. If the suffix is hpi or hpim, modify it accordingly.

 

指间的阳光 @.***

 

------------------ 原始邮件 ------------------ 发件人: "mne-tools/mne-python" @.>; 发送时间: 2024年8月28日(星期三) 上午6:41 @.>; @.**@.>; 主题: Re: [mne-tools/mne-python] MEG data problem consulting (Issue #12795)

OK. I'm not really sure what we would need to do to get MNE-Python to read this file. But I can confirm that https://github.com/neuroscan/curry-python-reader can read the file. From there, I'm able to shoehorn it into an MNE object, but some things are not accounted for (see below): In [1]: import mne In [2]: import curryreader as cr In [3]: curry_data = cr.read("test data/Acq 2024_08_08_0924_Sub1 Data.cdt", plotdata=0) INFO: Reading file Acq 2024_08_08_0924_Sub1 Data.cdt ... INFO: Number of samples = 878200, number of channels = 188, number of trials/epochs = 1, sampling frequency = 1000.0 Hz INFO: Found channel labels INFO: Found sensor positions INFO: Found events In [4]: info = mne.create_info(ch_names=curry_data["labels"], sfreq=curry_data["info"]["samplingfreq"], ch_types="mag") In [5]: raw = mne.io.RawArray(data=curry_data["data"].T, info=info) Creating RawArray with float64 data, n_channels=188, n_times=878200 Range : 0 ... 878199 = 0.000 ... 878.199 secs Ready.

Notes: According to the Data Parameters file (.cdt.dpa), there are 2 "groups": a MAG group with 177 channels, and an "other" group with 11 channels. Data for mag channels are in fT, so will need to be scaled by 1e-15. Data in "other" group are in uV, so will need to be scaled by 1e-6. As far as I can tell, curryreader doesn't account for / handle the units in any way. Something like scaler = np.repeat([1e-15, 1e-6], (177, 11)) should be a correct scaling factor, assuming the two channel groups aren't interleaved (I have no idea if they are or not). Once you work out the channel order question, then replace the last line above with raw = mne.io.RawArray(data=(scaler * curry_data["data"]).T, info=info)

Note that the info will still be incorrect, so you'll need to go back and correctly specify the ch_types param in create_info too (since they're not all Magnetometers, 11 of them are something else... not clear if EEG, ECG, EOG, EMG, GSR...)

Hopefully that's enough of a lead that you can work out how to get it into MNE quickly, and don't need to wait for us to get the reader bug corrected.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>