raphaelvallat / yasa

YASA (Yet Another Spindle Algorithm): a Python package to analyze polysomnographic sleep recordings.
https://raphaelvallat.com/yasa/
BSD 3-Clause "New" or "Revised" License
428 stars 115 forks source link

Important conversion problem - urgent help needed :) #165

Closed alivedpac closed 5 months ago

alivedpac commented 8 months ago

We're tying to read in an .eeg file with the mne read_raw_brainvision function: raw = mne.io.read_raw_brainvision(eeg_data_path, eog=('EOG_lefup', 'EOG_riglow', 'VEOG'), misc='auto', scale=1, preload=True, verbose=False) The data works perfectly while running the sleep staging algorithm. Nevertheless, as soon as we want to run sw_detection function we get in troubles regarding the format of the data. This is the code and output of the data.

impChannels = ['C3','C4', 'O1', 'P3']
data = raw_eeg.get_data(impChannels)
print(np.round(data[:, 0:5], 3))
print(raw_eeg.info)
[[-0.021 -0.021 -0.021 -0.021 -0.021]
 [-0.002 -0.002 -0.002 -0.002 -0.002]
 [-0.005 -0.005 -0.005 -0.005 -0.005]
 [-0.007 -0.007 -0.007 -0.007 -0.007]]
<Info | 9 non-empty values
 bads: []
 ch_names: F3, F4, C3, C4, P3, O1
 chs: 6 EEG
 custom_ref_applied: False
 file_id: 4 items (dict)
 highpass: 0.0 Hz
 lowpass: 50.0 Hz
 meas_date: 2023-04-12 23:38:30 UTC
 meas_id: 4 items (dict)
 nchan: 6
 projs: []
 sfreq: 100.0 Hz

grafik we tried to rescale as you already suggested elsewhere with the following:

data = raw_eeg.get_data(impChannels, units='uV')
print(np.round(data[:, 0:5], 3))

now, we have way too big data:

`[[-21312.797 -21316.311 -21320.446 -21309.85  -21303.207]
 [ -1797.296  -1802.727  -1808.457  -1811.523  -1808.294]
 [ -5276.187  -5282.263  -5281.657  -5281.113  -5278.127]
 [ -7207.639  -7210.505  -7211.93   -7205.757  -7197.609]]`

grafik

Still, as soon as we try to manually rescale to sensemaking numbers, we don't get any slow waves and only a very low range of data. The data checked in brainvision looks good, so we're really confused.

impChannels = ['C3','C4', 'O1', 'P3']
data = raw_eeg.get_data(impChannels, units='uV')
data = data / -1000 #as strange data format when loading .fif data
print(np.round(data[:, 0:5], 3))
[[21.313 21.316 21.32  21.31  21.303]
 [ 1.797  1.803  1.808  1.812  1.808]
 [ 5.276  5.282  5.282  5.281  5.278]
 [ 7.208  7.211  7.212  7.206  7.198]]

grafik

raphaelvallat commented 8 months ago

Hi,

Can you please plot the full time-series of one representative channel? I wonder if there are strong artefacts either at the beginning or end of the file that would mess up the calculation of the standard deviation. According to the read_raw_brainvision, the data is parsed with microvolts unit, and then converted internally to Volts to fit the MNE standard. Using units="uV" should give the correct results, assuming that the scale information in the header of your EEG file is correct.

If the amplitude does not match what you're seeing in the BrainVision software, I would recommend opening an issue in the MNE GitHub instead, since this is not a YASA error per se.

Thanks

alivedpac commented 5 months ago

Very sorry for the late reply. The problem was on my side (a very basic mistake): not rereferencing to the mastoid electrodes. Additionally, sth didn't work out with accessing the raw data which was solved by using an edf-export. Nevertheless, thanks a lot for your quick reply and help :) Have a great day!