search

sccn / eeglab

EEGLAB is an open source signal processing environment for electrophysiological signals running on Matlab and developed at the SCCN/UCSD
https://eeglab.ucsd.edu/
Other
580 stars 238 forks source link

MFF bug with old EGI files #263

Closed arild-lab closed 3 years ago

arild-lab commented 3 years ago

When I try to import some EGI data recorded over 10 years ago (recorded with EGI System 200, 64 channels), I get this error: "Block sample size problem, (Error occurred in function mff_import() at line 93). here's my matlab output:

You are using the latest version of EEGLAB. Signal Block Version Number: 1 Signal Block Data Size: 11180 Signal Block Header size. 564 Number of Signals: 65 Signal Block Samples: 43 Sample Rate: 200 Signal Depth: 32

Optional Header type: 1 Optional Header number of blocks total: 252 Optional Header number of Samples total: 1011219 Optional Header number of Signals: 129

ver

MATLAB Version: 9.8.0.1323502 (R2020a) MATLAB License Number: 129273 Operating System: Microsoft Windows 10 Education Version 10.0 (Build 17763) Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

MATLAB Version 9.8 (R2020a)
Deep Learning Toolbox Version 14.0 (R2020a)
EEGLAB Toolbox to process EEG data Version - see
ERP PCA Toolbox Version 2.91
FastICA for Matlab 7.x and 6.x Version 2.5, October 19
FieldTrip Version unknown www.fieldtriptoolbox.org Optimization Toolbox Version 8.5 (R2020a)
Parallel Computing Toolbox Version 7.2 (R2020a)
Signal Processing Toolbox Version 8.4 (R2020a)
Statistics and Machine Learning Toolbox Version 11.7 (R2020a)
Wavelet Toolbox Version 5.4 (R2020a)

arild-lab commented 3 years ago

my example file was too big :-(

arnodelorme commented 3 years ago

Could you post it at (drag and drop)

https://drive.google.com/drive/folders/1FgNv9TuuEVHahs9mCfp5lgBpG7z0co71?usp=sharing

Thank you

arild-lab commented 3 years ago

I posted a single subject zipped mff file there that illustrates the bug: 211.mff.zip

On Mon, Mar 8, 2021 at 4:29 PM Arnaud Delorme notifications@github.com wrote:

Could you post it at (drag and drop)

https://drive.google.com/drive/folders/1FgNv9TuuEVHahs9mCfp5lgBpG7z0co71?usp=sharing

Thank you

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sccn/eeglab/issues/263#issuecomment-793092978, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATB6EHIUO2Q5AB2RZHDUD43TCU6U7ANCNFSM4YP77I7Q .

arnodelorme commented 3 years ago

It is strange. It seems that the end of the file switch from 65 to 129 channels. The importer does not support this kind of file. It would be possible to import the beginning (or the end) of the file only but this is such a special case, I do not think it is worth the effort.

arild-lab commented 3 years ago

Hi Arnaud, that's weird--this is a set of 30 participants recorded on an EGI 200 system with GSN adult 65 vs.2 electrode net. There should be no switch to 129 channels at the end, unless something has happened somewhere in the path from the EGI format to matlab... Can I give you the original raw session files instead? Arild

On Wed, Mar 17, 2021 at 3:30 PM Arnaud Delorme @.***> wrote:

It is strange. It seems that the end of the file switch from 65 to 129 channels. The importer does not support this kind of file. It would be possible to import the beginning of the file only but this is such a special case, I do not think it is worth the effort.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sccn/eeglab/issues/263#issuecomment-801352352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATB6EHOVEV7K4GVQSXA5UCDTED7PFANCNFSM4YP77I7Q .

arild-lab commented 3 years ago

hi, ok, I checked a few more files and correct--they have multiple sensoy laouts--must have been an error during data collection. The issue disappears when segmenting in Netstation (which does not complain!). I will post a file that does not have the issue (but maybe that one will import without problem, I have to test). Arild

On Wed, Mar 17, 2021 at 3:54 PM Arild Hestvik @.***> wrote:

Hi Arnaud, that's weird--this is a set of 30 participants recorded on an EGI 200 system with GSN adult 65 vs.2 electrode net. There should be no switch to 129 channels at the end, unless something has happened somewhere in the path from the EGI format to matlab... Can I give you the original raw session files instead? Arild

On Wed, Mar 17, 2021 at 3:30 PM Arnaud Delorme @.***> wrote:

It is strange. It seems that the end of the file switch from 65 to 129 channels. The importer does not support this kind of file. It would be possible to import the beginning of the file only but this is such a special case, I do not think it is worth the effort.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sccn/eeglab/issues/263#issuecomment-801352352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATB6EHOVEV7K4GVQSXA5UCDTED7PFANCNFSM4YP77I7Q .

arild-lab commented 3 years ago

I reran EEGLAB's mff import with a file that I know do not have the extra montage at the end, and now get another error:

You are using the latest version of EEGLAB. Signal Block Version Number: 1 Signal Block Data Size: 4160 Signal Block Header size. 564 Number of Signals: 65 Signal Block Samples: 16 Sample Rate: 200 Signal Depth: 32

Optional Header type: 1 Optional Header number of blocks total: 308 Optional Header number of Samples total: 1221944 Optional Header number of Signals: 65

eeg_checkset note: upper time limit (xmax) adjusted so (xmax-xmin)*srate+1 = number of frames Make sure you to use the MFF ressource path name not the relative path (crash often related to that issue) ** MFF Version: 3 File's Recording Time: 2006-04-18T11:17:42.705000-05:00 XMLLocalFileResource: error: could not add neighbor set: class java.lang.NumberFormatException Warning: channel labels should not be empty, creating unique labels Channel structure size not consistent with the data so changes will be ignored Use the function pop_select(EEG, 'nochannel', [x]); if you wish the remove data channels [Fatal Error] Events_DIN_1.xml:11:26: Invalid byte 1 of 1-byte UTF-8 sequence.

MFFRootDoc: error: org.xml.sax.SAXParseException; systemId: file:/G:/My%20Drive/FG/FG-kids%20file%20transfer/215.mff/Events_DIN_1.xml; lineNumber: 11; columnNumber: 26; Invalid byte 1 of 1-byte UTF-8 sequence.

On Wed, Mar 17, 2021 at 4:05 PM Arild Hestvik @.***> wrote:

hi, ok, I checked a few more files and correct--they have multiple sensoy laouts--must have been an error during data collection. The issue disappears when segmenting in Netstation (which does not complain!). I will post a file that does not have the issue (but maybe that one will import without problem, I have to test). Arild

On Wed, Mar 17, 2021 at 3:54 PM Arild Hestvik @.***> wrote:

Hi Arnaud, that's weird--this is a set of 30 participants recorded on an EGI 200 system with GSN adult 65 vs.2 electrode net. There should be no switch to 129 channels at the end, unless something has happened somewhere in the path from the EGI format to matlab... Can I give you the original raw session files instead? Arild

On Wed, Mar 17, 2021 at 3:30 PM Arnaud Delorme @.***> wrote:

It is strange. It seems that the end of the file switch from 65 to 129 channels. The importer does not support this kind of file. It would be possible to import the beginning of the file only but this is such a special case, I do not think it is worth the effort.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sccn/eeglab/issues/263#issuecomment-801352352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATB6EHOVEV7K4GVQSXA5UCDTED7PFANCNFSM4YP77I7Q .

arild-lab commented 3 years ago

Here;s a file without the extra sensor layout at the end--it still does not import wtht the latest MFF: https://drive.google.com/file/d/1bpE4xeWks5jWLRb8PqeIG_XMcfzqdUpC/view?usp=sharing

On Wed, Mar 17, 2021 at 4:05 PM Arild Hestvik @.***> wrote:

hi, ok, I checked a few more files and correct--they have multiple sensoy laouts--must have been an error during data collection. The issue disappears when segmenting in Netstation (which does not complain!). I will post a file that does not have the issue (but maybe that one will import without problem, I have to test). Arild

On Wed, Mar 17, 2021 at 3:54 PM Arild Hestvik @.***> wrote:

Hi Arnaud, that's weird--this is a set of 30 participants recorded on an EGI 200 system with GSN adult 65 vs.2 electrode net. There should be no switch to 129 channels at the end, unless something has happened somewhere in the path from the EGI format to matlab... Can I give you the original raw session files instead? Arild

On Wed, Mar 17, 2021 at 3:30 PM Arnaud Delorme @.***> wrote:

It is strange. It seems that the end of the file switch from 65 to 129 channels. The importer does not support this kind of file. It would be possible to import the beginning of the file only but this is such a special case, I do not think it is worth the effort.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sccn/eeglab/issues/263#issuecomment-801352352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATB6EHOVEV7K4GVQSXA5UCDTED7PFANCNFSM4YP77I7Q .

arnodelorme commented 3 years ago

Unfortunately, the event file has nonstandard characters (EEG_events_DIN_1.xml). This is a limitation of the EGI JAR library that EGI provides. Removing the special characters in a text editor fixes the issue.

https://drive.google.com/file/d/1bC7m5FRGTnQ20pRHQ-O3hJcD5gkSdH-q/view?usp=sharing

I cannot fix the issue. I would need to create a program to remove the special characters. I will let the EGI people know.