Issue with importing EEGLAB .set file

[moonj94]

Describe the bug

I'm trying to load some data from a .set file into python and I keep getting an error saying 'index out of range'. This problem exists for both read_raw_eeglab and read_epochs_eeglab. The problem exists regardless of whether or not there is a .fdt file in the folder.

Steps to reproduce

Download data from https://openneuro.org/datasets/ds003061/versions/1.1.0

run mne.io.read_epochs_eeglab(path)

Expected results

File should be loaded

Actual results

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
c:\Users\jaemoon.LARKGROUP\OneDrive - Lark Group\Desktop\python\forward_problem\openneuro.py in <module>
     [95](file:///c%3A/Users/jaemoon.LARKGROUP/OneDrive%20-%20Lark%20Group/Desktop/python/forward_problem/openneuro.py?line=94)         if d.endswith('.set'):
     [96](file:///c%3A/Users/jaemoon.LARKGROUP/OneDrive%20-%20Lark%20Group/Desktop/python/forward_problem/openneuro.py?line=95)           pathd=r'%s\%s' % (path,d)
---> [97](file:///c%3A/Users/jaemoon.LARKGROUP/OneDrive%20-%20Lark%20Group/Desktop/python/forward_problem/openneuro.py?line=96)           tmp=mne.io.read_epochs_eeglab(pathd)
     [98](file:///c%3A/Users/jaemoon.LARKGROUP/OneDrive%20-%20Lark%20Group/Desktop/python/forward_problem/openneuro.py?line=97) 

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\mne\io\eeglab\eeglab.py in read_epochs_eeglab(input_fname, events, event_id, eog, verbose, uint16_codec)
    275     epochs = EpochsEEGLAB(input_fname=input_fname, events=events, eog=eog,
    276                           event_id=event_id, verbose=verbose,
--> 277                           uint16_codec=uint16_codec)
    278     return epochs
    279 

<decorator-gen-262> in __init__(self, input_fname, events, event_id, tmin, baseline, reject, flat, reject_tmin, reject_tmax, eog, verbose, uint16_codec)

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\mne\io\eeglab\eeglab.py in __init__(self, input_fname, events, event_id, tmin, baseline, reject, flat, reject_tmin, reject_tmax, eog, verbose, uint16_codec)
    449         input_fname = _check_fname(fname=input_fname, must_exist=True,
    450                                    overwrite='read')
--> 451         eeg = _check_load_mat(input_fname, uint16_codec)
    452 
    453         if not ((events is None and event_id is None) or

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\mne\io\eeglab\eeglab.py in _check_load_mat(fname, uint16_codec)
     57     """Check if the mat struct contains 'EEG'."""
     58     from ...externals.pymatreader import read_mat
---> 59     eeg = read_mat(fname, uint16_codec=uint16_codec)
     60     if 'ALLEEG' in eeg:
     61         raise NotImplementedError(

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\mne\externals\pymatreader\pymatreader.py in read_mat(filename, variable_names, ignore_fields, uint16_codec)
     82     try:
     83         with open(filename, 'rb') as fid:  # avoid open file warnings on error
---> 84             mjv, _ = get_matfile_version(fid)
     85             extra_kwargs = {}
     86             if mjv == 1:

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\scipy\io\matlab\miobase.py in get_matfile_version(fileobj)
    223     tst_str = fileobj.read(4)
    224     fileobj.seek(0)
--> 225     maj_ind = int(tst_str[2] == b'I'[0])
    226     maj_val = int(tst_str[maj_ind])
    227     min_val = int(tst_str[1 - maj_ind])

IndexError: index out of range

Additional information

Platform: Windows-10-10.0.19041-SP0 Python: 3.7.7 (default, May 6 2020, 11:45:54) [MSC v.1916 64 bit (AMD64)] Executable: c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\python.exe CPU: Intel64 Family 6 Model 165 Stepping 2, GenuineIntel: 16 cores Memory: 31.7 GB

mne: 0.24.0 numpy: 1.21.6 {blas=NO_ATLAS_INFO, lapack=lapack} scipy: 1.6.2 matplotlib: 3.3.2 {backend=module://matplotlib_inline.backend_inline}

sklearn: 0.23.1 numba: 0.53.1 nibabel: 3.2.2 nilearn: 0.9.0 dipy: 1.3.0 cupy: Not found pandas: 1.3.4 mayavi: 4.7.2 pyvista: 0.33.3 {OpenGL 4.5.0 - Build 27.20.100.9316 via Intel(R) UHD Graphics} pyvistaqt: 0.9.0 ipyvtklink: Not found vtk: 9.0.1 PyQt5: unknown ipympl: Not found mne_qt_browser: Not found

[drammock]

The last line of the traceback:

c:\Users\jaemoon.LARKGROUP\Anaconda3\envs\nc4\lib\site-packages\scipy\io\matlab\miobase.py in get_matfile_version(fileobj)
    223     tst_str = fileobj.read(4)
    224     fileobj.seek(0)
--> 225     maj_ind = int(tst_str[2] == b'I'[0])
    226     maj_val = int(tst_str[maj_ind])
    227     min_val = int(tst_str[1 - maj_ind])

says the error is occurring in scipy.io.matlab.miobase.py. So either this is a bug in scipy (not MNE-Python) or there is something wrong with the file. Can you try reading this .set file in EEGLAB and make sure it's actually readable / correctly formatted / not corrupted?

[moonj94]

Thanks for getting back to me @drammock. It looks like the way in which the files were downloaded (datalad) led to the corruption of the files. When I download a file individually I no longer get the problem. Sorry about the commotion.

Do you know of an efficient way of downloading such files from the web via a python script/function?

[drammock]

if datalad is corrupting files when downloading them, you should report that bug to datalad. Meanwhile you can try https://github.com/hoechenberger/openneuro-py