mne.io.read_raw_curry *Curry 8 Raw Float

[hannekevandijk]

Describe the bug

I have trouble loading data that has been saved with Curry 8 default format (CURRY Raw Float Format *.cdt). Since others have to do the EEG recordings and to make things on that end as fool-proof as possible I'd much prefer the default saving option. Doing this saves a .cdt file and a .cdt.dpa file, but no other files. Reading through the sourcecode of read_raw_curry, i guess mne expects a .cdt.cef or .cdt.ceo file as well.

"Curry 8": {
        "info": ".cdt.dpa",
        "data": ".cdt",
        "labels": ".cdt.dpa",
        "events_cef": ".cdt.cef",
        "events_ceo": ".cdt.ceo",
        "hpi": ".cdt.hpi"
}

I do have a trigger channel with events, but no hpi. When I try to load the data I get this error:

In [129]: raw = mne.io.read_raw_curry(eeg.files[2], preload=True)
Leaving device<->head transform as None (no landmarks found)
Reading 0 ... 5797887  =      0.000 ...  1415.500 secs...
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-129-6f9a2fd61bab> in <module>
----> 1 raw = mne.io.read_raw_curry(eeg.files[2], preload=True)

<decorator-gen-264> in read_raw_curry(fname, preload, verbose)

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/curry/curry.py in read_raw_curry(fname, preload, verbose)
    475         A Raw object containing Curry data.
    476     """
--> 477     return RawCurry(fname, preload, verbose)
    478 
    479 

<decorator-gen-265> in __init__(self, fname, preload, verbose)

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/curry/curry.py in __init__(self, fname, preload, verbose)
    508         raw_extras = dict(is_ascii=is_ascii)
    509 
--> 510         super(RawCurry, self).__init__(
    511             info, preload, filenames=[data_fname], last_samps=last_samps,
    512             orig_format='int', raw_extras=[raw_extras], verbose=verbose)

<decorator-gen-197> in __init__(self, info, preload, first_samps, last_samps, filenames, raw_extras, orig_format, dtype, buffer_size_sec, orig_units, verbose)

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/base.py in __init__(self, info, preload, first_samps, last_samps, filenames, raw_extras, orig_format, dtype, buffer_size_sec, orig_units, verbose)
    286         # If we have True or a string, actually do the preloading
    287         if load_from_disk:
--> 288             self._preload_data(preload)
    289         self._init_kwargs = _get_argvalues()
    290 

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/base.py in _preload_data(self, preload)
    566         logger.info('Reading %d ... %d  =  %9.3f ... %9.3f secs...' %
    567                     (0, len(self.times) - 1, 0., self.times[-1]))
--> 568         self._data = self._read_segment(
    569             data_buffer=data_buffer, projector=self._projector)
    570         assert len(self._data) == self.info['nchan']

<decorator-gen-199> in _read_segment(self, start, stop, sel, data_buffer, projector, verbose)

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/base.py in _read_segment(***failed resolving arguments***)
    451             # reindex back to original file
    452             orig_idx = _convert_slice(self._read_picks[fi][need_idx])
--> 453             _ReadSegmentFileProtector(self)._read_segment_file(
    454                 data[:, this_sl], orig_idx, fi,
    455                 int(start_file), int(stop_file), cals, mult)

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/base.py in _read_segment_file(self, data, idx, fi, start, stop, cals, mult)
   2111 
   2112     def _read_segment_file(self, data, idx, fi, start, stop, cals, mult):
-> 2113         return self.__raw.__class__._read_segment_file(
   2114             self, data, idx, fi, start, stop, cals, mult)
   2115 

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/curry/curry.py in _read_segment_file(self, data, idx, fi, start, stop, cals, mult)
    538 
    539         else:
--> 540             _read_segments_file(
    541                 self, data, idx, fi, start, stop, cals, mult, dtype="<f4")

/opt/anaconda/envs/tf20/lib/python3.9/site-packages/mne/io/utils.py in _read_segments_file(raw, data, idx, fi, start, stop, cals, mult, dtype, n_channels, offset, trigger_ch)
    220             block = np.fromfile(fid, dtype, count)
    221             if block.size != count:
--> 222                 raise RuntimeError('Incorrect number of samples (%s != %s), '
    223                                    'please report this error to MNE-Python '
    224                                    'developers' % (block.size, count))

RuntimeError: Incorrect number of samples (39818 != 24999975), please report this error to MNE-Python developers

So here you are :D

If I save the file as 'CURRY Raw ASCII Format (*.cdt)' strangly it does work. By the way, Curry then also saves the additional .cef file.

This is my version and (conda) environment information:

mne.sys_info()
Qt WebEngine seems to be initialized from a plugin. Please set Qt::AA_ShareOpenGLContexts using QCoreApplication::setAttribute before constructing QGuiApplication.
Platform:       Linux-4.15.0-156-generic-x86_64-with-glibc2.27
Python:         3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]
Executable:     /opt/anaconda/envs/tf20/bin/python
CPU:            x86_64: 32 cores
Memory:         62.9 GB

mne:            0.24.0
numpy:          1.19.5 {blas=openblas, lapack=openblas}
scipy:          1.5.4
matplotlib:     3.4.2 {backend=Qt5Agg}

sklearn:        0.24.2
numba:          0.54.0
nibabel:        Not found
nilearn:        Not found
dipy:           Not found
cupy:           Not found
pandas:         1.3.3
mayavi:         Not found
pyvista:        Not found
pyvistaqt:      Not found
ipyvtklink:     Not found
vtk:            Not found
PyQt5:          5.9.2
ipympl:         Not found
mne_qt_browser: Not found

[welcome[bot]]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[larsoner]

I have trouble loading data that has been saved with Curry 8 default format (CURRY Raw Float Format *.cdt). Since others have to do the EEG recordings and to make things on that end as fool-proof as possible I'd much prefer the default saving option.

Can you upload the problematic file somewhere so that someone can look? If you can also save it out in any format that MNE does support / can read and upload it to the same place, that would make it even easier to debug.

If there's any way you could take a ~1-2 sec recording (or crop to this duration) and export and upload a failing-format and working-format file, that would be ideal!

[agramfort]

@larsoner I just shared the files with you privately

[larsoner]

@larsoner I just shared the files with you privately

I think this is not quite enough to properly debug. I can't tell what the result is supposed to be from the broken files alone.

@hannekevandijk could you do a quick recording like I describe above? If not, could you at least also share with @agramfort the ASCII-output version of the bad file? That way we have one version that reads correctly (ASCII) and one that we need to fix (raw float). It helps to know the number of channels and samples and expected values, so being able to cross-check with something that does work is super helpful!

[hannekevandijk]

Hi Sorry for my extremely late reply... I have now found out that curry changed the extension from cdt.dpa to cdt.dpo, and cdt.cef is changed to cdt.ceo. Simply changing the extension before reading it with mne_python solves it....

[nestorDario]

Funciono cambiando la extensiones