Issue loading NIRx dyads with mne-0.24.0

[claytonjschneider]

I'm not sure if this is possible, but I think the version of mne this package relies on should be updated to mne-1.0.3. NIRx files cannot be loaded in this outdated version (for me at least). The sfreq attribute of raw.info is read as a numpy.float64 object, which causes an error in mne-0.24.0.

[claytonjschneider]

Upgrading mne back to 1.0 also gives me an error: No module named 'mne.externals' on import of hypyp.fnirs_tools.load_fnirs

[katied-neuro]

Would you happen to have sample of the data (or point to an open repository of similar data, like same NIRx system) you are trying to load?

[claytonjschneider]

2020-03-05_001.zip

[ghazalehran]

Dear @claytonjschneider

Thanks for your feedback, and thanks for sharing your data as well. I just ran the code with your data and did not face the bug you mentioned; here is snip of my code:

Is it similar to yours?

[claytonjschneider]

That's exactly what I'm running. Hm. I think I will try with a new virtual environment and fresh install of HyPyP. Just for reference, here's the error I get when I run that code:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-0b66dfea9451> in <module>
      9     print(sub2)
     10 
---> 11     fnirs_participant_1 = load_fnirs(sub1, sub2, attr=None, preload=False, verbose=None)[0]
     12     fnirs_participant_2 = load_fnirs(sub1, sub2, attr=None, preload=False, verbose=None)[1]
     13 

~/.local/lib/python3.7/site-packages/hypyp/fnirs_tools.py in load_fnirs(path1, path2, attr, preload, verbose)
     48   elif os.path.isdir(path1):
     49     if attr is None:
---> 50       data_1 = mne.io.read_raw_nirx(path1, saturated='annotate', preload=preload, verbose=verbose)
     51       data_2 = mne.io.read_raw_nirx(path2, saturated='annotate', preload=preload, verbose=verbose)
     52     else:

~/.local/lib/python3.7/site-packages/mne/io/nirx/nirx.py in read_raw_nirx(fname, saturated, preload, verbose)
     48     %(nirx_notes)s
     49     """
---> 50     return RawNIRX(fname, saturated, preload, verbose)
     51 
     52 

<decorator-gen-258> in __init__(self, fname, saturated, preload, verbose)

~/.local/lib/python3.7/site-packages/mne/io/nirx/nirx.py in __init__(self, fname, saturated, preload, verbose)
    434                 ch_names.append(list())
    435         annot = Annotations(onset, duration, description, ch_names=ch_names)
--> 436         self.set_annotations(annot)
    437 
    438     def _read_segment_file(self, data, idx, fi, start, stop, cals, mult):

<decorator-gen-201> in set_annotations(self, annotations, emit_warning, on_missing, verbose)

~/.local/lib/python3.7/site-packages/mne/io/base.py in set_annotations(self, annotations, emit_warning, on_missing, verbose)
    690             new_annotations._prune_ch_names(self.info, on_missing)
    691             if annotations.orig_time is None:
--> 692                 new_annotations.crop(0, self.times[-1] + delta,
    693                                      emit_warning=emit_warning)
    694                 new_annotations.onset += self._first_time

~/.local/lib/python3.7/site-packages/mne/io/base.py in times(self)
   1577     def times(self):
   1578         """Time points."""
-> 1579         out = _arange_div(self.n_times, float(self.info['sfreq']))
   1580         out.flags['WRITEABLE'] = False
   1581         return out

TypeError: expected dtype object, got 'numpy.dtype[float64]'

[claytonjschneider]

Hi all,

I've been trying to debug this for much of the day, still to no avail. I'm going to include as much as I can in this comment. I assumed the problem was from my existing, newer install of mne (1.0), so I created a new venv and only installed hypyp on it. I also had to install PyQt5 separately, I think the 'wheel was broken'? Dont' quite remember the error.

I confirmed that I am able to load my data with mne 1.0, but for some reason the hypyp.fnirs_tools.load_fnirs function breaks during its call to mne.io.read_raw_nirx.

>>> python_version()
3.7.5
>>> import hypyp
>>> hypyp.__version__
0.4.0b2
>>> import mne
>>> mne.__version__
0.24.0
>>> from hypyp.fnirs_tools import load_fnirs
>>> from hypyp.fnirs_tools import make_fnirs_montage
>>> from hypyp.fnirs_tools import fnirs_epoch
>>> from hypyp.fnirs_tools import fnirs_montage_ui

then I set up my data, directory, and

fnirs_participant_1 = load_fnirs(sub1, sub2, attr=None, preload=False, verbose=None)

when I run into this error.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-416440b8499f> in <module>
      9     print(sub2)
     10 
---> 11     fnirs_participant_1 = load_fnirs(sub1, sub2, attr=None, preload=False, verbose=None)[0]
     12     fnirs_participant_2 = load_fnirs(sub1, sub2, attr=None, preload=False, verbose=None)[1]
     13 

~/.local/lib/python3.7/site-packages/hypyp/fnirs_tools.py in load_fnirs(path1, path2, attr, preload, verbose)
     48   elif os.path.isdir(path1):
     49     if attr is None:
---> 50       data_1 = mne.io.read_raw_nirx(path1, saturated='annotate', preload=preload, verbose=verbose)
     51       data_2 = mne.io.read_raw_nirx(path2, saturated='annotate', preload=preload, verbose=verbose)
     52     else:

~/.local/lib/python3.7/site-packages/mne/io/nirx/nirx.py in read_raw_nirx(fname, saturated, preload, verbose)
     48     %(nirx_notes)s
     49     """
---> 50     return RawNIRX(fname, saturated, preload, verbose)
     51 
     52 

<decorator-gen-258> in __init__(self, fname, saturated, preload, verbose)

~/.local/lib/python3.7/site-packages/mne/io/nirx/nirx.py in __init__(self, fname, saturated, preload, verbose)
    434                 ch_names.append(list())
    435         annot = Annotations(onset, duration, description, ch_names=ch_names)
--> 436         self.set_annotations(annot)
    437 
    438     def _read_segment_file(self, data, idx, fi, start, stop, cals, mult):

<decorator-gen-201> in set_annotations(self, annotations, emit_warning, on_missing, verbose)

~/.local/lib/python3.7/site-packages/mne/io/base.py in set_annotations(self, annotations, emit_warning, on_missing, verbose)
    690             new_annotations._prune_ch_names(self.info, on_missing)
    691             if annotations.orig_time is None:
--> 692                 new_annotations.crop(0, self.times[-1] + delta,
    693                                      emit_warning=emit_warning)
    694                 new_annotations.onset += self._first_time

~/.local/lib/python3.7/site-packages/mne/io/base.py in times(self)
   1577     def times(self):
   1578         """Time points."""
-> 1579         out = _arange_div(self.n_times, float(self.info['sfreq']))
   1580         out.flags['WRITEABLE'] = False
   1581         return out

TypeError: expected dtype object, got 'numpy.dtype[float64]'

I was brought to this page https://mne.discourse.group/t/expected-dtype-object-got-numpy-dtype-float64/3151 from searching, as it has apparently caught at least one other with mne version 0.24. So what I want to know is, is this the version of mne you are using @ghazalehran ? It seems like it must be an environment problem, since you're able to load the data, but hypyp and PyQt5 are the only modules I installed at all. Is the dependency not correct in the requirements.txt file?

[claytonjschneider]

@ghazalehran when you have a moment could you please share your environment's pip freeze or some equivalent? I am assuming this is an environment issue. Here's mine below, after making a new venv and installing HyPyP, PyQt5, and ipykernel.

astropy==4.1
autoreject==0.2.2
backcall==0.2.0
cycler==0.11.0
decorator==5.1.1
entrypoints==0.4
hypyp==0.2.0a4
importlib-metadata==4.8.3
importlib-resources==5.4.0
ipykernel==5.5.6
ipython==7.16.3
ipython-genutils==0.2.0
jedi==0.17.2
joblib==0.14.1
jupyter-client==7.1.2
jupyter-core==4.9.2
kiwisolver==1.3.1
llvmlite==0.36.0
matplotlib==3.3.4
meshio==4.4.6
mne==0.20.8
nest-asyncio==1.5.5
numba==0.53.1
numpy==1.19.5
pandas==1.1.5
parso==0.7.1
patsy==0.5.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.4.0
pkg_resources==0.0.0
prompt-toolkit==3.0.29
ptyprocess==0.7.0
Pygments==2.12.0
pyparsing==3.0.9
PyQt5==5.15.6
PyQt5-Qt5==5.15.2
PyQt5-sip==12.9.1
PySide2==5.15.2.1
python-dateutil==2.8.2
pytz==2022.1
pyzmq==23.1.0
scikit-learn==0.22.2.post1
scipy==1.5.4
shiboken2==5.15.2.1
six==1.16.0
statsmodels==0.11.1
tornado==6.1
tqdm==4.64.0
traitlets==4.3.3
typing_extensions==4.1.1
wcwidth==0.2.5
zipp==3.6.0

[claytonjschneider]

It has turned out to be an environment issue. Our lab server is heavily managed and so I often have trouble getting access to a specific version of Python and meeting package requirements. That was the case here, and once upgrading both my Python version, pip, and then installing h5io post-hoc (version may be broken in requirements.txt? unsure.) I have successfully loaded data in. My MNE version is now >1.0 and I believe this problem was caused by my Python v<3.7, which would prevent me from loading a newer release of MNE. Excited to get to work with this toolbox!