cannnot read nihon kohden .EEG file by mne python

jordandc1 commented 3 years ago

Describe the bug

I recently got a .eeg file from a hospital, but I cannot read it by MNE-python. I noticed that the version (EEG-1200A V01.00) is not supported at this moment. Can anyone help with this problem?

Steps to reproduce

import matplotlib
import pathlib
import mne

sample_data_dir = mne.datasets.sample.data_path()
raw_path = sample_data_dir / 'MEG' / 'sample' / 'FA0015AA.EEG'
raw = mne.io.read_raw_nihon(raw_path)
print(raw.info)
raw.plot()

Expected results

The .EEG data should be loaded.

Actual results

Loading FA0015AA.EEG
Reading header from C:\Users\Surface\mne_data\MNE-sample-data\MEG\sample\FA0015AA.EEG
Traceback (most recent call last):
  File "C:\Users\Surface\PycharmProjects\mne_plot\plot_test.py", line 13, in <module>
    raw = mne.io.read_raw_nihon(raw_path)
  File "C:\Users\Surface\PycharmProjects\mne_plot\venv\lib\site-packages\mne\io\nihon\nihon.py", line 46, in read_raw_nihon
    return RawNihon(fname, preload, verbose)
  File "<decorator-gen-243>", line 24, in __init__
  File "C:\Users\Surface\PycharmProjects\mne_plot\venv\lib\site-packages\mne\io\nihon\nihon.py", line 316, in __init__
    header = _read_nihon_header(fname)
  File "C:\Users\Surface\PycharmProjects\mne_plot\venv\lib\site-packages\mne\io\nihon\nihon.py", line 136, in _read_nihon_header
    raise ValueError(
ValueError: Not a valid Nihon Kohden EEG file (EEG-1200A V01.00)

Process finished with exit code 1

Additional information

Platform: Windows-10-10.0.19041-SP0
Python: 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)]
Executable: c:\users\surface\anaconda3\python.exe
CPU: Intel64 Family 6 Model 126 Stepping 5, GenuineIntel: 8 cores
Memory: 15.6 GB
mne: 0.23.0
numpy: 1.20.1 {blas=mkl_rt, lapack=mkl_rt}
scipy: 1.6.2
matplotlib: 3.3.4 {backend=Qt5Agg}
sklearn: 0.24.1
numba: 0.53.1
nibabel: 3.2.1
nilearn: Not found
dipy: Not found
cupy: Not found
pandas: 1.2.4
mayavi: 4.7.3
pyvista: Not found
vtk: 9.0.1
PyQt5: 5.15.4

welcome[bot] commented 3 years ago

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

agramfort commented 3 years ago

can you see with @fraimondo who wrote this code? maybe share the file with him privately?

fraimondo commented 3 years ago

This file version is currently not supported. Brainstorm currently supports this kind of files: https://github.com/brainstorm-tools/brainstorm3/blob/master/toolbox/io/in_fopen_nk.m

I guess we might be able to use the brainstorm code to figure out the file format. But I don't know what happens with the license in that case (brainstorm is GPLv3)

larsoner commented 3 years ago

I guess we might be able to use the brainstorm code to figure out the file format. But I don't know what happens with the license in that case (brainstorm is GPLv3)

You cannot read this code to then implement things in MNE without explicit permission from the Brainstorm folks to relicense this part of their code (hopefully just one file?) under MNE's BSD license. Maybe they would grant permission to do so if you asked very nicely in an email :)

fraimondo commented 3 years ago

That was my point. BSD is not compatible with GPLv3, so we'll need their permissions. In any case, it can't be done quickly. At least I don't have the time right now.

zhengliuer commented 3 years ago

Hi, as I know, Neuro Workbench supports convert EEG files to EDF format.

leticiabragas2 commented 3 years ago

Not a valid Nihon Kohden EEG file (d) Help

ValueError Traceback (most recent call last)

in ----> 1 raw = mne.io.read_raw_nihon('DA0935FF.EEG')#lê o arquivo 2 print(raw.info)#printa informações sobre o arquivo 3 # raw.crop(20).load_data() ~/.local/lib/python3.8/site-packages/mne/io/nihon/nihon.py in read_raw_nihon(fname, preload, verbose) 44 mne.io.Raw : Documentation of attribute and methods. 45 """ ---> 46 return RawNihon(fname, preload, verbose) 47 48 in __init__(self, fname, preload, verbose) ~/.local/lib/python3.8/site-packages/mne/io/nihon/nihon.py in __init__(self, fname, preload, verbose) 314 logger.info('Loading %s' % data_name) 315 --> 316 header = _read_nihon_header(fname) 317 metadata = _read_nihon_metadata(fname) 318 ~/.local/lib/python3.8/site-packages/mne/io/nihon/nihon.py in _read_nihon_header(fname) 134 version = np.fromfile(fid, '|S16', 1).astype('U16')[0] 135 if version not in _valid_headers: --> 136 raise ValueError( 137 'Not a valid Nihon Kohden EEG file ({})'.format(version)) 138 ValueError: Not a valid Nihon Kohden EEG file (d)

MrNobodyCali commented 3 years ago

Hi, as I know, Neuro Workbench supports convert EEG files to EDF format.

Hi, though Neuro Workbench supports convert EEG files to EDF format, the length of the label string seems to be limited. The maximum length of label string in Neuro Workbench is more than EDF files or the export process causes the problem. Do you have any insight or solution to this problem?

mclenk commented 1 year ago

@fraimondo Any luck on this issue? I'm currently encountering the same problem :/

jordandc1 commented 1 year ago

@fraimondo Any luck on this issue? I'm currently encountering the same problem :/

You can use Neuro Workbench to convert EEG files to EDF format.

zhengliuer commented 1 year ago

No, jusr export as edf using the official software.

获取Outlook for Androidhttps://aka.ms/AAb9ysg

From: CHAO DU @.> Sent: Thursday, January 19, 2023 10:10:56 AM To: mne-tools/mne-python @.> Cc: ZhengLiu97 @.>; Comment @.> Subject: Re: [mne-tools/mne-python] cannnot read nihon kohden .EEG file by mne python (#9462)

@fraimondohttps://github.com/fraimondo Any luck on this issue? I'm currently encountering the same problem :/

You can use Neuro Workbench to convert EEG files to EDF format.

― Reply to this email directly, view it on GitHubhttps://github.com/mne-tools/mne-python/issues/9462#issuecomment-1396352282, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APQZLWK35LL3QAAOJ7CPGITWTCPDBANCNFSM46PWO6WQ. You are receiving this because you commented.Message ID: @.***>

jacobshaw42 commented 1 year ago

Hello, I am was having this issue and I think after making a few small changes to the package, I was able to load the data correctly from EEG-1200A V01.00

Changes: 1) at line 59, uncommented the version https://github.com/mne-tools/mne-python/blob/main/mne/io/nihon/nihon.py

2) My PNT file had a lowercase suffix, so at line 67 I added a condition to change to lowercase if the uppercase suffix was not found. https://github.com/mne-tools/mne-python/blob/main/mne/io/nihon/nihon.py

After that, the only thing that did not load well was the annotations. It appeared to be the they were "out of data range". I tracked this to the crop method of the BaseRaw object.

3) After commenting out the line 634 and 640 https://github.com/mne-tools/mne-python/blob/main/mne/io/base.py, it appears to load without errors.

I created a fresh python 3.9.8 environment and only installed mne and pandas.

This appears to load, can anyone tell me if I am missing something?

drammock commented 1 year ago

My PNT file had a lowercase suffix

This seems fine, we should support both upper/lower case file extensions

It appeared to be the [annotations] were "out of data range" [...] After commenting out the line 634 and 640 it appears to load without errors.

Yes, but are the annotations visible / in the correct location?

jacobshaw42 commented 1 year ago

Yes, but are the annotations visible / in the correct location?

After loading, I apply the to_data_frame method on the _annotations and I get a dataframe that shows visible onsets, durations (all 0.0) and descriptions that appear how I would expect.

drammock commented 1 year ago

So the onset times are correct? Do you have some gold standard to compare them to? (like a log file from the experiment's stimulus presentation code for example)? We likely can't implement your fix as proposed because changes to base.py will affect all file types (not just Nihon Koden) and simply removing the crop lines will almost certainly cause problems (hopefully indicated by failed unit tests). So I'm trying to figure out if the file is successfully loading with correct or incorrect annotation onset times. If they're incorrect, we need to figure out what the correct offset is, and shift them. If they're correct, we need to alter our code to avoid the cropping step for this type of file (or possibly it's revealed a bug that we should fix).

jacobshaw42 commented 1 year ago

When I read the log file as bytes and remove all \x00, the onsets seem reasonable.

I am waiting on a response from those who provided me with the files, as to whether those onsets are correct.

Update: I have confirmed everything looks correct. However, there are apparently "sub logs" that I am also needing to extract from the same .LOG file. This does not appear to be part of the current reading of the annotations.

Second Update: I was able to get the "sub log", but to do this, I did get some inspiration from the one of the brainstorm MATLAB functions.

@drammock - If any of this is acceptable to be added to the package, please let me know and I can share.

JohnAtl commented 1 year ago

Any update on this? The function mne.io.read_raw_nihon() seems to read NK .eeg files correctly (at least without error), yet mne.io.read_raw() lists .eeg as an unsupported type.

cbrnr commented 1 year ago

mne.io.read_raw() is a convenience wrapper, where we try to guess the correct reader from the given file extension. This is not always possible, because for .eeg we have a conflict with the BrainVision format, which also uses that extension. We could add the NK reader though, because people are supposed to pass the .vhdr file for BrainVision. WDYT? I could add that quickly in #11521.

JohnAtl commented 1 year ago

Sounds like a plan! Thanks!It would also be possible to give the file a “sniff test” and see if the NK signature is near the beginning. Or people could just use .vhdr if that’s what they are supposed to do.On Mar 2, 2023, at 11:28 AM, Clemens Brunner @.***> wrote: mne.io.read_raw() is a convenience wrapper, where we try to guess the correct reader from the given file extension. This is not always possible, because for .eeg we have a conflict with the BrainVision format, which also uses that extension. We could add the NK reader though, because people are supposed to pass the .vhdr file for BrainVision. WDYT? I could add that quickly in #11521.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

cbrnr commented 1 year ago

Yes, sniffing for magic bytes (at the beginning of a file) would be a nice extension. For now, I have enabled the NK format in #11521.

LakshTomar commented 4 months ago

still encounter the same issue is there any solution possible to open these .eeg files

mne-tools / mne-python