Please support the files saved in China (currently only ['en_US.utf8', 'de_DE', 'fr_FR', 'it_IT')

[PullGuy]

READ THIS FIRST: If you are having trouble getting MNE-Python to work with your own data, you should ask for help on one of our other channels:

• email list
• Gitter (chat)

Our GitHub issue tracker is only used to report bugs and suggest improvements to MNE-Python. For any other questions, please use the email list or Gitter. Usage questions that are posted as GitHub issues are usually closed without being answered. See the FAQ entry on filing bug reports for more guidance. If you're pretty sure your problem is a bug in MNE-Python, please delete this section and fill in the headings below, replacing the placeholder text with your own responses. Surround any code samples with triple backticks above and below the code block (see the GitHub guide to markdown for help with issue formatting). Alternatively, you can put your MWE in a public gist and link to it in this issue.

Describe the bug

I'm in China, and I used a NIRSport device to collect fNIRs data, where the date information is in a Chinese language format. When I read data like this: data = mne.io.read_raw_nirx(r'C:UsersDuckDocumentsGithubfNIRs-Data-ProcessSubject1') Python returns an error message:

RuntimeWarning: Extraction of measurement date from NIRX file failed. This can be caused by files saved in certain locales (currently only ['en_US.utf8', 'de_DE', 'fr_FR', 'it_IT'] supported). Please report this as a github issue. The date is being set to January 1st, 2000, instead of '"2021Äê7ÔÂ3ÈÕ""16:33:23.262"'.

I looked at the *.hdr file where the date information is：Date="2021年7月3日"

It also gives the following error:

`ValueError Traceback (most recent call last) Cell In[47], line 1 ----> 1 data = read_raw_nirx(r"c:\Users\DUCK\Documents\Github\fNIRs-Data-Process\Subject1").load_data()

File :12, in load_data(self, verbose)

File c:\Users\DUCK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:592, in BaseRaw.load_data(self, verbose) 573 """Load raw data. 574 575 Parameters (...) 589 .. versionadded:: 0.10.0 590 """ 591 if not self.preload: --> 592 self._preload_data(True) 593 return self

File c:\Users\DUCK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:604, in BaseRaw._preload_data(self, preload) 599 data_buffer = None 600 logger.info( 601 "Reading %d ... %d = %9.3f ... %9.3f secs..." 602 % (0, len(self.times) - 1, 0.0, self.times[-1]) 603 ) --> 604 self._data = self._read_segment(data_buffer=data_buffer) ... --> 566 x.shape = (stop - start, -1) 567 x = x[:, cols] 568 return x

ValueError: cannot reshape array of size 1 into shape (18885,newaxis)`

Steps to reproduce

Expected results

I want it to read the data correctly and do something else

Actual results

Replace this text with the actual output, traceback, screenshot, or other description of the results.

Additional information

[drammock]

thank you for reporting, I'll try to fix this week.

[drammock]

Through a bit of trial and error I was able to figure out what encoding is used in your file:

In [1]: foo = "2021Äê7ÔÂ3ÈÕ""16:33:23.262"  # what you pasted in the GitHub issue
In [2]: foo.encode("windows-1252").decode("GB18030")
Out[2]: '2021年7月3日16:33:23.262'

I think probably allowing the user to specify what encoding is used will be part of the solution, and the other part is making MNE smarter about parsing a measurement date --- i.e., making it know what 年 (year), 月 (month), and 日 (day) mean.

It would be helpful to have the full traceback, you've omitted some steps here:

--> 604 self._data = self._read_segment(data_buffer=data_buffer)
...
--> 566 x.shape = (stop - start, -1)

Even more helpful would be to have a link to the file, if you're able to share it.

[PullGuy]

OK, the data file is attached here. Subject1.zip

In my opinion, a better solution would be to read the *.nirs file directly, as in MATLAB, and the extraction of date information should not affect other data processing that followed.

and, why does this error happen? ValueError: cannot reshape array of size 1 into shape (18885,newaxis)，Is it because of a date information processing error?

[PullGuy]

The Full TraceBack are here:

ValueError Traceback (most recent call last) Cell In[27], line 1 ----> 1 data.load_data()

File :12, in load_data(self, verbose)

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:592, in BaseRaw.load_data(self, verbose) 573 """Load raw data. 574 575 Parameters (...) 589 .. versionadded:: 0.10.0 590 """ 591 if not self.preload: --> 592 self._preload_data(True) 593 return self

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:604, in BaseRaw._preload_data(self, preload) 599 data_buffer = None 600 logger.info( 601 "Reading %d ... %d = %9.3f ... %9.3f secs..." 602 % (0, len(self.times) - 1, 0.0, self.times[-1]) 603 ) --> 604 self._data = self._read_segment(data_buffer=data_buffer) 605 assert len(self._data) == self.info["nchan"] 606 self.preload = True

File :12, in _read_segment(self, start, stop, sel, data_buffer, verbose)

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:480, in BaseRaw._read_segment(failed resolving arguments) 478 # reindex back to original file 479 orig_idx = _convert_slice(self._read_picks[fi][need_idx]) --> 480 _ReadSegmentFileProtector(self)._read_segment_file( 481 data[:, this_sl], 482 orig_idx, 483 fi, 484 int(start_file), 485 int(stop_file), 486 cals, 487 mult, 488 ) 489 offset += n_read 490 return data

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\base.py:2623, in _ReadSegmentFileProtector._read_segment_file(self, data, idx, fi, start, stop, cals, mult) 2622 def _read_segment_file(self, data, idx, fi, start, stop, cals, mult): -> 2623 return self.raw.class__._read_segment_file( 2624 self, data, idx, fi, start, stop, cals, mult 2625 )

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\nirx\nirx.py:534, in RawNIRX._read_segment_file(self, data, idx, fi, start, stop, cals, mult) 532 wls = list() 533 for key in ("wl1", "wl2"): --> 534 d = _read_csv_rows_cols( 535 self._raw_extras[fi]["files"][key], 536 start, 537 stop, 538 sd_index, 539 self._raw_extras[fi]["bounds"][key], 540 ).T 541 nan_mask = self._raw_extras[fi]["nan_mask"].get(key, None) 542 if nan_mask is not None:

File c:\Users\CK\AppData\Local\Programs\Python\Python311\Lib\site-packages\mne\io\nirx\nirx.py:566, in _read_csv_rows_cols(fname, start, stop, cols, bounds, sep, replace) 564 data = replace(data) 565 x = np.fromstring(data, float, sep=sep) --> 566 x.shape = (stop - start, -1) 567 x = x[:, cols] 568 return x

ValueError: cannot reshape array of size 1 into shape (18885,newaxis)

[drammock]

sorry for the long wait on this one! With MNE-Python main (and with the 1.8 release coming out next month), it will be possible to pass read_raw_nirx(..., encoding="GB18030") to read these files successfully.

Regarding your second question about ValueError: cannot reshape array of size 1 into shape (18885,newaxis) --- I cannot replicate that error happening with the zipped version of the file you shared. If you're still experiencing that problem on MNE-Python current main (or on 1.8 after it comes out), please open a new issue about it.