kutaslab / mkpy

Kutas Lab data file utilities
https://kutaslab.github.io/mkpy
Other
1 stars 3 forks source link

ValueError: name already used as a name or title #61

Closed hanjiaxu closed 2 years ago

hanjiaxu commented 2 years ago

Hello, I am using the create_mkdata function but encountered the following error message. I wonder if you may have any insight on where the issue comes from. Thank you!

Code: sub00.create_mkdata('sub00', 'sub00.crw', 'sub00.x.log','sub00.yhdr')

Error Message:

ValueError Traceback (most recent call last) /var/folders/bg/18k6cn8s6w133ymk3m8lxmnh0000gn/T/ipykernel_2337/3503169862.py in ----> 1 sub00.create_mkdata('sub00', 'sub00.crw', 'sub00.x.log','sub00.yhdr')

~/opt/anaconda3/lib/python3.9/site-packages/mkpy-0.2.7-py3.9-macosx-10.9-x86_64.egg/mkpy/mkh5.py in create_mkdata(self, h5_path, eeg_f, log_f, yhdr_f, with_log_events, *args, **kwargs) 2355 yhdr_f = str(yhdr_f) 2356 -> 2357 (attr, data) = self._read_raw_log(eeg_f, log_f, with_log_events=with_log_events) 2358 2359 hio = mkh5.HeaderIO()

~/opt/anaconda3/lib/python3.9/site-packages/mkpy-0.2.7-py3.9-macosx-10.9-x86_64.egg/mkpy/mkh5.py in _read_raw_log(self, eeg_f, log_f, with_log_events) 3454 ) 3455 # mkh5 dblock dtype -> 3456 dt_data = np.dtype( 3457 {"names": dt_names, "formats": dt_formats, "titles": dt_titles} 3458 )

ValueError: name already used as a name or title

turbach commented 2 years ago

Hi, it could be something in the sub00.yhdr file, can you please upload a copy? If it has privileged information such as IRB human subject data please replace the values with XXXXX before uploading. What I need to see is the YAML document structure (---) and the YAML keys for the key: value maps.

hanjiaxu commented 2 years ago

Yes, and thank you so much! I created a very simple .yhdr file from an example I found (please see attached). sub00.yhdr.zip

turbach commented 2 years ago

Thanks that's fine. Hmm ... from error messages it looks like you are running conda on a Mac, I missed that the first time through. If so, no surprise there's trouble ... the mkpy conda package is built and tested on x86_64 linux only (https://kutaslab.github.io/mkpy/Installation.html). Can you try the same thing on a 64-bit Intel or AMD linux server?

hanjiaxu commented 2 years ago

Thank you for your response! I just tried it in a Linux subsystem, but I run into the same error message. Maybe I didn't create .yhdr file correctly?


ValueError Traceback (most recent call last) Cell In [3], line 1 ----> 1 sub00.create_mkdata('sub00', 'sub00.crw', 'sub00.x.log','sub00.yhdr')

File ~/anaconda3/envs/mkpy_28092022/lib/python3.9/site-packages/mkpy/mkh5.py:2357, in mkh5.create_mkdata(self, h5_path, eeg_f, log_f, yhdr_f, with_log_events, *args, **kwargs) 2354 log_f = str(log_f) 2355 yhdr_f = str(yhdr_f) -> 2357 (attr, data) = self._read_raw_log(eeg_f, log_f, with_log_events=with_log_events) 2359 hio = mkh5.HeaderIO() 2360 hio.new(attr, yhdr_f) # merge the .crw and yhdr into the new header

File ~/anaconda3/envs/mkpy_28092022/lib/python3.9/site-packages/mkpy/mkh5.py:3456, in mkh5._read_raw_log(self, eeg_f, log_f, with_log_events) 3452 dt_titles.extend( 3453 ["digchan{0:04d}".format(n[0]) for n in enumerate(channel_names)] 3454 ) 3455 # mkh5 dblock dtype -> 3456 dt_data = np.dtype( 3457 {"names": dt_names, "formats": dt_formats, "titles": dt_titles} 3458 ) 3460 # load eeg streams and build the crw_tick index 3461 data = np.zeros((len(raw_evcodes),), dtype=dt_data)

ValueError: name already used as a name or title

turbach commented 2 years ago

The yhdr looks OK and thanks, the complete error messages are super helpful. Now it is reporting duplicate channel names in the .crw file (dig_chan). Seems unlikely but can you run this linux command, redact any personally identifiable information and upload the output:

$headinfo sub00.crw

turbach commented 2 years ago

Update: According to the Action CI, the current v0.2.7 mkpy conda package seems to be installing correctly and passing all the IO pytests with Python 3.7, 3.8 and 3.9.

If you replace your sub00.crw and sub00.log with our test files and still get this same Value Error I would suspect an issue with your mkpy conda environment/installation. If not, then something about your sub00.crw file.

You can download our .crw and .log test files from here in tests/data.

hanjiaxu commented 2 years ago

Thank you so much for pointing me to the test files - they are super helpful! The test files work well, so I looked into the .crw file, and it seems that we do have duplicate channel names. Here is the output from headinfo:

subdesc: subject dyam00 expdesc: batch1 channels: 8 chndesc: 0:test 1:test 2:HE 3:MiPf
4:MiCe 5:MiPa 6:MiOc 7:A2

data type: epoch length: 0 ms data resolution (points per 10 uV): 0 presampling: 0 ms sampling rate: 250.00 Hz channel precision: 1

turbach commented 2 years ago

OK nicely done and that's good news. Now my guess is that your mkpy installation is behaving properly and reading your .crw files correctly and then throwing the error when it encounters the duplicate channel names.

The next step is to make a working copy of your sub00.crw and use the edhead linux command to revise one or both of the duplicate channel names to make them unique (verify with headinfo).

I'm cautiously optimistic mkpy will be happy with that. If so, you can update existing .crw files the same way and revise the dig header to record new .crws with unique channel names in future.

hanjiaxu commented 2 years ago

Thank you so much for all the help! I changed the channel name using edhead and now I can use mkpy to load my data!

turbach commented 2 years ago

Great, glad to hear, I'll close this issue.