catalystneuro / nwb-conversion-tools

Create NWB files by converting and combining neural data in proprietary formats and adding essential metadata.
https://nwb-conversion-tools.readthedocs.io/en/main/
BSD 3-Clause "New" or "Revised" License
25 stars 12 forks source link

[neuralynx] Exploit metadata exposed from neo neuralynxrawio #596

Closed JuliaSprenger closed 1 year ago

JuliaSprenger commented 2 years ago

Motivation

The complete neuralynx files (including the headers) are parsed by the neo neuralynxio and do not need to be loaded and parsed again using an alternative approach here. This PR benefits from the changes in https://github.com/NeuralEnsemble/python-neo/pull/1137, but also works with older versions of neo.

How to test the behavior?

Check the NWB metadata when loading any neuralynx dataset. The essential session start time and identifier should be the same as before.

Checklist

JuliaSprenger commented 2 years ago

Currently the tests fail as additional metadata are not permitted to be added to the set of NWB metadata. Are there any fields these additional metadata should be written to? Or should I permit additional metadata keys for the validation?

bendichter commented 2 years ago

@JuliaSprenger thanks for putting this together! We are usually interested in a few metadata fields. It is not our intention to read all of the header as it is for neo. I'll go through and make adjustments to this PR

JuliaSprenger commented 2 years ago

It is not our intention to read all of the header as it is for neo. I'll go through and make adjustments to this PR

So you intentionally go for a lossy conversion and don't even want to put the additional metadata in some comments or notes to be tracked with the data?

codecov[bot] commented 2 years ago

Codecov Report

Merging #596 (2723dd5) into main (c098ba2) will decrease coverage by 0.09%. The diff coverage is 93.75%.

:exclamation: Current head 2723dd5 differs from pull request most recent head dc3f5d2. Consider uploading reports for the commit dc3f5d2 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #596      +/-   ##
==========================================
- Coverage   88.44%   88.34%   -0.10%     
==========================================
  Files          59       59              
  Lines        3228     3218      -10     
==========================================
- Hits         2855     2843      -12     
- Misses        373      375       +2     
Flag Coverage Δ
unittests 88.34% <93.75%> (-0.10%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...rfaces/ecephys/neuralynx/neuralynxdatainterface.py 97.10% <93.75%> (ø)
...version_tools/tools/roiextractors/roiextractors.py 80.00% <0.00%> (-1.54%) :arrow_down:
CodyCBakerPhD commented 2 years ago

@JuliaSprenger Just a heads up - this project has shifted over to https://github.com/catalystneuro/neuroconv now

CodyCBakerPhD commented 1 year ago

Has moved to https://github.com/catalystneuro/neuroconv/pull/170