Create a Silver Lab specific Matlab API for NWB2

[jonc125]

See https://github.com/SilverLabUCL/AnalysisPipeline/issues/12 for background

[alessandrofelder]

As of now, we have managed to write an NWB file (schema version 2.1.0) using the python API (pynwb-1.1.2) and then read that file using MatLab (cloned from https://github.com/NeurodataWithoutBorders/matnwb/commit/3353784d1784e7a20327d9141a082994aefb3bae) but for this to work we had to (hackily) change the namespace hdmf-common to hdmf_common in schema->core->nwb.namespace.yaml and then run generateCore.m again.

[ageorgou]

Just a bit more information. matnwb is not completely in sync with pynwb, and they suggest changing the namespace definitions to match those from pynwb if a file has been created using the latter.

We did also try commit https://github.com/NeurodataWithoutBorders/matnwb/commit/2c0a852f894f17ef893134637d36623f1d4a5863 as suggested by NeurodataWithoutBorders/matnwb#170, which should help.

[bendichter]

Thanks for cross-linking the issue. We just pushed some changes to matnwb for reading nwb-schema 1.2.0 files, which should work for current files created by pynwb. Could you test read/write now?

Also, the link to the background in the first post of this thread is dead.

[alessandrofelder]

The test data I used was written using pynwb 1.1.2, h5py 2.9.0 and python 3.6.9

Reading from MatLab works for me, only using the 'ignorecache' option of nwbRead, but without hacky modification from commit https://github.com/NeurodataWithoutBorders/matnwb/commit/2c0a852f894f17ef893134637d36623f1d4a5863 until https://github.com/NeurodataWithoutBorders/matnwb/commit/ea9c25ac5686ce5f388297832c64cf6d05a4b7bc (including both these commits) but fails for later commits with 'Property id must be a types.core.ElementIdentifiers.'

Without the ignorecache option, nwbRead works the first time I run it on my test data, but fails the second time with 'Property vectordata.angle_deg should be one of type(s) {types.hdmf-common.VectorData}.' I think this is due to the generateSpec method affecting +types\+core\PlaneSegmentation.m based on some metadata in my nwb file (?).

After generateSpec is called, PlaneSegmentation.m changes and newly contains

%% VALIDATORS

[...]
function val = validate_pixel_mask(obj, val)
    val = types.util.checkDtype('pixel_mask', 'types.hdmf_common.VectorData', val);
end
function val = validate_pixel_mask_index(obj, val)
    val = types.util.checkDtype('pixel_mask_index', 'types.hdmf_common.VectorIndex', val);
end

[...]

function val = validate_vectordata(obj, val)
    constrained = { 'types.hdmf-common.VectorData' };
    types.util.checkSet('vectordata', struct(), constrained, val);
end
function val = validate_vectorindex(obj, val)
    constrained = { 'types.hdmf-common.VectorIndex' };
    types.util.checkSet('vectorindex', struct(), constrained, val);
end

[...]
``

where not all dashes in occurrences of 'hdmf-common' are corrected to 'hdmf_common'...

Currently trying to make sense of this, so any pointers welcome :)

[bendichter]

@alessandrofelder thanks for this report. I'll let the matnwb team know. If you could provide the files that produce this error or the scripts to produce them, that would be helpful.

[bendichter]

It looks like you are writing in pynwb and reading in matnwb. Is that right?

[lawrence-mbf]

@alessandrofelder When you nwbRead, are there still NwbFile objects present? You may need to clear those objects before reading again.

[alessandrofelder]

@bendichter and @ln-vidrio - thanks a lot for your help! The latest merge into master seems to have fixed all my issues!

Yes, that is right - I am writing from pynwb and reading in matnwb. Don't think any NwbFile objects are still present when I nwbRead - I always tried to read onto the same variable in MatLab. Again, all seems to work for me, now.

For your and future reference, I was using this python class to write a test NWB file.

Again, many many thanks for fixing this so quickly!