NeurodataWithoutBorders / matnwb

A Matlab interface for reading and writing NWB files
BSD 2-Clause "Simplified" License
49 stars 32 forks source link

how do you add electrodes to an electrode group? #49

Closed bendichter closed 6 years ago

lawrence-mbf commented 6 years ago

Not my wheelhouse but you might need to use ElectricalTable instead? Nothing in the schema points to the possibility of a direct connection between an ElectrodeGroup and a given electrode. How does pynwb do it?

bendichter commented 6 years ago

It's the other way- The ElectrodeTable has a column for ElectrodeGroup. Sorry, the way I phrased it was confusing. pynwb has the convenience function nwbfile.add_electrode, which makes it seem to be the other way around. Could you give me example code for constructing an ElectrodeTable and ElectrodeTableRegion that references it?

lawrence-mbf commented 6 years ago

The schema for ElectrodeGroup is as follows:

- attributes:
  - doc: Value is 'Metadata about a physical grouping of channels'
    dtype: text
    name: help
    value: A physical grouping of channels
  - doc: description of this electrode group
    dtype: text
    name: description
  - doc: description of location of this electrode group
    dtype: text
    name: location
  doc: One of possibly many groups, one for each electrode group.
  links:
  - doc: the device that was used to record from this electrode group
    name: device
    quantity: '?'
    target_type: Device
  neurodata_type_def: ElectrodeGroup
  neurodata_type_inc: NWBContainer

So the ElectrodeGroup contains a link to a Device (optional) which in turn describes the individual Electrodes? Everything else is just description.

bendichter commented 6 years ago

Yeah the link between an electrode and ElectrodeGroup is through a column in the ElectrodeTable.

lawrence-mbf commented 6 years ago

So in general_extracellular_ephys, the schema specifies an electrodes dataset which is an ElectrodeTable. This probably acts as a global registry of all electrodes. This is what types.core.NWBFile.validate_general_extracellular_ephys looks like:

namedprops = struct();
namedprops.electrodes = 'types.core.ElectrodeTable';
constrained = {'types.core.ElectrodeGroup'};
types.util.checkSet('general_extracellular_ephys', namedprops, constrained, val);

This is all optional in the NWBFile though so that's why it's not instantiated with nwbfile.

bendichter commented 6 years ago

Yes, this is a global registry for all electrodes. How would you create this table, add rows to it, and add it to the file? I think it will require soft links to the electrode groups.

lawrence-mbf commented 6 years ago

It's actually an Object Reference, so here's an example of adding one row:

nwb = nwbfile();
eg = types.core.ElectrodeGroup();
nwb.general_extracellular_ephys.set('testEg', eg);
ov = types.untyped.ObjectView('/general/extracellular_ephys/testEg');
variables = {'id', 'x', 'y', 'z', 'imp', 'location', 'filtering', 'description', 'group', 'group_name'};
tbl = table(int64(1), 2, 3, 4, 100, {'location'}, {'filtering'}, {'description'}, ov, {'testEg'}, 'VariableNames', variables);
et = types.core.ElectrodeTable('data', tbl);
nwb.general_extracellular_ephys.set('electrodes', et);

To add rows to a table, do a vertical concatenation using a cell array representing that row.

tbl = [tbl; {int64(5), 6, 7, 8, 200, 'loc2', 'filter2', 'descr2', ov, 'testEg'}];
bendichter commented 6 years ago

Perfect, thanks!

On Fri, Jul 20, 2018 at 1:12 PM ln-vidrio notifications@github.com wrote:

It's actually an Object Reference, so here's an example of adding one row:

nwb = nwbfile(); eg = types.core.ElectrodeGroup(); nwb.general_extracellular_ephys.set('testEg', eg); ov = types.untyped.ObjectView('/general/extracellular_ephys/testEg'); variables = {}; tbl = table(int64(1), 2, 3, 4, 100, {'location'}, {'filtering'}, {'description'}, ov, {'testEg'}, 'VariableNames', variables); et = types.core.ElectrodeTable('data', tbl); nwb.general_extracellular_ephys.set('electrodes', et);

To add rows to a table, do a vertical concatenation using a cell array representing that row.

tbl = [tbl; {int64(5), 6, 7, 8, 200, 'loc2', 'filter2', 'descr2', ov, 'testEg'}];

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NeurodataWithoutBorders/matnwb/issues/49#issuecomment-406665946, or mute the thread https://github.com/notifications/unsubscribe-auth/AAziEu6ui0IiJAakzsmtHaFUwO_CTubTks5uIg9egaJpZM4VXVCG .

--

Ben Dichter, PhD Data Science Consultant

bendichter commented 6 years ago

Could you help me understand the difference between a SoftLink and an ObjectView?

bendichter commented 6 years ago

Also, how would I create an ElectrodeTableRegion object that references specific rows to ElectrodeTable?

lawrence-mbf commented 6 years ago

In terms of matnwb, the only difference is the underlying HDF5 export type. An ExternalLink exports to external link. A SoftLink exports to a soft link. An ObjectView exports to a HDF5 Object Reference. A RegionView exports to a HDF5 Region Reference.

There's a bit of overlap in functionality in matnwb, but the distinction becomes important with lower level HDF5.

lawrence-mbf commented 6 years ago

In regards to ElectrodeTableRegion, you use types.untyped.RegionView and give two arguments:

  1. The absolute path within the file as usual with types.untyped.ObjectView
  2. The index of rows you want out of the referenced Object as a cell array of bounds (so {[1 6] [9 10]} indexes [1:6 9:10] for some 1-dimensional structure (arrays or tables)). This is a bit convoluted but it's a direct mapping of the storage method used by pynwb's Region References.
bendichter commented 6 years ago

edit: I narrowed down the code a little. It looks like the error is with writing the ElectrodeTable.

Could you help me out here? What's going on with this error:

file = nwbfile( ...
    'source', 'a test source', ...
    'session_description', 'a test NWB File', ...
    'identifier', 'TEST123', ...
    'session_start_time', datestr([1970, 1, 1, 12, 0, 0], 'yyyy-mm-dd HH:MM:SS'), ...
    'file_create_date', datestr([2017, 4, 15, 12, 0, 0], 'yyyy-mm-dd HH:MM:SS'));

dev = types.core.Device( ...
    'source', 'source');
file.general_devices.set('dev1', dev);

eg = types.core.ElectrodeGroup('source', 'source', ...
    'description', 'a test ElectrodeGroup', ...
    'location', 'unknown', ...
    'device', types.untyped.SoftLink('/general/devices/dev1'));
file.general_extracellular_ephys.set('electrode_group', eg);
ov = types.untyped.ObjectView('/general/extracellular_ephys/electrode_group');

variables = {'id', 'x', 'y', 'z', 'imp', 'location', 'filtering', 'description', 'group', 'group_name'};
tbl = table(int64(1), NaN, NaN, NaN, NaN, {'location'}, {'filtering'}, {'label'}, ov, {'electrode_group'},...
    'VariableNames', variables);
for i = 2:10
    tbl = [tbl; {int64(i), NaN, NaN, NaN, NaN, 'location', 'filtering', 'label', ov, 'electrode_group'}];
end

et = types.core.ElectrodeTable('data', tbl);
file.general_extracellular_ephys.set('electrodes', et);

filename = 'mat_test.nwb';
nwbExport(file, filename)

error:

Error using hdf5lib2
The class of input data must be char instead of cell when the HDF5 class is H5T_STRING.

Error in H5D.write (line 71)
H5ML.hdf5lib2('H5Dwrite', varargin{:});            

Error in io.writeTable (line 54)
H5D.write(did, tid, sid, sid, 'H5P_DEFAULT', data);

Error in types.core.NWBData/export (line 48)
                io.writeTable(fid, fullpath, obj.data);

Error in types.core.ElectrodeTable/export (line 38)
        refs = export@types.core.NWBData(obj, fid, fullpath, refs);

Error in types.untyped.Set/export (line 83)
                    refs = v.export(fid, propfp, refs); 

Error in types.core.NWBFile/export (line 439)
            refs = obj.general_extracellular_ephys.export(fid, [fullpath '/general/extracellular_ephys'], refs);

Error in nwbfile/export (line 41)
            refs = export@types.core.NWBFile(obj, fid, '/', {});

Error in nwbExport (line 27)
export(nwb, fn);

Error in untitled (line 50)
nwbExport(file, filename)

@ln-vidrio

lawrence-mbf commented 6 years ago

try fff0be4f29eca71804b821551358429fa80a8126. I'm pretty sure I fixed this already but it might've been an accidental revert.

bendichter commented 6 years ago

Fixed. Thanks!