error reading ecphys file in pynwb

[bendichter]

two issues identified:

1) When I create a units table with only spike_times as a column:

nwb.units = types.core.Units('colnames', {'spike_times',}, ...
    'description', 'units table', ...
    'id', types.core.ElementIdentifiers('data', 0:length(spike_times_index.data) - 1));
nwb.units.spike_times = spike_times_vector;
nwb.units.spike_times_index = spike_times_index;

the colnames variable is length 1 and is automatically converted to a string. This causes an issue for pynwb which expects colnames to be a list, and causes an error on read:

TypeError: incorrect type for 'colnames' (got 'str', expected 'ndarray, list, tuple, Dataset, AbstractDataChunkIterator or FORMDataset')

Would it be possible to ensure that colnames is always a list even if it is length 1?

2) It looks like the dates are not being converted properly into strings. I input the following into the NWBFile constructor:

date = datetime(2018, 3, 1, 12, 0, 0);
session_start_time = datetime(date, 'Format', 'yyyy-MM-dd''T''HH:mm:SSZZ',...
    'TimeZone', 'local');
nwb = nwbfile( 'source', 'acquired on rig2', ...
    'session_description', 'a test NWB File', ...
    'identifier', 'mouse004_day4', ...
    'session_start_time', session_start_time);

which appears to have an ISO valid time:

>> session_start_time

session_start_time = 

  datetime

   2018-03-01T12:00:00+0200

however in the HDF5 there are no hyphens or colons (20180301T120000+0200) and pynwb cannot read the time properly.

[lawrence-mbf]

What's the error message when pynwb reads the time value in?

[bendichter]

$ python -m pynwb.validate ecephys_tutorial.nwb
Validating ecephys_tutorial.nwb against core namespace
 - found the following errors:
Units/colnames (units.colnames): incorrect type - expected 'ascii', got 'utf'
root/file_create_date (file_create_date): incorrect type - expected 'isodatetime', got 'ascii'
root/session_start_time (session_start_time): incorrect type - expected 'isodatetime', got 'utf'
root/timestamps_reference_time (timestamps_reference_time): incorrect type - expected 'isodatetime', got 'utf'

[bendichter]

using the validator, not reading in, but reading would be similar

[lawrence-mbf]

I believe this actually works in through the reader though considering that session_start_time is required for a NWB file.

[lawrence-mbf]

In any case, I can just update the format to match.

[bendichter]

actually you are right, this does go through the reader and gets parsed correctly in pynwb, but the pynwb validator does not like it

[lawrence-mbf]

bd03e830e9cc24611832868f8cee52cf8c01a388 Should fix both time formatting issues and the checks to set scalar attributes. Also, the seconds indicator in the format string yyyy-MM-dd''T''HH:mm:SSZZ is incorrect as capital S values are interpreted to be fractional seconds. Regular seconds are lowercase s.

[bendichter]

ahh ok thanks for the correction

[bendichter]

@ln-vidrio Looks like it's still missing a colon for the timezone offset

[lawrence-mbf]

06ce9a658ad15b82b5196334d1159e5708d2169c This should fix it.

[bendichter]

yay!

Bens-MacBook-Pro:~ bendichter$ python -m pynwb.validate ~/dev/matnwb/ecephys_tutorial.nwb
Validating /Users/bendichter/dev/matnwb/ecephys_tutorial.nwb against core namespace
 - no errors found.
[]