Closed jc192015 closed 2 weeks ago
Hi @jc192015 is the id
column also matching in size? That's considered part of the table even if it's not in colnames
Yes, the length of id column is the same as others.
From: Lawrence @.> Sent: Thursday, July 27, 2023 2:03 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
Hi @jc192015https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jc192015&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=HSyntg0pElgyqDTNBJE0rbam97rqZBo4TSP1fzxzldL7f175JuOLZy5Ic_Z32EuY&s=EPQ4xytzcbl-CPTldx55_fQy9KtI1zzmc62PpeGmcGw&e= is the id column also matching in size? That's considered part of the table even if it's not in colnames
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1654256841&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=HSyntg0pElgyqDTNBJE0rbam97rqZBo4TSP1fzxzldL7f175JuOLZy5Ic_Z32EuY&s=VDML2QS3n2ZEx8tpvaN7ETmtA9IVX4W6G9nOR_vUFS0&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAURSDTJFLI72P66JKS3XSK3PLANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=HSyntg0pElgyqDTNBJE0rbam97rqZBo4TSP1fzxzldL7f175JuOLZy5Ic_Z32EuY&s=0T-7Q5kadhMtcwfIctHKmQfo2DgT79YkKfPfsR9eve8&e=. You are receiving this because you were mentioned.Message ID: @.***>
@jc192015 Is the data in each VectorIndex column the same length as id
? When you have VectorIndex columns their lengths are factored into the size of the table, instead of the base column.
timeseries_index
for instance should have the same number of elements as id
. However, timeseries
does not have to and can be much larger.
@jc192015 Can you also post the full stack of the error?
%%%%%%%%%%% Validation results %%%%%%%%%
(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320.nwb warnings.warn(msg) C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " Traceback (most recent call last): File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1256, in construct obj = self.new_container(cls, builder.source, parent, builder.attributes.get(self.spec.id_key()), File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1269, in new_container obj.init(kwargs) File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call return func(args[0], pargs) File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb\epoch.py", line 32, in init super().init(kwargs) File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call return func(args[0], pargs) File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\common\table.py", line 361, in init__ raise ValueError("columns must be the same length") ValueError: columns must be the same length
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\nwbinspector.exe__main.py", line 7, in
%%%%%%%%% Validation ended, look at the last sentence, 'Could not construct TimeIntervals object due to: columns must be the same length' %%%%%%%%%
Would it be possible to share either this NWB file or alternatively a minimal script that produces a NWB file which also fails the nwbinspector?
Furthermore, what version of the schema are you currently using? VectorData
should be in hdmf_common
if you're writing a new file with a new schema.
How can I share the NWB file with you?
Is there a more secure way we could communicate? I can share a onedrive link that you can upload your file to but I will need at least an email address to share it with. Either the one you're currently using or a burner one you can use publicly.
jc19@bcm.edu
Sent an invite, please check your inbox and spam folder. If you have not received it in a while please let me know and we'll have to figure out another way.
uploaded. Thank you in advance.
Try transposing the data provided for the column CueTime
. nwbinspector
(and by extension pynwb and DANDI) prefer C-style row-major ordering of their data whereas MATLAB prefers Fortran-style column-major ordering so when you're using multi-dimensional data like CueTime
make sure that the last dimension matches as that is what is used for height checks in the table.
Note that for QOL reasons, vectors for the most part do not care what dimension they are on so you do not need to change those values in the table.
After transposing "CueTime" , I got the following outcome.
%%%%%%%%%% Validation %%%%%%%%%
(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320.nwb warnings.warn(msg) C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " Inspecting NWBFiles...: 0%| | 0/1 [00:00<?, ?it/s]C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
NWBInspector Report Summary
Timestamp: 2023-07-27 15:20:24.448653-05:00 Platform: Windows-10-10.0.19045-SP0 NWBInspector version: 0.4.29
Found 53 issues over 1 files: 10 - PYNWB_VALIDATION 19 - CRITICAL 10 - BEST_PRACTICE_VIOLATION 14 - BEST_PRACTICE_SUGGESTION
0.0 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: root/general/experimenter - 'None' object at location 'general/experimenter' Message: incorrect shape - expected an array of shape '[None]', got non-array data 'Jia Zhu'
0.1 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: root/general/related_publications - 'None' object at location 'general/related_publications' Message: incorrect shape - expected an array of shape '[None]', got non-array data 'Jia Zhu, , Hana HasanbegoviÄ, , Liu D. Liu, Zhenyu Gao, #, Nuo Li, #Activity map of a cortico-cerebellar loop underlying motor planning2023, Nature Neuroscience'
0.2 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex/description - 'None' object at location 'intervals/trials/timeseries_index' Message: argument missing
0.3 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex - 'None' object at location 'intervals/trials/timeseries_index' Message: incorrect type - expected 'uint8', got 'float64'
0.4 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex/description - 'None' object at location 'units/spike_times_index' Message: argument missing
0.5 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex - 'None' object at location 'units/spike_times_index' Message: incorrect type - expected 'uint8', got 'float64'
0.6 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms/unit - 'None' object at location 'units/waveforms' Message: argument missing
0.7 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms - 'None' object at location 'units/waveforms' Message: incorrect type - expected 'numeric', got 'object'
0.8 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms - 'None' object at location 'units/waveforms' Message: incorrect shape - expected '[None, None]', got '(9,)'
0.9 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorData/description - 'None' object at location 'units/trials_index' Message: argument missing
1.10 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit8SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.11 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit8CS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.12 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit7SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.13 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit6SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.14 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit5SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.15 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit4SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.16 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit3SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.17 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit2SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.18 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit1SS' Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
1.19 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_subject_age - 'Subject' object at location '/general/subject' Message: Subject age, '20180503', does not follow ISO 8601 duration format, e.g. 'P2Y' for 2 years or 'P23W' for 23 weeks. You may also specify a range using a '/' separator, e.g., 'P1D/P3D' for an age range somewhere from 1 to 3 days. If you cannot specify the upper bound of the range, you may leave the right side blank, e.g., 'P90Y/' means 90 years old or older.
1.20 D:\Data_share\NWB\BAYLORNL23_20190320.nwb and 8 other files: check_dynamic_table_region_data_validity - 'DynamicTableRegion' object with name 'electrodes' Message: Some elements of electrodes are out of range because they are greater than the length of the target table. Note that data should contain indices, not ids.
2.21 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'OptogeneticSeries' object at location '/stimulus/presentation/laser_power' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.22 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit8SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.23 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit8CS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.24 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit7SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.25 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit6SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.26 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit5SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.27 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit4SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.28 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit3SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.29 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit2SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
2.30 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit1SS' Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.
3.31 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_description - 'Device' object at location '/general/devices/laser-473nm (Utralaser)' Message: Description is missing.
3.32 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_description - 'Device' object at location '/general/devices/Cambridge NeuroTech 64ch H2 probe 2x32 shank1:ch1-32 shank2:ch33-64 (Cambridge NeuroTech)' Message: Description is missing.
3.33 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'OptogeneticSeries' object at location '/stimulus/presentation/laser_power' Message: data is not compressed. Consider enabling compression when writing a dataset.
3.34 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'SpikeEventSeries' object at location '/analysis/unit8SS' Message: data is not compressed. Consider enabling compression when writing a dataset.
3.35 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'SpikeEventSeries' object at location '/analysis/unit6SS' Message: data is not compressed. Consider enabling compression when writing a dataset.
3.36 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_experimenter_form - 'NWBFile' object at location '/' Message: The name of experimenter 'Jia Zhu' does not match any of the accepted DANDI forms: 'LastName, Firstname', 'LastName, FirstName MiddleInitial.' or 'LastName, FirstName, MiddleName'.
3.37 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_doi_publications - 'NWBFile' object at location '/' Message: Metadata /general/related_publications 'Jia Zhu, , Hana HasanbegoviÄ, , Liu D. Liu, Zhenyu Gao, #, Nuo Li, #Activity map of a cortico-cerebellar loop underlying motor planning2023, Nature Neuroscience' does not start with 'doi: ###' and is not an external 'doi' link.
3.38 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'HitR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.39 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'HitL' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.40 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'ErrR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.41 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'ErrL' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.42 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'NoLickR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.43 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'LickEarly' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 1.49KB.
3.44 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binarycapability - 'TimeIntervals' object with name 'trials' Message: Column 'GoodTrials' uses 'integers' but has binary values [0 1]. Consider making it boolean instead and renaming the column to start with 'is'; doing so will save 639.00B.
%%%%%%%%%%%% CRITICAL label appears %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%% Fake upload %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
(base) C:\Windows\System32\000572>dandi organize D:\Data_share\NWB\ -f dry 2023-07-27 15:23:17,785 [ INFO] Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. 2023-07-27 15:23:24,395 [ INFO] NumExpr defaulting to 8 threads. 2023-07-27 15:23:26,576 [ INFO] Loading metadata from 1 files [Parallel(n_jobs=-1)]: Using backend LokyBackend with 40 concurrent workers. C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed. warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, " [Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 12.4s [Parallel(n_jobs=-1)]: Done 1 out of 1 | elapsed: 12.4s finished DRY: D:\Data_share\NWB\BAYLORNL23_20190320.nwb -> sub-BAYLORNL23\sub-BAYLORNL23_ecephys+ogen.nwb 2023-07-27 15:23:39,131 [ INFO] Organized 0 out of 1 paths. Visit C:\Windows\System32\000572/ 2023-07-27 15:23:39,131 [ INFO] Logs saved in C:\Users\NuoLiLab\AppData\Local\dandi\dandi-cli\Logs\20230727202316Z-21988.log
After transposing the data related to "unitXXX, " I got the following outcome.
%%%%%%%%%%%% Validation %%%%%%%%%%%%%%%%%%%%%
(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320T.nwb warnings.warn(msg) Traceback (most recent call last): File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1256, in construct obj = self.new_container(cls, builder.source, parent, builder.attributes.get(self.spec.id_key()), File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1269, in new_container obj.init__(kwargs) File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call return func(args[0], pargs) File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py", line 125, in init raise ValueError('Must provide the same number of timestamps and spike events') ValueError: Must provide the same number of timestamps and spike events
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\nwbinspector.exe__main.py", line 7, in
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Fake update %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
(base) C:\Windows\System32\000572>dandi organize D:\Data_share\NWB\ -f dry 2023-07-27 15:34:02,754 [ INFO] Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. 2023-07-27 15:34:02,754 [ INFO] NumExpr defaulting to 8 threads. 2023-07-27 15:34:04,826 [ INFO] Loading metadata from 1 files [Parallel(n_jobs=-1)]: Using backend LokyBackend with 40 concurrent workers. [Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 4.2s [Parallel(n_jobs=-1)]: Done 1 out of 1 | elapsed: 4.2s finished 2023-07-27 15:34:09,182 [ WARNING] Completely empty record for D:\Data_share\NWB\BAYLORNL23_20190320T.nwb 2023-07-27 15:34:09,183 [ INFO] Logs saved in C:\Users\NuoLiLab\AppData\Local\dandi\dandi-cli\Logs\20230727203401Z-14636.log Error: 1 out of 1 files were found not containing all necessary metadata: D:\Data_share\NWB\BAYLORNL23_20190320T.nwb
@jc192015 Is this dataset already on DANDI or are you trying to upload it do DANDI?
As far as I can tell, the Spike Event Series shapes look right but I'll need to investigate this further. The error you're getting is also a bug in pynwb.
Yes, I am struggling to upload nwbs to DANDI.
I see, I will need to warn you that DANDIsets need to be at least schema version 2.1 or higher. The file you've created is still version 2.0b, you will most likely need to adapt the script to at least schema version 2.1. This may also impact why the nwbinspector is failing your file.
Note we are still looking at it, the error still indicates an shape error which I have not been able to validate.
I don't know how to modify the following code to meet MatNWB 2.6.0 requirment. That is why I am still using 2.0b.
trials_epoch = types.core.TimeIntervals(... 'start_time', intervals_trials_start_time,... 'colnames', [data.trialTypeStr; data.trialPropertiesHash.keyNames .';... {'start_time'; 'stop_time'}],... 'description', 'trial data and properties', ... 'stop_time', intervals_trials_stop_time,... 'id', intervals_trials_id_data,... 'timeseries', intervals_trials_timeseries,... 'timeseries_index', intervals_trials_timeseries_index);
trials_epoch = types.core.TimeIntervals(...
I believe the main change for TimeIntervals
is that the timeseries
columns has changed to the TimeSeriesReferenceVectorData
type instead of using the base VectorData
. Other than changing the name of the type, the definition for that column is still the same.
@jc192015 my suggestion would be to at least upgrade the schema to 2.1.0 using the MAtNWB version that you have. I then recommend working with the file using the newest release of MatNWB since that supports all schema versions after 2.1.0 (the MatNWB version only specifies the most recent supported schema).
I generated a new nwb file using MatNWB 2.6.0.2. But I got an error message indicating that "Could not construct Units object due to: columns must be the same length." But I have checked the "Units" . The length of "id", "soike_times_index", and "waveforms" are the same. I uploaded related files "BAYLORNL23_20190320-2.6.0.2.nwb" and "Message for file made on 20230728_NL23_20190320-2.6.0.2.txt" to the onedrive that you invited me yesterday for your reference. Thank you.
I generated a new nwb file using MatNWB 2.6.0.2. But I got an error message indicating that "Could not construct Units object due to: columns must be the same length." But I have checked the "Units" . The length of "id", "soike_times_index", and "waveforms" are the same. I uploaded related files "BAYLORNL23_20190320-2.6.0.2.nwb" and "Message for file made on 20230728_NL23_20190320-2.6.0.2.txt" to the onedrive that you invited me yesterday for your reference. Thank you.
From: Lawrence @.> Sent: Friday, July 28, 2023 8:46 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
@jc192015https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jc192015&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=WHIWfYJMF-7hJZ-OnHdY2XVs4ma8v-2Cb4Pw6HIR-kSBprrPkHnsapM944FEydb2&s=AKbimVc1M1HqHygNNMa6kKkzy0rS6hsK6bIQMUn6LFo&e= my suggestion would be to at least upgrade the schema to 2.1.0 using the MAtNWB version that you have. I then recommend working with the file using the newest release of MatNWB since that supports all schema versions after 2.1.0 (the MatNWB version only specifies the most recent supported schema).
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655711002&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=WHIWfYJMF-7hJZ-OnHdY2XVs4ma8v-2Cb4Pw6HIR-kSBprrPkHnsapM944FEydb2&s=07v6cS_kbquS7N-M2VQfY75zdkU-dr5UC8HZsBF7lXE&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUUS4LDWJE5G7XDGLH3XSO7CZANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=WHIWfYJMF-7hJZ-OnHdY2XVs4ma8v-2Cb4Pw6HIR-kSBprrPkHnsapM944FEydb2&s=4H-Sz9fk-rzAU8EPG-cCnQv10zuyan8iuDJtELKY2b8&e=. You are receiving this because you were mentioned.Message ID: @.***>
There is a column called trials
in the Units table. If it is supposed to be there you will need a VectorIndex named trials_index
with 9 elements. We do have trials
-specific tables so my assumption is that it's not supposed to be there and should perhaps be in the intervals_trials
table or elsewhere.
When adding trails_index as VectorIndex to type.core.Units (2.6.0), I got the following error message:
Unrecognized method, property, or field 'vectorindex' for class 'types.core.Units'. Error in convertTrials4jia (line 643) nwb.units.vectorindex.set('trials_index', trials_idx);
That is why I avoided to add trials_index to Units. Do I need to remove trials" from Units?
From: Lawrence @.> Sent: Friday, July 28, 2023 10:09 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
There is a column called trials in the Units table. If it is supposed to be there you will need a VectorIndex named trials_index with 9 elements. We do have trials-specific tables so my assumption is that it's not supposed to be there and should perhaps be in the intervals_trials table or elsewhere.
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655860266&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=UFbxOWzFr_TUUYlk_ZyEWaSM7Z8jvguBVR4ungTClenKXU40x7Te1BTXICycM8OF&s=POG6f4h0GsyrFzUT5tkirWtlcskF412Vo-aBk-eCSOY&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUQHLP4B5CFLZOEKGLLXSPI3VANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=UFbxOWzFr_TUUYlk_ZyEWaSM7Z8jvguBVR4ungTClenKXU40x7Te1BTXICycM8OF&s=3kxKLN6COipSgcXAXlixm3bbfDVzj18BOeaTEcFBlF8&e=. You are receiving this because you were mentioned.Message ID: @.***>
In 2.6.0
all VectorIndex objects are valid VectorData
objects so you should add trials_index
to the vectordata
property in the Units table instead. In fact, this should be done for all VectorIndex objects created in the script.
Could you please tell me how to add trials_index to the vectordata property in the Units table?
The following is the original code for initializing units, trials, and trials_index and adding trials and trials_index to units.
nwb.units = types.core.Units('colnames',... {'spike_times', 'trials', 'trials_index', 'waveforms'},... 'description', 'Analysed Spike Events');
unitTrials = types.hdmf_common.VectorData(... 'description', 'A large group of trial IDs for each unit',... 'data', []);
trials_idx = types.hdmf_common.VectorIndex(... 'data', [],... 'target', types.untyped.ObjectView('/units/trials'));
nwb.units.vectorindex.set('trials_index', trials_idx); nwb.units.vectordata.set('trials', unitTrials);
From: Lawrence @.> Sent: Friday, July 28, 2023 10:25 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
In 2.6.0 all VectorIndex objects are valid VectorData objects so you should add trials_index to the vectordata property in the Units table instead. In fact, this should be done for all VectorIndex objects created in the script.
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655883072&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=tpB90m6ChJVvTFu_NHzbFuFYni09-q3mpYBmXYbaerM4RANvuBXdjXsorzyzrzIW&s=VUzNMjHcYy3O64IZRSU1qTT0gFpZQ3JD9HdRkZpHj08&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUWIDXIPLKUNNDWZRFTXSPKVZANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=tpB90m6ChJVvTFu_NHzbFuFYni09-q3mpYBmXYbaerM4RANvuBXdjXsorzyzrzIW&s=kgpxEDOnRSiVWUY0FygSsqNAnjY5xDfk1Lb1O4gZtyg&e=. You are receiving this because you were mentioned.Message ID: @.***>
Try this:
nwb.units = types.core.Units('colnames',...
{'spike_times', 'trials', 'waveforms'},...
'description', 'Analysed Spike Events');
unitTrials = types.hdmf_common.VectorData(...
'description', 'A large group of trial IDs for each unit',...
'data', []);
trials_idx = types.hdmf_common.VectorIndex(...
'data', [],...
'target', types.untyped.ObjectView('/units/trials'));
nwb.units.vectordata.set('trials_index', trials_idx);
nwb.units.vectordata.set('trials', unitTrials);
Also note that you should remove any vector index columns from colnames
. They, along with the vectordata they are referencing, consitute a single column in the Units table (and all Dynamic Table objects).
I modified the code, used nwbRead to validate the output nwb, and got message "Unrecognized field name "internal". Is there still something wrong with units?
nwbRead('C:\Users\hi2ch\Desktop\nwb\out\BAYLORNL23_20190320.nwb')
ans =
NwbFile with properties:
nwb_version: '2.6.0'
file_create_date: [1×1 types.untyped.DataStub]
identifier: 'BAYLORNL23_20190320'
session_description: 'Animal `BAYLORNL23` on Session `20190320`'
session_start_time: 2019-03-20T09:59:31.000000-05:00
timestamps_reference_time: 2019-03-20T09:59:31.000000-05:00
acquisition: [0×1 types.untyped.Set]
analysis: [9×1 types.untyped.Set]
general: [0×1 types.untyped.Set]
general_data_collection: 'extracellularDataType: sorted spike times, spike waveform snipets, spike band voltage traces↵cellType: Identified PC, others↵identificationMethod: SS and CS spike waveforms and cross-correlogram, spike waveforms↵amplifierRolloff: ↵spikeSorting: manual:matclust↵ADunit: 14 bits'
general_devices: [2×1 types.untyped.Set]
general_experiment_description: 'experimentType: behavior, extracellular, photostim↵referenceAtlas: Allen Reference Atlas↵task_keyword: detection, discrimination, lick Left lick Right, delay, somatosensory, motor, response, reward'
general_experimenter: [1×1 types.untyped.DataStub]
general_extracellular_ephys: [1×1 types.untyped.Set]
general_extracellular_ephys_electrodes: [1×1 types.hdmf_common.DynamicTable]
general_institution: 'Baylor College of Medicine, Houston, TX 77030'
general_intracellular_ephys: [0×1 types.untyped.Set]
general_intracellular_ephys_experimental_conditions: []
general_intracellular_ephys_filtering: []
general_intracellular_ephys_intracellular_recordings: []
general_intracellular_ephys_repetitions: []
general_intracellular_ephys_sequential_recordings: []
general_intracellular_ephys_simultaneous_recordings: []
general_intracellular_ephys_sweep_table: []
general_keywords: [1×1 types.untyped.DataStub]
general_lab: 'Nuo Li'
general_notes: []
general_optogenetics: [1×1 types.untyped.Set]
general_optophysiology: [0×1 types.untyped.Set]
general_pharmacology: []
general_protocol: 'IACUC'
general_related_publications: [1×1 types.untyped.DataStub]
general_session_id: '20190320'
general_slices: []
general_source_script: 'convertTrials4jia'
general_source_script_file_name: 'convertTrials4jia.m'
general_stimulus: 'photostim'
general_subject: [1×1 types.core.Subject]
general_surgery: 'Mice were prepared for photoinhibition and electrophysiology with a clear-skull cap and a headpost. The scalp and periosteum over the dorsal surface of the skull were removed. A layer of cyanoacrylate adhesive (Krazy glue, Elmer's Products Inc.) was directly applied to the intact skull. A custom made headpost was placed on the skull with its posterior edge aligned with the suture lambda and cemented in place with clear dental acrylic (Lang Dental Jet Repair Acrylic; 1223-clear). A thin layer of clear dental acrylic was applied over the cyanoacrylate adhesive covering the entire exposed skull, followed by a thin layer of clear nail polish (Electron Microscopy Sciences, 72180).'
general_virus: []
intervals: [0×1 types.untyped.Set]
intervals_epochs: []
intervals_invalid_times: []
intervals_trials: [1×1 types.core.TimeIntervals]
processing: [0×1 types.untyped.Set]
scratch: [0×1 types.untyped.Set]
stimulus_presentation: [2×1 types.untyped.Set]
stimulus_templates: [0×1 types.untyped.Set]
units: [1×1 types.core.Units]
Unrecognized field name "internal".
From: Lawrence @.> Sent: Friday, July 28, 2023 10:50 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
Try this:
nwb.units = types.core.Units('colnames',... {'spike_times', 'trials', 'waveforms'},... 'description', 'Analysed Spike Events');
unitTrials = types.hdmf_common.VectorData(... 'description', 'A large group of trial IDs for each unit',... 'data', []);
trials_idx = types.hdmf_common.VectorIndex(... 'data', [],... 'target', types.untyped.ObjectView('/units/trials'));
nwb.units.vectordata.set('trials_index', trials_idx); nwb.units.vectordata.set('trials', unitTrials);
Also note that you should remove any vector index columns from colnames. They, along with the vectordata they are referencing, consitute a single column in the Units table (and all Dynamic Table objects).
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655915614&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=oErMIGrfeN1mHBtIXEo5_kigGs2aRJzFYZD-8z1LRtxP5hsovXxSBccXTCb7lzgV&s=aKLK1_AnEvPSPC3OmAIClLDv7pYmjuRNMHUH5oNFTiU&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUSGVTUJW4H6DUV7AI3XSPNWHANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=oErMIGrfeN1mHBtIXEo5_kigGs2aRJzFYZD-8z1LRtxP5hsovXxSBccXTCb7lzgV&s=3VKHkKNQskAHojc02FbTw36v8GAaUMqIeU9hZ7Rt8ME&e=. You are receiving this because you were mentioned.Message ID: @.***>
It looks like the file is displayed properly so I don't think so. Could you share that exported file in onedrive I can confirm if that's true or not?
BAYLORNL23_20190320-2.6.0.2.new.nwb has been uploaded. Thank you so much.
From: Lawrence @.> Sent: Friday, July 28, 2023 11:06 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
It looks like the file is displayed properly so I don't think so. Could you share that exported file in onedrive I can confirm if that's true or not?
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655942754&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=fGCNxrq2AzL8XiVF9vxMjKkC3ohfLLZfgvtm1pTOyvxFM7VGr2szA06KPwGAVjqz&s=Ce5SJjcpak7-MGMKi6iDFMRKuJZJQ1aL8MSOxlBIsDM&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUS422EN37RSOI7YBBDXSPPP5ANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=fGCNxrq2AzL8XiVF9vxMjKkC3ohfLLZfgvtm1pTOyvxFM7VGr2szA06KPwGAVjqz&s=5F1iu4rP_G91IIv4Jd48NIa1qsRWHpLxwK_fOoArfKQ&e=. You are receiving this because you were mentioned.Message ID: @.***>
I don't have any issues reading this file. That error that you're seeing might be from either the script or some other thing you've set up to run after reading the file.
Running the validation, the "Critical" label is still there. I uploaded "Messages for_BAYLORNL23_20190320-2.6.0.2.new.txt" for your reference.
From: Lawrence @.> Sent: Friday, July 28, 2023 11:24 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
I don't have any issues reading this file. That error that you're seeing might be from either the script or some other thing you've set up to run after reading the file.
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655965217&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=L02ECxhVxT3086ZSSxCxCv1vWxMXdVxEbce1XJQTQ2UXmGhI8cppUF5gikf7OQG2&s=IbKahsEoiDHuq3FURBu92b04dffWKPAOBrSJPRuWLKM&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUULFV6TAJKMHDA45B3XSPRUBANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=L02ECxhVxT3086ZSSxCxCv1vWxMXdVxEbce1XJQTQ2UXmGhI8cppUF5gikf7OQG2&s=38KifV4BGbv3aCYqize-40bvJzuQFiXa4JK0IE7kyEI&e=. You are receiving this because you were mentioned.Message ID: @.***>
Oh that's interesting. I guess the second dimension is now the one that is checked for valid spike event series. Can you transpose the spike event series data? If you did before generating this file then undo that transpose.
If undoing the transpose, I got the following:
hdmf.build.errors.ConstructError: (root/analysis/unit1SS GroupBuilder {'attributes': {'comments': 'cellType: PCSimpleSpikes', 'description': 'single unit1SS with cell type information and approximate depth', 'help': 'Snapshots of spike events from data.', 'namespace': 'core', 'neurodata_type': 'SpikeEventSeries'}, 'groups': {}, 'datasets': {'control': root/analysis/unit1SS/control DatasetBuilder {'attributes': {}, 'data':
From: Lawrence @.> Sent: Friday, July 28, 2023 11:47 AM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
Oh that's interesting. I guess the second dimension is now the one that is checked for valid spike event series. Can you transpose the spike event series data? If you did before generating this file then undo that transpose.
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1655994775&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=d2mCWBVvCHyW7vMsaEb8GV3XzkEV7bXa-LXaPyyq-6U9RGXjt91BfrN0oIv00jPa&s=CBwqn5ihDbTqfLc5jYHeKNQjxd-hDGt2_hlOriOu_po&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUWATQJWMXU4WX4ZHNTXSPUIRANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=d2mCWBVvCHyW7vMsaEb8GV3XzkEV7bXa-LXaPyyq-6U9RGXjt91BfrN0oIv00jPa&s=hcxN84IwAG7F45nB-LlvJ4q6CWd2q_lDYkQMy_sqN2k&e=. You are receiving this because you were mentioned.Message ID: @.***>
Very strange. Could you share this version with the onedrive so we can take a look at it?
BAYLORNL23_20190320-2.6.0.2.newWT.nwb has been uploaded. Thank you.
From: Lawrence @.> Sent: Friday, July 28, 2023 12:22 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
Very strange. Could you share this version with the onedrive so we can take a look at it?
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1656045089&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=J1eAL0gI52LwENHZIMpHOCENF1AdTypvHqifrmOK6cfYh2wIXAdMdGtr6AWukWPh&s=O1CqBIgzlBxbq6ZUuN59ak33nripzvrf0f33Tk8zIiQ&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUUWWORJBAD4HQT2HBLXSPYMZANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=J1eAL0gI52LwENHZIMpHOCENF1AdTypvHqifrmOK6cfYh2wIXAdMdGtr6AWukWPh&s=Y4MNHaIbXzgNu0KhcjDI54OX5sBwtC6MaNLxkI7_jYk&e=. You are receiving this because you were mentioned.Message ID: @.***>
@jc192015 I have opened a new issue here for discussion regarding this error: https://github.com/NeurodataWithoutBorders/pynwb/issues/1746
Hopefully this is easily resolvable.
Thank you so much for your time and help.
From: Lawrence @.> Sent: Friday, July 28, 2023 12:37 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Chen, Jian @.>; Mention @.> Subject: Re: [NeurodataWithoutBorders/matnwb] [Bug]: (Issue #527)
CAUTION: This email is not from a BCM Source. Only click links or open attachments you know are safe.
@jc192015https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jc192015&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=EgX5J1rLXeNT6GX6zuoY4e3LVi3yHaEj8XsJs7vUOYjehJ36CeNvXvhdrEr8F_tP&s=B-eS76K58lrcRT4KT4fi7yvQMsM0iEg2_nE7by-He3o&e= I have opened a new issue here for discussion regarding this error: NeurodataWithoutBorders/pynwb#1746https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_pynwb_issues_1746&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=EgX5J1rLXeNT6GX6zuoY4e3LVi3yHaEj8XsJs7vUOYjehJ36CeNvXvhdrEr8F_tP&s=gbvpAPpR5z7tF39E7yNdwoxf1RjqI_KQGWO0ZB01yAA&e=
Hopefully this is easily resolvable.
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NeurodataWithoutBorders_matnwb_issues_527-23issuecomment-2D1656089295&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=EgX5J1rLXeNT6GX6zuoY4e3LVi3yHaEj8XsJs7vUOYjehJ36CeNvXvhdrEr8F_tP&s=A6bECQW2aPkr2vOsuypn7w7tQ-gJU-MdmJIlMzVuuhI&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQUAUW2JOWMT547NVJ5S6TXSP2HBANCNFSM6AAAAAA22P4MAA&d=DwMCaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=9Gl8TqypehHm7acAeymJOA&m=EgX5J1rLXeNT6GX6zuoY4e3LVi3yHaEj8XsJs7vUOYjehJ36CeNvXvhdrEr8F_tP&s=jLrBK-RfPQklMubfMQVOXOGURIkyuxAhk31An6gWKF8&e=. You are receiving this because you were mentioned.Message ID: @.***>
What happened?
When running NWB validation for a created nwb file, I got the following error message indicating that "Could not construct TimeIntervals object due to: columns must be the same length. " I checked the length of columns (all elements in 'colnames' ) are the same. Could you please let me know how to fix that? Thank you in advance for your time.
Steps to Reproduce
Error Message
Operating System
Windows
Matlab Version
R2023a
Code of Conduct