[Bug]: New matnwb not reading files

fsantamaria commented 1 month ago

What happened?

I upgraded matlab and downloaded the latest mantnwb (2.7.0). I added the path ($HOME/matnwb). When I type w=nwbRead('myfyle.nwb) I get this error. I have tried using gerneateExtensions for 2.7.0 and 2.6.0 (my files in older version), but maybe I'm doing this incorrectly . I use this to generateExtensions generateExtension('$HOME/matnwb/nwb-schema/2.7.0/core/nwb.namespace.yaml'). Am I doing something wrong?

Steps to Reproduce

git clone https://github.com/NeurodataWithoutBorders/matnwb.git
In matlab
addpath()
w=nwbRead(myfile)

Error Message

Error using ind2sub (line 27)
Size vector must be a real vector with at least 2 elements.

Error in types.untyped.DataStub/load_mat_style (line 49)
        [points{:}] = ind2sub(dataDimensions, orderedSelection);
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in types.untyped.DataStub/load (line 124)
                data = obj.load_mat_style(varargin{1});
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in indexing (line 157)
                B = builtin('subsref', obj, S);
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in types.util.checkDtype>unwrapValue (line 162)
            unwrapped = wrapped.load(1);
                        ^^^^^^^^^^^^^^^
Error in types.util.checkDtype (line 123)
    value = unwrapValue(value);
            ^^^^^^^^^^^^^^^^^^
Error in types.core.NWBFile/validate_file_create_date (line 335)
        val = types.util.checkDtype('file_create_date', 'datetime', val);
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in types.core.NWBFile/set.file_create_date (line 187)
        obj.file_create_date = obj.validate_file_create_date(val);
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in types.core.NWBFile (line 127)
        obj.file_create_date = p.Results.file_create_date;
        ^^^^^^^^^^^^^^^^^^^^
Error in NwbFile (line 16)
            obj = obj@types.core.NWBFile(varargin{:});
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 76)
            parsed = NwbFile(kwargs{:});
                     ^^^^^^^^^^^^^^^^^^
Error in nwbRead (line 79)
    nwb = io.parseGroup(filename, h5info(filename), Blacklist);
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Operating System

Linux

Matlab Version

2024b

Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] Have you ensured this bug was not already reported?

rly commented 1 month ago

Hi @fsantamaria, how did you (or someone else) generate the NWB file that you cannot read? Would it be possible for you to send us one of the files that generates this error? It would help us investigate the issue. The issue seems to be with validating the file_create_date value.

fsantamaria commented 1 month ago

Hi, Thanks for the quick reply. I just generated a file with a minimum of data points. FYI, I'm running simulations in Neuron Python and saving the results in nwb. Because of legacy issues and my expertise I then upload in matlab for analsyis.

Here's the file and just confirmed that generates the same error message.

Thanks for the help.

Best, Fidel

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: Ryan Ly @.> Sent: Tuesday, October 15, 2024 4:40 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Hi @fsantamariahttps://github.com/fsantamaria, how did you (or someone else) generate the NWB file that you cannot read? Would it be possible for you to send us one of the files that generates this error? It would help us investigate the issue. The issue seems to be with validating the file_create_date value.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2415214924, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3G5G6WMDNV3JPIONFDZ3WDTVAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVGIYTIOJSGQ. You are receiving this because you were mentioned.Message ID: @.***>

fsantamaria commented 1 month ago

Hi,

I brought my previous version of matnwb (2.6.0) from my backup. Just copied it directly. It is able to read the files. Of course, it sends a warning when trying to read the recent files I created in python with 2.7.0.

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: Ryan Ly @.> Sent: Tuesday, October 15, 2024 4:40 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Hi @fsantamariahttps://github.com/fsantamaria, how did you (or someone else) generate the NWB file that you cannot read? Would it be possible for you to send us one of the files that generates this error? It would help us investigate the issue. The issue seems to be with validating the file_create_date value.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2415214924, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3G5G6WMDNV3JPIONFDZ3WDTVAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVGIYTIOJSGQ. You are receiving this because you were mentioned.Message ID: @.***>

ehennestad commented 1 month ago

Hi,

I am not able to find the file you attached. Is it possible to either upload the file to a google drive/dropbox and create a shared link ?

fsantamaria commented 1 month ago

Let's try this [https://res.cdn.office.net/assets/mail/file-icon/png/generic_16x16.png]Rpfcf_WT_test25b000.nwb https://utsacloud-my.sharepoint.com/:u:/g/personal/fidel_santamaria_utsa_edu/EZ-nGV8HhEJOjlkrG_t8g-QBwT9f6Lfmg61l5Kh5-nWnqw?e=j7YEym

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: Fidel Santamaria @.> Sent: Tuesday, October 15, 2024 4:57 PM To: NeurodataWithoutBorders/matnwb @.>; NeurodataWithoutBorders/matnwb @.> Cc: Mention @.> Subject: Re: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

Hi, Thanks for the quick reply. I just generated a file with a minimum of data points. FYI, I'm running simulations in Neuron Python and saving the results in nwb. Because of legacy issues and my expertise I then upload in matlab for analsyis.

Here's the file and just confirmed that generates the same error message.

Thanks for the help.

Best, Fidel

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: Ryan Ly @.> Sent: Tuesday, October 15, 2024 4:40 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Hi @fsantamariahttps://github.com/fsantamaria, how did you (or someone else) generate the NWB file that you cannot read? Would it be possible for you to send us one of the files that generates this error? It would help us investigate the issue. The issue seems to be with validating the file_create_date value.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2415214924, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3G5G6WMDNV3JPIONFDZ3WDTVAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVGIYTIOJSGQ. You are receiving this because you were mentioned.

ehennestad commented 1 month ago

Thanks for sharing the file!

I am able to open it in MATLAB R2023b, using both an older version of matnwb and the newest version from GitHub. However, I get the same error as you when I use MATLAB R2024b, and this is due to a breaking change in the ind2sub function introduced with MATLAB R2024.

I unfortunately can not propose a fix right now, but will work on this tomorrow. In the mean time, you could try using an older release of MATLAB if this is available to you.

fsantamaria commented 1 month ago

Thanks for figuring this out. I'll check if I can download an older version.

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: ehennestad @.> Sent: Wednesday, October 16, 2024 2:15 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Thanks for sharing the file!

I am able to open it in MATLAB R2023b, using both an older version of matnwb and the newest version from GitHub. However, I get the same error as you when I use MATLAB R2024b, and this is due to a breaking change in the ind2sub function introduced with MATLAB R2024.

I unfortunately can not propose a fix right now, but will work on this tomorrow. In the mean time, you could try using an older release of MATLAB if this is available to you.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2417741579, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3AXDMXGM4SQ7JEQ7L3Z323OPAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXG42DCNJXHE. You are receiving this because you were mentioned.Message ID: @.***>

fsantamaria commented 1 month ago

Sorry, forgot to mention. I am able to open the files in 2024b using the older version of matnwb (2.6.0). Hope this info helps.

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: ehennestad @.> Sent: Wednesday, October 16, 2024 2:15 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Thanks for sharing the file!

I am able to open it in MATLAB R2023b, using both an older version of matnwb and the newest version from GitHub. However, I get the same error as you when I use MATLAB R2024b, and this is due to a breaking change in the ind2sub function introduced with MATLAB R2024.

I unfortunately can not propose a fix right now, but will work on this tomorrow. In the mean time, you could try using an older release of MATLAB if this is available to you.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2417741579, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3AXDMXGM4SQ7JEQ7L3Z323OPAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXG42DCNJXHE. You are receiving this because you were mentioned.Message ID: @.***>

ehennestad commented 1 month ago

Sorry, forgot to mention. I am able to open the files in 2024b using the older version of matnwb (2.6.0). Hope this info helps.

That is interesting. Did you use the exact same file? I tried the same, but for me this did not work (of course, I might have a different version of matnwb, but based on the error and the cause of the error I am curious to understand how you could open the file without error using an older version of matnwb).

I added a proposed fix on this branch: https://github.com/NeurodataWithoutBorders/matnwb/tree/601-fix-ind2sub-with-scalar-size

You could try it out and let us know if it works. I will update you when this is merged to main.

fsantamaria commented 4 weeks ago

Hi, yes, I used the same file that I sent you. Right now we're plowing ahead with 2023b. The only piece of info is that when I used 2024b with nwb 2.6.0 I got a warning about the 2.7.0 schema of the new file. But the data looked ok.

Best,

--

Fidel Santamaria, PhD

Professor

Department of Neuroscience, Developmental and Regenerative Biology

One UTSA circle

University of Texas at San Antonio,

San Antonio, TX 78249

Office: (210) 458-6910

Lab: (210) 458-6982

Fax: (210) 458-5658

Office/Lab Location: BSB 1.03.30

http://www.utsa.edu/santamarialab/index.htm

From: ehennestad @.> Sent: Wednesday, October 16, 2024 5:17 PM To: NeurodataWithoutBorders/matnwb @.> Cc: Fidel Santamaria @.>; Mention @.> Subject: [EXTERNAL] Re: [NeurodataWithoutBorders/matnwb] [Bug]: New matnwb not reading files (Issue #601)

EXTERNAL EMAIL This email originated outside of The University of Texas at San Antonio. Please exercise caution when clicking on links or opening attachments.

Sorry, forgot to mention. I am able to open the files in 2024b using the older version of matnwb (2.6.0). Hope this info helps.

That is interesting. Did you use the exact same file? I tried the same, but for me this did not work (of course, I might have a different version of matnwb, but based on the error and the cause of the error I am curious to understand how you could open the file without error using an older version of matnwb).

I added a proposed fix on this branch:

https://github.com/NeurodataWithoutBorders/matnwb/tree/601-fix-ind2sub-with-scalar-size

You could try it out and let us know if it works. I will update you when this is merged to main.

— Reply to this email directly, view it on GitHubhttps://github.com/NeurodataWithoutBorders/matnwb/issues/601#issuecomment-2418064299, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADNHS3HFUVD6GYSLZGRSM2TZ33QYPAVCNFSM6AAAAABQABZYL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYGA3DIMRZHE. You are receiving this because you were mentioned.

ehennestad commented 3 weeks ago

Hi, yes, I used the same file that I sent you. Right now we're plowing ahead with 2023b. The only piece of info is that when I used 2024b with nwb 2.6.0 I got a warning about the 2.7.0 schema of the new file. But the data looked ok.

The latest commit of matnwb does support nwb files created with v2.7.0, so you could try that if you find that your version of matnwb does not correctly handle v2.7.0 files

NeurodataWithoutBorders / matnwb