search

flatironinstitute / neurosift

Browser-based NWB visualization and DANDI exploration
Apache License 2.0
44 stars 6 forks source link

How can I view a remote NWB file hosted somewhere else than Dandi? #129

Closed ehennestad closed 7 months ago

ehennestad commented 7 months ago

Visualize remote NWB files hosted on non-DANDI web servers or cloud storage.

I did not immediately understand how to do this

ehennestad commented 7 months ago

Also, are there any requirements to where the file is stored, i.e does it need to be in an s3 bucket and available through the s3 protocol, or can it be on any cloud storage with api access?

magland commented 7 months ago

Hi @ehennestad

Right now the requirements are:

Do you have a URL for the NWB file?

Then you just do:

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=https://api.dandiarchive.org/api/assets/37f5e3ce-e1b7-48a7-a47d-6745875eacaa/download/

replacing the url query parameter.

ehennestad commented 7 months ago

Awesome! Thanks

cdesyoun commented 3 months ago

Hi, I am working on the NEMAR project, which manages EEG/MEG/iEEG datasets (https://nemsr.org/).

For testing NWB files, I created sample files on NEMAR storage. These files can be accessed via the download URL, for example, "https://nemar-dev.ucsd.edu/dataexplorer/download?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/download".

When I test this link, https://neurosift.app/?p=/nwb&url="https://nemar-dev.ucsd.edu/dataexplorer/download?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/download", it results in a forever loading page.

What do I need to know to use the Neurosift interface correctly?

ehennestad commented 3 months ago

When I test this link, https://neurosift.app/?p=/nwb&url="https://nemar-dev.ucsd.edu/dataexplorer/download?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/download", it results in a forever loading page.

Is that link for a single file? The URL should point to one file

magland commented 3 months ago

Hi @cdesyoun

I fixed the query parsing for neurosift so that solves one issue.

BTW, your trailing quotation mark was not included in the hyperlink, but now it should work either way.

The other issue is that the server (https://nemar-dev.ucsd.edu) does not have CORS enabled for the https://neurosift.app domain. Is it possible to have the admins modify the CORS configuration? That should solve the problem. If so, they should enable HEAD and GET requests for https://neurosift.app and also (for development) http://localhost:4200. It also needs to support Range requests because neurosift requests specific byte ranges.

If it is not possible to modify the CORS configuration in this way, you'll need to set up a proxy server, which involves some effort and which will incur addition expense in terms of bandwidth.

cdesyoun commented 3 months ago

When I test this link, https://neurosift.app/?p=/nwb&url="https://nemar-dev.ucsd.edu/dataexplorer/download?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/download", it results in a forever loading page.

Is that link for a single file? The URL should point to one file

Yes, it is a single file.

cdesyoun commented 3 months ago

Hi @cdesyoun

I fixed the query parsing for neurosift so that solves one issue.

BTW, your trailing quotation mark was not included in the hyperlink, but now it should work either way.

The other issue is that the server (https://nemar-dev.ucsd.edu) does not have CORS enabled for the https://neurosift.app domain. Is it possible to have the admins modify the CORS configuration? That should solve the problem. If so, they should enable HEAD and GET requests for https://neurosift.app and also (for development) http://localhost:4200. It also needs to support Range requests because neurosift requests specific byte ranges.

If it is not possible to modify the CORS configuration in this way, you'll need to set up a proxy server, which involves some effort and which will incur addition expense in terms of bandwidth.

Thank you for the quick update, @magland. I will give it a try.

magland commented 3 months ago

Sounds good @cdesyoun . Let me know how that goes. I think it would be great to be able to point neurosift to your database.

magland commented 2 months ago

@cdesyoun did you get this working? I got an email notification but I don't see the comment here. The link in the notification was working for me....

cdesyoun commented 2 months ago

@magland Yes, it is working. I resolved the CORS issue on the nemar-dev site.

cdesyoun commented 2 months ago

@magland , When accessing the following URL:

https://neurosift.app/?p=/nwb&url="https://nemar-dev.ucsd.edu/dataexplorer/downloadfile?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/sub-01_ses-20140828T132700_ecephys+image.nwb",

I encountered the error messages below:

Error loading file: Problem loading file. Unable to load root group. This could be due to a CORS configuration issue.

I guess that the file name, "sub-01_ses-20140828T132700_ecephys+image.nwb", is not allowed due to the presence of a plus sign (+).

magland commented 2 months ago

@cdesyoun Try pasting this into your browser address bar

https://nemar-dev.ucsd.edu/dataexplorer/downloadfile?filepath=/expanse/projects/nemar/openneuro/nwb_example_files/sub-01_ses-20140828T132700_ecephys+image.nwb

File not found?

cdesyoun commented 2 months ago

Thanks for finding the bug, @magland. I noticed that this type of file name is not allowed on the NEMAR site. I will resolve this issue.