search

MetaCell / nwb-explorer

NWB Explorer is a web application to visualise and analyse the content of NWB:N 2 files
Other
23 stars 16 forks source link

GETs remote file twice #264

Open yarikoptic opened 3 years ago

yarikoptic commented 3 years ago

visiting http://nwbexplorer.opensourcebrain.org/nwbfile=https://datasets-dev.datalad.org/dandi/dandisets/000027/sub-RAT123/sub-RAT123.nwb and looking at apache logs, shows 

XXX - - [05/Aug/2021:17:05:10 -0400] "GET /dandi/dandisets/000027/sub-RAT123/sub-RAT123.nwb HTTP/1.1" 200 18792
XXX - - [05/Aug/2021:17:05:18 -0400] "GET /dandi/dandisets/000027/sub-RAT123/sub-RAT123.nwb HTTP/1.1" 200 18792

with I guess first one to display details, and then 2nd one appears shortly after a spinner in Python tab stops spinning so - for that Python session? ... For a more sizeable http://nwbexplorer.opensourcebrain.org/nwbfile=https://datasets-dev.datalad.org/dandi/dandisets/000005/sub-anm184389/sub-anm184389_ses-20130207_behavior+ecephys.nwb - the same effect of two full 286MB download requests.

XXX - - [05/Aug/2021:17:10:55 -0400] "GET /dandi/dandisets/000005/sub-anm184389/sub-anm184389_ses-20130207_behavior+ecephys.nwb HTTP/1.1" 200 299000608
XXX - - [05/Aug/2021:17:11:28 -0400] "GET /dandi/dandisets/000005/sub-anm184389/sub-anm184389_ses-20130207_behavior+ecephys.nwb HTTP/1.1" 200 299000608

It seems that ideally it should be avoided, and a singular GET should be sufficient.

filippomc commented 3 years ago

Hi @yarikoptic thanks for the detailed analysis. Yes this is certainly related to the python console file loading.

A strategy can be to make a local copy of the downloaded file and use that copy for the Python console. Need to ensure that the Python console load starts after the dowload is complete

filippomc commented 3 years ago

Relevant code parts:

Handle action file loading action from the frontend : https://github.com/MetaCell/nwb-explorer/blob/c7e45cd75970465a490e1dbdd20429ce31571fa4/webapp/redux/middleware/nwbMiddleware.js#L117

Trigger notebook load action: https://github.com/MetaCell/nwb-explorer/blob/c7e45cd75970465a490e1dbdd20429ce31571fa4/webapp/redux/middleware/nwbMiddleware.js#L133 The trigger must be changed, here we are not sure that the download was complete.

Backend entry point: https://github.com/MetaCell/nwb-explorer/blob/c7e45cd75970465a490e1dbdd20429ce31571fa4/nwb_explorer/nwb_main.py#L46