search

NeurodataWithoutBorders / nwbwidgets

Explore the hierarchical structure of NWB 2.0 files and visualize data with Jupyter widgets.
https://nwb-widgets.readthedocs.io/en/latest/
Other
47 stars 22 forks source link

Where can I get sample data? #2

Open davclark opened 5 years ago

davclark commented 5 years ago

I've set up a sample Gigantum project to demo nwb-jupyter-widgets: gigantum.com/tinydav/nwb-ipy-widget-testing

Unfortunately, it won't work without data. Can you point me to where I might get YutaMouse41-150903.nwb or some other good demo file?

bendichter commented 5 years ago

Hey @davclark! Here's a link: https://drive.google.com/file/d/1ceoaieEpn7NVmU36o-yRTWwJFYjjWdJQ/view?usp=sharing

bendichter commented 5 years ago

LMK if you require smaller file size and I'll get you what you need.

davclark commented 5 years ago

I'm downloading the file you linked now, and I'll figure out a way to work with it. That said, a lot of services limit files to 5GB (e.g., Box, Backblaze, and unfortunately, the current gigantum cloud implementation).

So, something < 5GB would also be nice... and maybe a relatively tiny file for folks who want to quickly try something out on their local machine (running on try.gigantum.com should be pretty fast even for very large files, since storage on S3 will be close to compute).

bendichter commented 5 years ago

Sounds good. I can get you a smaller file, but realistically it will be pretty common for NWB files to be large, so if we want this tool to play well with NWB we're going to have to address this problem eventually.

davclark commented 5 years ago

Multi-part uploads / downloads is in our pipeline! So, I'll definitely plan to have a huge file as part of our demo soon, and hopefully we'll achieve a nice integration. There's even potential to do very clever things with HDF5 on a backend (which is actually something I built for Brian Wandell in like 2006... but he didn't like it at the time). I wonder if it might make sense to explore something like Cloud-optimized GeoTIFF or Zarr at some point.

But anyway, let's not get ahead of ourselves ;)

davclark commented 5 years ago

Ugh. I'm getting:

ValueError: incorrect shape for 'data' (got '(1302080, 32, 8), expected '((None,), (None, None))')

Which I assume is a corrupt download. Historically for me, GDrive has mostly been a way to corrupt large files... do you have a reliable way to get files out of there?

bendichter commented 5 years ago

That's not a corrupt download, it's a pynwb bug I have fixed but haven't yet released. I'll do that today or tomorrow. I use a Globus Connect endpoint for my google drive and that works really well for me.

On Wed, Mar 13, 2019 at 3:29 PM Dav Clark notifications@github.com wrote:

Ugh. I'm getting:

ValueError: incorrect shape for 'data' (got '(1302080, 32, 8), expected '((None,), (None, None))')

Which I assume is a corrupt download. Historically for me, GDrive has mostly been a way to corrupt large files... do you have a reliable way to get files out of there?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NeurodataWithoutBorders/nwb-jupyter-widgets/issues/2#issuecomment-472571538, or mute the thread https://github.com/notifications/unsubscribe-auth/AAziEttgHgIhHJco_vcj_obsVgnSG8xyks5vWVGlgaJpZM4br8H0 .

--

Ben Dichter, PhD Data Science Consultant

bendichter commented 5 years ago

@davclark we just pushed a new release of pynwb, which should fix the bug