AllenInstitute / openscope_databook

OpenScope databook: a collaborative, versioned, data-centric collection of foundational analyses for reproducible systems neuroscience 🐁🧠🔬🖥️📈
https://alleninstitute.github.io/openscope_databook
Other
64 stars 19 forks source link

Handle `out of space` error for tests #287

Closed rcpeene closed 1 year ago

rcpeene commented 1 year ago

After adding cebra dependency, the tests all fail before they really start and yield no space left on device; the Github Ubuntu runner can't contain all the dependencies that we have (cebra is large). We need to find a way to get more space.

rcpeene commented 1 year ago

Some options:

jeromelecoq commented 1 year ago

@fcollman up our test unit. We need to confirm this fixes our issues.

jeromelecoq commented 1 year ago

We have issues with simultaneous access to NWB files which are not allowed by h5py. Solution is to force renaming of raw files when running in tests to paralleled access. Potentially use a random string with a env variable boolean set by a GitHub action.

fcollman commented 1 year ago

https://docs.h5py.org/en/stable/mpi.html

this suggests that parallel read is allowed

jeromelecoq commented 1 year ago

Enforcing that all notebook work with a. different session fixes the access issue. We will continue this way for now.