catalystneuro / datta-lab-to-nwb

MIT License
1 stars 2 forks source link

Set up NWB Inspector #25

Closed pauladkisson closed 1 year ago

pauladkisson commented 1 year ago

See docs.

I would love to somehow include NWB inspector in github's CI, but it's probably not feasible bc of file size issues.

CodyCBakerPhD commented 1 year ago

The question is not how to setup the Inspector, but how to setup the entire conversion pipeline to run on stubbed data files

As you say there's an issue with file size so you can't just upload all the raw data for the project - this would be especially problematic for your .pqt files

But for other projects it would not be unreasonable to setup a cache on the project using the GIN testing data, plus perhaps some small behavior/processed examples just to ensure the pipeline is able to run going into the future

In that case you could run a full conversion on the stubbed data files and automatically run the latest version of the inspector over time (especially useful if/when additional checks are added to the inspector) all within the Github Actions

CodyCBakerPhD commented 1 year ago

It's not generally seen as a top priority though - a lot of these projects, especially for this paper - are just one-and-done deals and the code isn't rerun ever again

pauladkisson commented 1 year ago

Closing this since this project is basically one-and-done conversion.