DataBiosphere / data-store

AWS and GCP data storage system for genomic data.
https://dss.dev.ucsc-cgp-redwood.org
Other
3 stars 2 forks source link

Add some documentation on flash-flood (closes #78) #165

Closed natanlao closed 3 years ago

chmreid commented 4 years ago

One other point I wanted to raise - this PR adds a jupyter notebook to the repo, but we don't have any other notebooks in this repo and we don't have jupyter notebook in any of our dependencies.

I am reluctant to add a notebook to this repo, for those reasons and because (as mentioned above) this notebook probably fits better for the flash-flood repo. If the notebook contained data store-specific demonstration of flash-flood usage with a data store deployment, then I'd definitely feel better about adding the first Jupyter notebook to the data store docs.

natanlao commented 4 years ago

The notebook is not specific to the data store - it is more general and reads a bit like an expanded flash-flood readme - so it is probably more appropriate for the flash-flood repository.

My bad - I clearly misunderstood the intent of the original issue. I understand now that you wanted an application of flash flood to the data store and how it might be extended, not a summary of its functionality in the context of our last meeting.

(For example, if I wanted to build a search engine that would index data in the data store, what would the data look like? If I received an event from a flash-flood event stream telling me a bundle was uploaded, how would I access the metadata of the new bundle?)

I'm happy to make the adjustments to flash-flood.md that you've outlined and wait for your PR as guidance.

One other point I wanted to raise - this PR adds a jupyter notebook to the repo, but we don't have any other notebooks in this repo and we don't have jupyter notebook in any of our dependencies.

I always thought "vignette" was synonymous with "Jupyter notebook." Google shows that is not the case. I can convert this to just Markdown, or whichever you think is most appropriate.