zarr-developers / VirtualiZarr

Create virtual Zarr stores from archival data files using xarray syntax
https://virtualizarr.readthedocs.io/en/stable/api.html
Apache License 2.0
123 stars 24 forks source link

Create a community structure for sharing VirtualiZarr workflows and Icechunk virtual stores / Kerchunk references #320

Open maxrjones opened 9 hours ago

maxrjones commented 9 hours ago

Many Kerchunk workflows were developed as one-off Jupyter Notebooks that were shared on as a GitHub Gist or at most Medium blog posts/conference presentations. While all these examples were fantastic, it was often difficult to find examples and understand their differences. https://github.com/ProjectPythia/kerchunk-cookbook provided a more consistent structure, but was built after the fact by a small number of people. I think it would be valuable to promote a structure for sharing VirtualiZarr workflows earlier in process, so that they are open, findable, and ideally consistently structured. I also think there's a lot to learn from STAC in this type of community organization and would like to propose mirroring the stactools-packages structure. In this model, we would:

I think it would be great if we had a way for people to easily clone their virtual data stores to a publicly accessible location (to my knowledge this isn't in place for STAC). IIRC @norlandrhagen suggested source.coop as a potential hub for sharing the actual virtual stores.