search

zarr-developers / zeps

Zarr Enhancement Proposals
https://zarr.dev/zeps
Creative Commons Zero v1.0 Universal
12 stars 15 forks source link

Zep 1 - Zarr version 3 #1

Closed alimanfoo closed 2 years ago

alimanfoo commented 2 years ago

This PR adds a new ZEP as a companion to the work that has been done towards a new major version of the Zarr core specifications.

jstriebel commented 2 years ago

Awesome, looking forward to have the initial v3 spec in place! :tada:

From the current ZEP content I'd assume that this ZEP also includes the sharding extension, is that correct? Please let me know if I can help somewhere, e.g. filling in the parts about sharding for Detailed description, Related work and Discussion.

alimanfoo commented 2 years ago

Hi @jstriebel, thanks so much for the offer of help. I've added some more content for the detailed description section. It would be fantastic if you could take a look and do the next editing pass. Please feel free to make edits to the existing content if you can see a way to improve or clarify, and to add any content you think relevant. It would be great if you could add some content about sharding to the relevant sections. I don't know if you can push directly to this PR branch, but if you can feel free to do so.

I left open for the time being whether this ZEP covers the sharding protocol extension or not. The same question applies to the other extensions that are currently published on the zarr-specs site. I'm happy either way, whether to cover them all in this ZEP or add separate ZEPs for each extension, happy to follow advice on how best to handle that (@joshmoore, @MSanKeys963, ...).

alimanfoo commented 2 years ago

Btw as a side note, @jstriebel I took the liberty to list both you and I as editors of the Zarr v3 core spec in the front matter. I was following the W3C convention there of surfacing spec editors. I hope that's OK with you and everyone, but please do let me know if any objections.

jstriebel commented 2 years ago

Hi @jstriebel, thanks so much for the offer of help. I've added some more content for the detailed description section. It would be fantastic if you could take a look and do the next editing pass. Please feel free to make edits to the existing content if you can see a way to improve or clarify, and to add any content you think relevant. It would be great if you could add some content about sharding to the relevant sections. I don't know if you can push directly to this PR branch, but if you can feel free to do so.

Sorry for the late reply. The content looks great to me so far, I just added some initial sections about sharding and storage transformers here: https://github.com/alimanfoo/zeps/pull/1 Not sure if this fits the ZEP format in this way, please feel free to adapt this accordingly.

rabernat commented 2 years ago

It seems like this file should be renamed to draft/ZEP0001.md

https://github.com/zarr-developers/zeps/blob/main/draft/ZEP0001.md

alimanfoo commented 2 years ago

Hi @MSanKeys963, I merged your PR, thanks so much for that. I've also renamed the zep file as requested. So, I think this is ready for merging.