clbarnes / zarr3-rs

Prototype implementation of zarr v3 in rust
MIT License
18 stars 0 forks source link

Codec refactor: partial enc/de, reduce allocations, assoc. types #13

Open clbarnes opened 1 year ago

clbarnes commented 1 year ago

Associated types

Codecs which currently return boxed readers and writers could use an associated type on the trait instead. Boxed dyn trait objects would probably still be needed for e.g. impl ABCodec for CodecChain and impl BBCodec for &[BBCodec], but it could reduce the number of required boxes. Boxed objects may also still be necessary for partial IO

(en|de)code_into

Where arrays need to be passed around, the output array could also be given (as a mutable view). This would itself be the decoded representation, and reduce allocations/ clones.

Partial (en|de)code

We have partial IO at the store level; codec level is necessary to make sharding worthwhile.