earth-mover / icechunk

Open-source, cloud-native transactional tensor storage engine
https://icechunk.io
Apache License 2.0
291 stars 17 forks source link

Add rust native `getsize` implementation #389

Open mpiannucci opened 6 days ago

mpiannucci commented 6 days ago

Currently we use the default zarr python getsize implementation, but we can do something much more efficient in the icechunk core