JuliaDataCubes / YAXArrays.jl

Yet Another XArray-like Julia package
https://juliadatacubes.github.io/YAXArrays.jl/
Other
101 stars 18 forks source link

savecube very slow for small chunksizes #350

Open felixcremer opened 11 months ago

felixcremer commented 11 months ago

When I save a cube with small chunk sizes the buffer size is not corrected in get_copy_buffer_size which leads to a very slow saving of the data cube. We might want to increase the buffer size in line 67 of Cubes/Rechunker.jl

Balinus commented 8 months ago

Hello, what would be an example of a "small chunksizes"? Is it an absolute size or is it relative to the size of the Dataset/Cube?

I'm wondering if my writing speed is due to the small chunksizes or the compression. Thanks!