Currently a Variable has its data represented as an OnDiskArray which implements a convenient getindex method. However, this method does not follow all of Julia's getindex rules (e.g. a[1:1,:,:] returns a 2d instead of 3d array), is not type stable, because the element type and array dimensionality are not encoded and does not allow for simple operations like sums, maxima, minima etc. All this would be solved by making OnDiskArray a subtype of AbstractDiskArray from DiskArrays.jl.
Currently NetCDF.jl Zarr.jl and ArchGDAL use the DIskArrays interface to provide an interface for indexing into arrays that live on Disk. If package maintainers think this might be an option, I can prepare a PR.
meggart
commented
3 years ago
Currently a
Variablehas its data represented as an OnDiskArray which implements a convenientgetindexmethod. However, this method does not follow all of Julia's getindex rules (e.g. a[1:1,:,:] returns a 2d instead of 3d array), is not type stable, because the element type and array dimensionality are not encoded and does not allow for simple operations like sums, maxima, minima etc. All this would be solved by makingOnDiskArraya subtype ofAbstractDiskArrayfrom DiskArrays.jl.Currently NetCDF.jl Zarr.jl and ArchGDAL use the DIskArrays interface to provide an interface for indexing into arrays that live on Disk. If package maintainers think this might be an option, I can prepare a PR.