search

JuliaDataCubes / YAXArrays.jl

Yet Another XArray-like Julia package
https://juliadatacubes.github.io/YAXArrays.jl/
Other
89 stars 14 forks source link

Google Zarr store : MethodError: no method matching rebuild(::String, ::UnitRange{Int64}) #317

Closed Balinus closed 11 months ago

Balinus commented 11 months ago

Hello!

I get an error when trying to read a Zarr store on Google platform. Here's a MWE, and current versions used. 

  [179af706] CFTime v0.1.2
  [336ed68f] CSV v0.10.11
  [a93c6f00] DataFrames v1.6.1
  [0703355e] DimensionalData v0.24.13
  [c27321d9] Glob v1.3.1
  [91a5bcdd] Plots v1.38.17
  [d330b81b] PyPlot v2.11.2
  [1a8c2f83] Query v1.0.0
  [c21b50f5] YAXArrays v0.5.0
  [0a941bbe] Zarr v0.9.1

using YAXArrays
using Zarr
using DimensionalData
store = "gs://cmip6/CMIP6/CMIP/MPI-M/ICON-ESM-LR/historical/r5i1p1f1/day/pr/gn/v20210215/"
g = open_dataset(zopen(store, consolidated=true))

MethodError: no method matching rebuild(::String, ::UnitRange{Int64})

Closest candidates are:
  rebuild(::D, ::Any) where D<:DimensionalData.Dimensions.Dimension
   @ DimensionalData ~/.julia/packages/DimensionalData/4TpBG/src/Dimensions/dimension.jl:157
  rebuild(::YAXArray, ::AbstractArray, ::Tuple, ::Tuple, ::Any, ::Any)
   @ YAXArrays ~/.julia/packages/YAXArrays/lvvMa/src/Cubes/Cubes.jl:184
  rebuild(::AbstractDimStack, ::Any)
   @ DimensionalData ~/.julia/packages/DimensionalData/4TpBG/src/stack/stack.jl:135
  ...

Stacktrace:
  [1] toaxis(dimname::String, g::YAXArrayBase.ZarrDataset, offs::Int64, len::Int64)
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/lvvMa/src/DatasetAPI/Datasets.jl:222
  [2] (::YAXArrays.Datasets.var"#37#39"{YAXArrayBase.ZarrDataset})(d::Tuple{String, Int64, Int64})
    @ YAXArrays.Datasets ./none:0
  [3] iterate
    @ ./generator.jl:47 [inlined]
  [4] _all(f::Base.var"#372#374", itr::Base.Generator{Set{Tuple{String, Int64, Int64}}, YAXArrays.Datasets.var"#37#39"{YAXArrayBase.ZarrDataset}}, #unused#::Colon)
    @ Base ./reduce.jl:1292
  [5] all
    @ ./reduce.jl:1278 [inlined]
  [6] Dict(kv::Base.Generator{Set{Tuple{String, Int64, Int64}}, YAXArrays.Datasets.var"#37#39"{YAXArrayBase.ZarrDataset}})
    @ Base ./dict.jl:111
  [7] collectdims(g::YAXArrayBase.ZarrDataset)
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/lvvMa/src/DatasetAPI/Datasets.jl:213
  [8] open_dataset(g::ZGroup{GCStore}; driver::Symbol)
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/lvvMa/src/DatasetAPI/Datasets.jl:293
  [9] open_dataset(g::ZGroup{GCStore})
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/lvvMa/src/DatasetAPI/Datasets.jl:290
 [10] top-level scope
    @ In[46]:5
Balinus commented 11 months ago

Forgot to add that the zstore seems reachable:

store = "gs://cmip6/CMIP6/CMIP/MPI-M/ICON-ESM-LR/historical/r5i1p1f1/day/pr/gn/v20210215/"
zopen(store, consolidated=true)

ZarrGroup at GCStore("cmip6") and path CMIP6/CMIP/MPI-M/ICON-ESM-LR/historical/r5i1p1f1/day/pr/gn/v20210215/
Variables: time_bnds latitude time longitude vertices_longitude pr vertices_latitude i
lazarusA commented 11 months ago

I just did a small PR to fix this. Here, in this branch

https://github.com/JuliaDataCubes/YAXArrays.jl/tree/fix_rebuild

waiting now for the tests to pass. In the mean time you could use this branch to open your data set.

lazarusA commented 11 months ago

fix by https://github.com/JuliaDataCubes/YAXArrays.jl/pull/318 in master.

Balinus commented 11 months ago

Great! Will test tomorrow morning.

Cheers!

Balinus commented 11 months ago

@lazarusA, I confirm it solves the problem of this particular dataset.

Thanks! :)