JuliaDataCubes / YAXArrays.jl

Yet Another XArray-like Julia package
https://juliadatacubes.github.io/YAXArrays.jl/
Other
101 stars 17 forks source link

mapCube does not work for combination of Dataset and datacube #206

Open felixcremer opened 1 year ago

felixcremer commented 1 year ago

When I want to combine a dataset with another cube in a mapCube call, I run into the dispatch of Tuples and I get the error below. When I am using one of the data cubes in the data set instead of the whole data set, the mapCube works. I would expect this mapCube call to do the respective computation for every subcube of the dataset, because all data cubes share the Timeaxis.

julia> freqsample
YAXArray Dataset
Dimensions: 
   Sample_5393         Axis with 3750 elements: 5393_1 5393_2 .. 5393_3749 5393_3750 
   Sample_28109        Axis with 5069 elements: 28109_1 28109_2 .. 28109_5068 28109_5069 
   Sample_27533        Axis with 5547 elements: 27533_1 27533_2 .. 27533_5546 27533_5547 
   Polarisation        Axis with 2 elements: VH VV 
   Sample_7482         Axis with 2597 elements: 7482_1 7482_2 .. 7482_2596 7482_2597 
   Time                Axis with 100 Elements from 2016-10-03T10:12:28 to 2020-02-09T10:12:44
   Sample_6187         Axis with 5595 elements: 6187_1 6187_2 .. 6187_5594 6187_5595 
   Sample_28045        Axis with 14126 elements: 28045_1 28045_2 .. 28045_14125 28045_14126 
   Sample_7555         Axis with 22929 elements: 7555_1 7555_2 .. 7555_22928 7555_22929 
   Sample_23787        Axis with 3707 elements: 23787_1 23787_2 .. 23787_3706 23787_3707 
   Sample_27588        Axis with 6924 elements: 27588_1 27588_2 .. 27588_6923 27588_6924 
   Sample_10016        Axis with 4324 elements: 10016_1 10016_2 .. 10016_4323 10016_4324 
Variables: ID_5393 ID_6187 ID_7482 ID_7555 ID_10016 ID_23787 ID_27533 ID_27588 ID_28045 ID_28109 

julia> freqsamplenorm = mapCube((x,y)->x ./ y, (freqsample, orgmean), indims=(InDims(:Time), InDims(:Time)), outdims=OutDims(:Time), inplace=false)
ERROR: MethodError: no method matching ndims(::Dataset)
Closest candidates are:
  ndims(::Union{HDF5.Attribute, HDF5.Dataset}) at ~/.julia/packages/HDF5/NqGY2/src/dataspaces.jl:257
  ndims(::DimensionalData.Dimensions.Dimension{<:AbstractArray}) at ~/.julia/packages/DimensionalData/K9D4P/src/Dimensions/dimension.jl:253
  ndims(::DimensionalData.Dimensions.Dimension) at ~/.julia/packages/DimensionalData/K9D4P/src/Dimensions/dimension.jl:252
  ...
Stacktrace:
  [1] dimnames(x::Dataset)
    @ YAXArrayBase ~/.julia/dev/YAXArrayBase/src/axisarrays/axisinterface.jl:46
  [2] caxes(x::Dataset)
    @ YAXArrays.Cubes ~/.julia/dev/YAXArrays/src/Cubes/Cubes.jl:171
  [3] findAxis(desc::YAXArrays.Cubes.Axes.ByName, c::Dataset)
    @ YAXArrays.Cubes.Axes ~/.julia/dev/YAXArrays/src/Cubes/Axes.jl:323
  [4] _broadcast_getindex_evalf
    @ ./broadcast.jl:670 [inlined]
  [5] _broadcast_getindex
    @ ./broadcast.jl:643 [inlined]
  [6] (::Base.Broadcast.var"#29#30"{Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(YAXArrays.Cubes.Axes.findAxis), Tuple{Tuple{YAXArrays.Cubes.Axes.ByName}, Base.RefValue{Dataset}}}})(k::Int64)
    @ Base.Broadcast ./broadcast.jl:1075
  [7] ntuple
    @ ./ntuple.jl:48 [inlined]
  [8] copy(bc::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(YAXArrays.Cubes.Axes.findAxis), Tuple{Tuple{YAXArrays.Cubes.Axes.ByName}, Base.RefValue{Dataset}}})
    @ Base.Broadcast ./broadcast.jl:1075
  [9] materialize
    @ ./broadcast.jl:860 [inlined]
 [10] YAXArrays.DAT.InputCube(c::Dataset, desc::InDims)
    @ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:76
 [11] (::YAXArrays.DAT.var"#30#33")(o::Tuple{Dataset, InDims})
    @ YAXArrays.DAT ./none:0
 [12] iterate
    @ ./generator.jl:47 [inlined]
 [13] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}, Tuple{InDims, InDims}}}, YAXArrays.DAT.var"#30#33"})
    @ Base ./array.jl:787
 [14] YAXArrays.DAT.DATConfig(cdata::Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}, indims::Tuple{InDims, InDims}, outdims::OutDims, inplace::Bool, max_cache::Float64, fu::Function, ispar::Bool, include_loopvars::Bool, allow_irregular::Bool, nthreads::Vector{Int64}, addargs::Tuple{}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:266
 [15] mapCube(::var"#43#44", ::Tuple{Dataset, YAXArray{Union{Missing, Float32}, 2, Matrix{Union{Missing, Float32}}, Vector{CubeAxis}}}; max_cache::Float64, indims::Tuple{InDims, InDims}, outdims::OutDims, inplace::Bool, ispar::Bool, debug::Bool, include_loopvars::Bool, showprog::Bool, irregular_loopranges::Bool, nthreads::Vector{Int64}, loopchunksize::Dict{Any, Any}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:434
 [16] top-level scope
    @ REPL[232]:1