JuliaDataCubes / YAXArrays.jl

Yet Another XArray-like Julia package
https://juliadatacubes.github.io/YAXArrays.jl/
Other
103 stars 18 forks source link

Cannot set the outtype of a mapCube call to Union{Missing, Float32} #213

Open felixcremer opened 1 year ago

felixcremer commented 1 year ago

I am trying to map a function which does not work out for all input pixels over a data cube which does not have Missing values. I wanted to convey the To allow for missing values in the output cube I tried to use the outtype keyword argument, but I get the following error:

julia> q5cube = mapCube(quant, s1db, 0.05, indims=InDims("Time"), outdims=OutDims(outax, outtype=Union{Missing, Float32}))
ERROR: MethodError: Cannot `convert` an object of type 
  Type{Union{Missing, Float32}} to an object of type 
  Union{Int64, DataType}
Closest candidates are:
  convert(::Type, ::GeoInterface.AbstractGeometryTrait, ::Any) at ~/.julia/packages/GeoInterface/J298z/src/fallbacks.jl:112
  convert(::Type{T}, ::T) where T at Base.jl:61
Stacktrace:
 [1] OutDims(axisdesc::Tuple{YAXArrays.Cubes.Axes.ByValue}, backend::Symbol, backendargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, update::Bool, artype::Type, chunksize::Symbol, outtype::Type)
   @ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/registration.jl:94
 [2] OutDims(axisdesc::CategoricalAxis{String, :Quantiles, Vector{String}}; backend::Symbol, update::Bool, artype::Type, chunksize::Symbol, outtype::Type, backendargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/registration.jl:126
 [3] top-level scope
   @ REPL[333]:1

Can we just widen the allowed type for the outtype value in OutDims?

meggart commented 1 year ago

Actually this should not be necessary. I had a similar discussion with @gdkrmr yesterday and thought output types were automatically widened to allow missings here. Does your output array not allow missings when you simply set outtype to Float32?

gdkrmr commented 1 year ago

Now that we allow the use of NaN instead of missing, I think we should not automatically use a Union{Missing, ....} any if the user wants to use missing, they can specify outtype = Union{Missing, Float32}.

Of course this is a breaking change and should not be done tomorrow but in the next major release.

felixcremer commented 1 year ago

I am currently trying to use the result of a mapcube in the ImageMorphology package which expects an AbstractArray{Int,2} and can't deal with missing. For these use cases, I think it would be good to not automatically widen the outtype to Union{Missing, ...} when the user specifies it explicitely, because it is also unintuitive, that the outtype is not what I set in the OutDims constructor.