search

JuliaGeo / GRIBDatasets.jl

A high level interface to GRIB encoded files.
MIT License
12 stars 5 forks source link

Error showing value of type GRIBDataset{Float64, 10} #13

Open jbbremnes opened 10 months ago

jbbremnes commented 10 months ago

Trying the example below with a grib file (186 MB) of forecasts from the Norwegian Meteorological Institute results in the following

julia> ds = GRIBDataset("MEPS_20230816_0600_h_9.grib2")
Dataset: /home/johnbb/tmp/MEPS_20230816_0600_h_9.grib2
Group: /

Dimensions
   x = 949
   y = 1069
   heightAboveGround = 2
   heightAboveGround_2 = 1
   heightAboveGround_3 = 1
   heightAboveGround_4 = 1
   heightAboveGround_5 = 3
   heightAboveGround_6 = 4
   isobaricInhPa = 13
   valid_time = 1

Variables
Error showing value of type GRIBDataset{Float64, 10}:
ERROR: MethodError: no method matching GRIBDatasets.Variable(::GRIBDataset{Float64, 10}, ::String, ::Tuple{GRIBDatasets.MessageDimension{GRIBDatasets.Horizontal}}, ::Matrix{Float64}, ::Dict{String, Any})

Closest candidates are:
  GRIBDatasets.Variable(::TP, ::String, ::Tuple{Vararg{GRIBDatasets.AbstractDim, N}}, ::TA, ::Dict{String, Any}) where {T, N, TA<:Union{Array{T, N}, DiskArrays.AbstractDiskArray{T, N}}, TP}
   @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/variables.jl:101
  GRIBDatasets.Variable(::GRIBDataset, ::Any)
   @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/variables.jl:129
  GRIBDatasets.Variable(::GRIBDataset, ::GRIBDatasets.AbstractDim)
   @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/variables.jl:163

Stacktrace:
  [1] GRIBDatasets.Variable(ds::GRIBDataset{Float64, 10}, dim::GRIBDatasets.MessageDimension{GRIBDatasets.Horizontal})
    @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/variables.jl:166
  [2] GRIBDatasets.Variable(ds::GRIBDataset{Float64, 10}, key::String)
    @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/variables.jl:134
  [3] cfvariable(ds::GRIBDataset{Float64, 10}, varname::String)
    @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/cfvariables.jl:4
  [4] getindex(ds::GRIBDataset{Float64, 10}, key::String)
    @ GRIBDatasets ~/.julia/packages/GRIBDatasets/Ia2Sr/src/dataset.jl:83
  [5] show(io::IOContext{Base.TTY}, ds::GRIBDataset{Float64, 10})
    @ CommonDataModel ~/.julia/packages/CommonDataModel/RSBF3/src/dataset.jl:95
  [6] show(io::IOContext{Base.TTY}, #unused#::MIME{Symbol("text/plain")}, x::GRIBDataset{Float64, 10})
    @ Base.Multimedia ./multimedia.jl:47
  [7] (::REPL.var"#55#56"{REPL.REPLDisplay{REPL.LineEditREPL}, MIME{Symbol("text/plain")}, Base.RefValue{Any}})(io::Any)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:276
  [8] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:557
  [9] display(d::REPL.REPLDisplay, mime::MIME{Symbol("text/plain")}, x::Any)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:262
 [10] display
    @ /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:281 [inlined]
 [11] display(x::Any)
    @ Base.Multimedia ./multimedia.jl:340
 [12] #invokelatest#2
    @ ./essentials.jl:816 [inlined]
 [13] invokelatest
    @ ./essentials.jl:813 [inlined]
 [14] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:305
 [15] (::REPL.var"#57#58"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:287
 [16] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:557
 [17] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:285
 [18] (::REPL.var"#do_respond#80"{Bool, Bool, REPL.var"#93#103"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:899
 [19] (::REPL.var"#98#108"{Regex, Regex, Int64, Int64, REPL.LineEdit.Prompt, REPL.LineEdit.Prompt, REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Any, ::Vararg{Any})
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:1236
 [20] #invokelatest#2
    @ ./essentials.jl:816 [inlined]
 [21] invokelatest
    @ ./essentials.jl:813 [inlined]
 [22] (::REPL.LineEdit.var"#27#28"{REPL.var"#98#108"{Regex, Regex, Int64, Int64, REPL.LineEdit.Prompt, REPL.LineEdit.Prompt, REPL.LineEdit.Prompt}, String})(s::Any, p::Any)
    @ REPL.LineEdit /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/LineEdit.jl:1603
 [23] prompt!(term::REPL.Terminals.TextTerminal, prompt::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/LineEdit.jl:2740
 [24] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/LineEdit.jl:2642
 [25] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL /disk2/software/julia-1.9.0/share/julia/stdlib/v1.9/REPL/src/REPL.jl:1300
 [26] (::REPL.var"#62#68"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ./task.jl:514

Looping through the file using GRIB.jl only works fine. This is in Julia 1.9.0 with the following packages installed

(tg) pkg> st
Status `~/tg/Project.toml`
  [b16dfd50] GRIB v0.3.0
  [82be9cdb] GRIBDatasets v0.2.1
tcarion commented 10 months ago

Thank you for reporting this issue!

This is due to the fact that coordinate variables defined on multiple dimensions (as is the case here with Lambert projection) are currently not handled properly. I will try to find a fix

jbbremnes commented 10 months ago

Thanks for looking into this. If you like, you may add the grib file or part of it to your grib examples.

tcarion commented 7 months ago

@jbbremnes, your file actually brought out multiple issues that wasn't accounted for properly by GRIBDatasets! It should be better now. First the lambert projection case is now handled.

But your file also presents a specific edge case where some of the variables are defined on multiple types of vertical levels. I don't know how to handle this, and it seems that the python cfgrib is not handling it either. So after PR #19 is merged, GRIBDatasets will give a more explanatory error:

julia> ds = GRIBDataset("MEPS_20230816_0600_h_9.grib2");
julia> ds["t"]
ERROR: The variable `t` is defined on multiple types of vertical levels. This is not supported by GRIBDatasets.
To overcome this issue, you can try to filter the GRIB file on some specific level. In your case, try to re-open the dataset with one of:
GRIBDataset("MEPS_20230816_0600_h_9.grib2", filter_by_values=Dict("typeOfLevel" => "heightAboveGround"))
GRIBDataset("MEPS_20230816_0600_h_9.grib2", filter_by_values=Dict("typeOfLevel" => "isobaricInhPa"))
GRIBDataset("MEPS_20230816_0600_h_9.grib2", filter_by_values=Dict("typeOfLevel" => "tropopause"))

As the error suggests, a workaround is to filter your grib file on some specific vertical level. This should work after you do that.

Would you have some smaller file with the same structure ? Like with a croped area ? If not I'll try do modify it myself and add it to the test samples.

jbbremnes commented 7 months ago

Thank you, @tcarion ! We do not have smaller files of similar content, so please feel free to reduce the current one. In December I will look into reading many grib files in parallel.