MethodError after uncomment some lines from DIVAnd_save.jl

[GeoTuxMan]

Hi !

I want to add new additional fields to DIVAnd product (e.g.: error standard deviation of Water body dissolved oxygen concentration, logarithm10 of number of data in bins, logarithm10 of number of outliess data in bins). So, I uncomment these lines from DIVAnd_save.jl : from 258 to 264 ; from 289 to 305

and I got an error:

[ Info: starting DIVA computations for Winter, Spring, Summer, Autumn [ Info: 2022-11-22T11:06:36.433 [ Info: Will write results in C:/Users/Vostro/Documents/my_project_dir_2022/2022\Water_body_dissolved_oxygen_concentration_ALLBS.4Danl.nc [ Info: Creating netCDF file C:/Users/Vostro/Documents/my_project_dir_2022/2022\Water_body_dissolved_oxygen_concentration_ALLBS.4Danl.nc ERROR: LoadError: MethodError: no method matching (::DIVAnd.var"#def4D#374"{DIVAnd.var"#defnD#373"{Int64, DataType, Symbol}})(::NCDataset{Nothing}, ::String, ::Type{Float32}, ::NTuple{4, String}) Closest candidates are: (::DIVAnd.var"#def4D#374")(::Any, ::Any, ::Any) at C:\Users\Vostro.julia\packages\DIVAnd\MV3j9\src\DIVAnd_save.jl:86 Stacktrace: [1] ncfile(ds::NCDataset{Nothing}, filename::String, xyi::Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}, Vector{DateTime}}, varname::String; ncvarattrib::OrderedDict{String, String}, ncglobalattrib::OrderedDict{String, String}, thresholds::Vector{Tuple{String, Float64}}, deflatelevel::Int64, chunksizes::Vector{Int64}, type_save::Type{Float32}, timeorigin::DateTime, checksum::Symbol, saveindex::Tuple{Colon, Colon, Colon}, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:climatology_bounds, :relerr), Tuple{Matrix{DateTime}, Bool}}}) @ DIVAnd C:\Users\Vostro.julia\packages\DIVAnd\MV3j9\src\DIVAnd_save.jl:258 [2] (::DIVAnd.var"#447#463"{Vector{Float64}, OrderedDict{String, String}, OrderedDict{String, String}, typeof(DIVAnd.distfun_m), Nothing, Int64, Bool, Bool, Dict{Any, Any}, Dict{Any, Any}, Int64, Float64, Int64, Float64, Float64, Tuple{}, Bool, Vector{Tuple{String, Float64}}, typeof(DIVAndgo), Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:solver,), Tuple{Symbol}}}, Tuple{Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{DateTime}}, Vector{Float64}, String, String, Bool, Matrix{DateTime}, Dict{Symbol, Any}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Vector{Float64}, Vector{Float64}, Int64, Tuple{Int64, Int64, Int64}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, DIVAnd.Anam.var"#invtrans#12"{DIVAnd.Anam.var"#invtrans#9#13"{Float64, Float64}}, DIVAnd.Anam.var"#trans#10"{DIVAnd.Anam.var"#trans#8#11"{Float64, Float64}}, Vector{DateTime}, TimeSelectorYearListMonthList{Vector{UnitRange{Int64}}, Vector{Vector{Int64}}}, Vector{Float64}, Int64})(ds::NCDataset{Nothing}) @ DIVAnd C:\Users\Vostro.julia\packages\DIVAnd\MV3j9\src\diva.jl:376 [3] NCDataset(::DIVAnd.var"#447#463"{Vector{Float64}, OrderedDict{String, String}, OrderedDict{String, String}, typeof(DIVAnd.distfun_m), Nothing, Int64, Bool, Bool, Dict{Any, Any}, Dict{Any, Any}, Int64, Float64, Int64, Float64, Float64, Tuple{}, Bool, Vector{Tuple{String, Float64}}, typeof(DIVAndgo), Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:solver,), Tuple{Symbol}}}, Tuple{Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{DateTime}}, Vector{Float64}, String, String, Bool, Matrix{DateTime}, Dict{Symbol, Any}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Vector{Float64}, Vector{Float64}, Int64, Tuple{Int64, Int64, Int64}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, DIVAnd.Anam.var"#invtrans#12"{DIVAnd.Anam.var"#invtrans#9#13"{Float64, Float64}}, DIVAnd.Anam.var"#trans#10"{DIVAnd.Anam.var"#trans#8#11"{Float64, Float64}}, Vector{DateTime}, TimeSelectorYearListMonthList{Vector{UnitRange{Int64}}, Vector{Vector{Int64}}}, Vector{Float64}, Int64}, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}) @ NCDatasets C:\Users\Vostro.julia\packages\NCDatasets\ipGBH\src\dataset.jl:241 [4] NCDataset(::Function, ::String, ::Vararg{String}) @ NCDatasets C:\Users\Vostro.julia\packages\NCDatasets\ipGBH\src\dataset.jl:238 [5] diva3d(xi::Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}, TimeSelectorYearListMonthList{Vector{UnitRange{Int64}}, Vector{Vector{Int64}}}}, x::Tuple{Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{DateTime}}, value::Vector{Float64}, len::Tuple{Array{Float64, 3}, Array{Float64, 3}, Array{Float64, 3}}, epsilon2::Vector{Float64}, filename::String, varname::String; datadir::String, bathname::String, bathisglobal::Bool, plotres::typeof(plotres2), timeorigin::DateTime, moddim::Vector{Float64}, zlevel::Symbol, ncvarattrib::OrderedDict{String, String}, ncglobalattrib::OrderedDict{String, String}, transform::Tuple{DIVAnd.Anam.var"#trans#10"{DIVAnd.Anam.var"#trans#8#11"{Float64, Float64}}, DIVAnd.Anam.var"#invtrans#12"{DIVAnd.Anam.var"#invtrans#9#13"{Float64, Float64}}}, distfun::typeof(DIVAnd.distfun_m), mask::Nothing, background::Nothing, background_epsilon2_factor::Nothing, background_lenz::Nothing, background_len::Nothing, background_lenz_factor::Int64, filterbackground::Int64, fitcorrlen::Bool, fithorzcorrlen::Bool, fitvertcorrlen::Bool, fithorz_param::Dict{Any, Any}, fitvert_param::Dict{Any, Any}, memtofit::Int64, overlapfactor::Float64, niter_e::Int64, minfield::Float64, maxfield::Float64, surfextend::Bool, velocity::Tuple{}, stat_per_timeslice::Bool, error_thresholds::Vector{Tuple{String, Float64}}, divamethod::typeof(DIVAndgo), kwargs::Base.Pairs{Symbol, Symbol, Tuple{Symbol}, NamedTuple{(:solver,), Tuple{Symbol}}}) @ DIVAnd C:\Users\Vostro.julia\packages\DIVAnd\MV3j9\src\diva.jl:375 [6] top-level scope @ C:\Users\Vostro\Documents\my_project_dir_2022\O2_AllSeason_ALLBS.jl:375 [7] include(fname::String) @ Base.MainInclude .\client.jl:476 [8] top-level scope @ REPL[2]:1 in expression starting at C:\Users\Vostro\Documents\my_project_dir_2022\O2_AllSeason_ALLBS.jl:375

Thanks ! George

[ctroupin]

Thanks for opening the issue George, I could not reply yesterday, I'm now trying on my machine with your code.

[ctroupin]

Found nothing yet, it took me a while just to reproduce the issue, which is not clear seeing the error message.

@Alexander-Barth @jmbeckers do you remember/know why those lines https://github.com/gher-uliege/DIVAnd.jl/blob/master/src/DIVAnd_save.jl#L258-L264 and https://github.com/gher-uliege/DIVAnd.jl/blob/master/src/DIVAnd_save.jl#L289-L305 were commented out?

[jmbeckers]

I think this has been commented a long time ago and then the way to write the nc files has changed later but without adapting the commented lines. Compare the old code

ncvar_err.attrib["long_name"] = "Error standard deviation of $(longname)"

ncvar_err.attrib["units"] = units

ncvar_err.attrib["valid_min"] = Float32(0.0)

ncvar_err.attrib["valid_max"] = Float32(3.6)

ncvar_err.attrib["_FillValue"] = Float32(fillval)

ncvar_err.attrib["missing_value"] = Float32(fillval)

ncvar_relerr = defVar(ds,"$(varname)_relerr", Float32, ("lon", "lat", "depth", "time"))

ncvar_relerr.attrib["long_name"] = "Relative error of $(longname)"

ncvar_relerr.attrib["valid_min"] = Float32(0.0)

ncvar_relerr.attrib["valid_max"] = Float32(1.0)

ncvar_relerr.attrib["_FillValue"] = Float32(fillval)

ncvar_relerr.attrib["missing_value"] = Float32(fillval)

end

ncvar_err = def4D(ds,"$(varname)_err", Float32, ("lon", "lat",

"depth", "time"))

with the new way to save _relerr.

So maybe just adapt in the same way ? I'm sure Alexander knows what to do but do not know when he can look into that

On 23/11/2022 17:30, Charles Troupin wrote:

Found nothing yet, it took me a while just to reproduce the issue, which is not clear seeing the error message.

@Alexander-Barth https://github.com/Alexander-Barth @jmbeckers https://github.com/jmbeckers do you remember/know why those lines https://github.com/gher-uliege/DIVAnd.jl/blob/master/src/DIVAnd_save.jl#L258-L264 and https://github.com/gher-uliege/DIVAnd.jl/blob/master/src/DIVAnd_save.jl#L289-L305 were commented out?

— Reply to this email directly, view it on GitHub https://github.com/gher-uliege/DIVAnd.jl/issues/111#issuecomment-1325345287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEWMLFTBVZP67GPF5RTJ3WLWJZBBDANCNFSM6AAAAAASI3DWOQ. You are receiving this because you were mentioned.Message ID: @.***>

-- Prof. Jean-Marie Beckers

[ctroupin]

Thanks, yes it's true the change was done some years ago, in fact what @GeoTuxMan is interested in is to have the variable "Logarithm10 of number of data in bins" and it seems the only place where it is defined is in src/DIVAnd_save.jl.

I've checked the last release (2021) of EMODnet Chemistry products and they don't have the variables "Logarithm10 of number of data in bins" and "Logarithm10 of number of outliers data in bins" so probably they are not needed anymore.

[GeoTuxMan]

OK, thank you very much !

[Alexander-Barth]

Actually, these lines have always been commented-out (the code that you see has automatically been generated by the tool ncgen from a NetCDF file from the DIVA-Fortran). The information on the data distribution is now captured in more precision with the obslon/obslat/... variables. Similarly the information on outliers can be saved in a separate file with the DIVAnd.saveobs function (using the exact location, and identifier). There is the function DIVAnd.binning if you want to have this information aggregated on a map.

Concerning the absolute error: I am not sure if the absolute error is really sufficiently reliable to be useful but I hope that I can talk with Jean-Marie next week to get his opinion about it.

George, can you also give us a bit more information on why you want to have the variables? They have not been part of any product generated by DIVAnd (only from DIVA). In particular, it would be useful to know if the accuracy of the absolute error was suitable for your use case.

[GeoTuxMan]

Hi Alex,

Thank you very much for your reply.

Well, I see those variables on Emodnet Chemistry --> Data product --> Map --> Northeast Atlantic Ocean--> Water body Chl --> additional fields

and I want them in my analysis ☺️

Yes, the accuracy of absolute/relative error it's ok for us

[ctroupin]

Yes, true, the Atlantic region has these fields, because they don't work with DIVAnd but with DIVA (the old, 2D, Fortran version).

[jmbeckers]

Do we need to add the feature in DIVAnd or can we close the issue?

[ctroupin]

As those additional fields are not required for EMODnet Chemistry (they were created by the old DIVA, let's close the issue.