gher-uliege / DIVAnd.jl

DIVAnd performs an n-dimensional variational analysis of arbitrarily located observations
GNU General Public License v2.0
70 stars 11 forks source link

Problem reading data without local_cdi_id and edmo_code #106

Closed MartinHanssonSMHI closed 1 year ago

MartinHanssonSMHI commented 1 year ago

We have problem reading ODV generated nc-files since the data do not contain local_cdi, edmo_code etc.

datafile2 = "C:/Work/DIVAnd/Test/data/data_from_Seabird_Collection_2022-06-01T13-55-00.nc" @time obsval2, obslon2, obslat2, obsdepth2, obstime2, obsid2 = NCODV.load(Float64, datafile2, "Temperature");

┌ Warning: No variable with the long_name attribute 'LOCAL_CDI_ID' in C:/Work/DIVAnd/Test/data/data_from_Seabird_Collection_2022-06-01T13-55-00.nc found. We use the empty string for LOCAL_CDI_ID instead. └ @ DIVAnd.NCODV C:\Users\a001109.julia\packages\DIVAnd\Iwjb1\src\NCODV.jl:327 No NetCDF variable found with attribute 'long_name' equal to 'EDMO_CODE' in C:\Work\DIVAnd\Test\data\data_from_Seabird_Collection_2022-06-01T13-55-00.nc

Stacktrace: [1] error(s::String) @ Base .\error.jl:33 [2] varbyattrib_first(ds::NCDataset{Nothing}; kwargs::Base.Iterators.Pairs{Symbol, String, Tuple{Symbol}, NamedTuple{(:long_name,), Tuple{String}}}) @ DIVAnd.NCODV C:\Users\a001109.julia\packages\DIVAnd\Iwjb1\src\NCODV.jl:15 [3] (::DIVAnd.NCODV.var"#10#11"{Int64, DataType, String, String, Vector{String}})(ds::NCDataset{Nothing}) @ DIVAnd.NCODV C:\Users\a001109.julia\packages\DIVAnd\Iwjb1\src\NCODV.jl:348 [4] NCDataset(f::DIVAnd.NCODV.var"#10#11"{Int64, DataType, String, String, Vector{String}}, args::String; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}) @ NCDatasets C:\Users\a001109.julia\packages\NCDatasets\XVX8L\src\dataset.jl:220 [5] NCDataset(f::Function, args::String) @ NCDatasets C:\Users\a001109.julia\packages\NCDatasets\XVX8L\src\dataset.jl:218 [6] #load#9 @ C:\Users\a001109.julia\packages\DIVAnd\Iwjb1\src\NCODV.jl:322 [inlined] [7] load(T::Type, fname::String, long_name::String) @ DIVAnd.NCODV C:\Users\a001109.julia\packages\DIVAnd\Iwjb1\src\NCODV.jl:320 [8] top-level scope @ .\timing.jl:210 [inlined] [9] top-level scope @ .\In[11]:0 [10] eval @ .\boot.jl:360 [inlined] [11] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String) @ Base .\loading.jl:1116

Julia 1.6.6

https://github.com/gher-uliege/DIVAnd.jl/blob/fbf6867703b589a9395ccc07d46a369e39e51dac/src/NCODV.jl#L345

Alexander-Barth commented 1 year ago

Even when by-passing the check of EDMO_CODE, I get an error when loading depth. How come that you variable depth variable is not called "Depth" ? I only see these:

      float var20(N_STATIONS, N_SAMPLES) ;
                var20:long_name = "Depth [true depth, m], lat = 58.0168" ;
                var20:units = "" ;
                var20:comment = "" ;
                var20:ancillary_variables = "var20_qc" ;
                var20:C_format = "%.2f" ;
                var20:FORTRAN_format = "F12.2" ;
                var20:_FillValue = -1.e+10f ;

       float var19(N_STATIONS, N_SAMPLES) ;
                var19:long_name = "Depth [salt water, m], lat = 58.0168" ;
                var19:units = "" ;
                var19:comment = "" ;
                var19:ancillary_variables = "var19_qc" ;
                var19:C_format = "%.2f" ;
                var19:FORTRAN_format = "F12.2" ;
                var19:_FillValue = -1.e+10f ;
MartinHanssonSMHI commented 1 year ago

Its generated in the SBE-software while processing and can be remove. We usually use pressure and can rename pressure to depth. MVH/M


Martin Hansson Oceanographic Unit Swedish Meteorological and Hydrological Institute (SMHI) National Oceanographic Data Centre (NODC) Sven Källfelts gata 15 SE-421 71 Västra Frölunda Sweden

Phone: +46 (0) 31 7518957 Mob: +46 (0) 700 900012 www.smhi.se/tema/havsmiljohttp://www.smhi.se/tema/havsmiljo

Från: Alexander Barth @.> Skickat: den 16 september 2022 14:04 Till: gher-uliege/DIVAnd.jl @.> Kopia: Hansson Martin @.>; Author @.> Ämne: Re: [gher-uliege/DIVAnd.jl] Problem reading data without local_cdi_id and edmo_code (Issue #106)

How come that you variable depth variable is not called "Depth" ? I only see these:

  float var20(N_STATIONS, N_SAMPLES) ;

            var20:long_name = "Depth [true depth, m], lat = 58.0168" ;

            var20:units = "" ;

            var20:comment = "" ;

            var20:ancillary_variables = "var20_qc" ;

            var20:C_format = "%.2f" ;

            var20:FORTRAN_format = "F12.2" ;

            var20:_FillValue = -1.e+10f ;

   float var19(N_STATIONS, N_SAMPLES) ;

            var19:long_name = "Depth [salt water, m], lat = 58.0168" ;

            var19:units = "" ;

            var19:comment = "" ;

            var19:ancillary_variables = "var19_qc" ;

            var19:C_format = "%.2f" ;

            var19:FORTRAN_format = "F12.2" ;

            var19:_FillValue = -1.e+10f ;

— Reply to this email directly, view it on GitHubhttps://github.com/gher-uliege/DIVAnd.jl/issues/106#issuecomment-1249280914, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT7EQN2LBGKN4UYHI7AS2ULV6RO3HANCNFSM6AAAAAAQOH6WV4. You are receiving this because you authored the thread.Message ID: @.**@.>>

jmbeckers commented 1 year ago

I think in the old DIVA version we searched for depth and if not found used pressure instead. Would that be a solution or is it better to force DIVAnd users to stick to depth ?

Alexander-Barth commented 1 year ago

Well depth is there, but it is not called in a way that we expected it in DIVAnd. Unfortunately, ODV does not use the CF standard names (nor the BODC names) so we have to rely on long_name, but this assumes that the long_name is consistently named (Depth as in the example files that I got from Reiner, but not Depth [true depth, m], lat = 58.0168). This should be easy that handle in ODV (outside of DIVAnd).

Please feel free to re-open if you have trouble renaming the attribute or ask in the ODV forum.