Closed qcazenave closed 1 year ago
Hello Quitterie,
I haven't read (yet) the solution you proposed in details (due to other constrains right now), however when you say
: they do not have information for the "LOCAL_CDI_ID" metadata. The "id" output variable is thus an empty variable.
we can in fact get the id
s for the WOD observations if we use:
@time obsvalwod,obslonwod,obslatwod,obsdepthwod,obstimewod,obsidwod =
WorldOceanDatabase.load(Float64,woddatadir,varname,prefixid = "1977-");
where prefixid
is used to set the EDMO code of the U.S. NODC.
This gives this type of output:
julia> obsid
2046877-element Vector{String}:
"1977-wod_007663161O"
"1977-wod_007663161O"
ā®
"1977-wod_018508837O"
"1977-wod_018508837O"
and the function WorldOceanDatabase.load
is part of the PhysOcean
module.
Thank you, Charles, for your answer.
So I probably did something wrong with the WOD dataset because when I use the WorldOceanDatabase.load
function, I end up with empty arrays.
And these warnings:
'[ Info: Loading files from C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\CTD\ocldb1692801986.13926.CTD.nc
[ Info: Loading files from C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\GLD\ocldb1692801986.13926.GLD.nc
[ Info: Loading files from C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\ocldb1692801986.13926.OSD.nc
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018588O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018590O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018592O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018594O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018596O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018598O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
ā Warning: File C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\OSD\wod_020018602O.nc does not exist
ā @ PhysOcean.WorldOceanDatabase C:\Users\qcazenav.julia\packages\PhysOcean\zhgOL\src\WorldOceanDatabase.jl:453
[ Info: Loading files from C:\Users\qcazenav\Documents\WOD_NE_Atlantic_Ocean\2022\WOD_raws\PFL\ocldb1692801986.13926.PFL.nc
'
The said missing files are indead missing but there are a lot more so I do not really understand why the output is empty...
yes indeed, strange it's empty.
Just in case: have you followed this?
Load a list profiles under the directory basedir assuming basedir was populated by WorldOceanDatabase.download. If the prefixid is specified, then the observations identifier are prefixed with prefixid.
I must admit I haven't used this function a lot but in the notebooks from the DIVA-workshops
project, the 90-full-analysis.ipynb
provides a good example.
Hi, I found the reason why it didn't work, it was a mistake from my side (not using the correct variable name) so we can close this issue ! Thanks again for your time.
Thanks, good it was too difficult to solve ;)
Dear DIVAnd developers,
I am looking for duplicates between EMODNet Chemistry and the WOD. To do so, I use the DIVAnd.Quadtrees.checkduplicates function as a first guess but then refine the duplicate detection focusing on consistent profiles instead of looking at each observation as if it were independent from the others. For this, I decided to use the information given by the "id" output variable of the function "load" (from NCODV and/or from PhysOcean.WorldOceanDatabase) but there is a problem with the measurements from the WOD : they do not have information for the "LOCAL_CDI_ID" metadata. The "id" output variable is thus an empty variable. Would it be possible to add an option in the "load" function so that if the "LOCAL_CDI_ID" is not available, then another parameter can be used (like the "Station" parameter for example, in the case of the WOD) ? I created locally a development package for DIVAnd to adapt the NCODV.load function (see code below). I was wondering whether it would be possible to modify the "official" packages accordingly (or maybe in a smarter way) ?
Thanks Quitterie
Following is the way I adapted the function:
function load_WOD(T, fname, long_name; qv_flags = ["good_value", "probably_good_value"], nchunk = 10)
[...] char metavar4(N_STATIONS, STRING20) ; metavar4:long_name = "Station" ; [...]
We use the empty string for LOCAL_CDI_ID instead. """ end
[...] char metavar4(N_STATIONS, STRING36) ; metavar4:long_name = "LOCAL_CDI_ID" ; [...]
We use the station name instead. """ ncvar_LOCAL_CDI_ID = varbyattrib_first(ds, long_name = "Station") if ndims(ncvar_LOCAL_CDI_ID) == 2 LOCAL_CDI_ID = chararray2strings(ncvar_LOCAL_CDI_ID.var[:]) else @warn """The variable with the long_name attribute \'Station\' is expected to have two dimensions. For example the output of 'ncdump -h' of $fname should contain: [...] char metavar4(N_STATIONS, STRING20) ; metavar4:long_name = "Station" ; [...]
end