Closed ctroupin closed 2 years ago
I would to everything to make the life of the users simple. So allow for netCDF files without flags, put out a warning and fill in the quality values by default to good and then proceed as usual
I think we have a document somewhere describing how the NetCDF from ODV need to be prepared. Can we include this in the documentation of this function ?
Note that this loop (https://github.com/gher-ulg/DIVAnd.jl/blob/master/src/NCODV.jl#L99) is quite performance critical. We need to be sure that for the common case (flags present), the code is not slower. @ctroupin , can you make a small NetCDF files producing this error ?
Yes, the preparation of the file is described step-by-step here: https://github.com/gher-ulg/Diva-Workshops/blob/master/tricks/ODV_netCDF_export.md and we indeed ask the user to export the quality flags, so the issue should not happen (it did happen because of some specific tests carried out for Reiner).
I agree with @jmbeckers: making the life of the users simple is certainly the good choice. So we can keep this issue as a possible enhancement for the future (maybe not prioritary).
Here is a small file: https://dox.ulg.ac.be/index.php/s/nGmsH5ydCAjgft5
Thanks @ctroupin for the small file. My branch can now read the file with a warning. Can you check the master branch and my branch with a large ODV files (with all the flags) to check if there is a slow down (for what should be the common case)?
ok I'll check that, I have large files that wait to be tested!
First results: reading a large netCDF ODV file (with all the flags and metadata) takes:
Thanks for the info. I guess this is the price to pay for making the code more general/complex. (If somebody wants to try a different approach, feel free to test it).
I'm trying to read an ODV netCDF but it fails with this message:
It seems it is because the variables storing the quality flags were not defined: https://github.com/gher-ulg/DIVAnd.jl/blob/master/src/NCODV.jl#L336
The question: shall we
load
andloadprof
sections or