aodn / imos-toolbox

Graphical tool for QC'ing and NetCDF'ing oceanographic datasets
GNU General Public License v3.0
46 stars 31 forks source link

Request to "generalize" adcp diagnostic variables related QC routines #752

Open sspagnol opened 3 years ago

sspagnol commented 3 years ago

This is just a request for further work on adcp related routines to not be hardcoded to be assumed as an RDI 4-beam instrument.

As background for some non-IMOS related work had to create a parser for a Rowe Technologies SeaSEVEN 600/1200 adcp, which is a 3-beam per freq instrument. Thankfully only had one freq recorded which was one less hurdle to clear.

Now a lot of the QC routines where written for RDI 4-beam instruments and have hardcoded references to 4 correlation mag or percent good diagnostic variables etc And so for this work had to modify for example imosCorrMagVelocitySetQC.m to first test for velocity variables then get number of beams and test for for them (they may have been smarter ways to do this but I needed something quick).

% get all necessary dimensions and variables id in sample_data struct
idUcur = 0;
idVcur = 0;
idWcur = 0;
idCspd = 0;
idCdir = 0;

% check if the data is compatible with the QC algorithm
% test split to first test for current variables, then
% correlation magnitude, of which the may be more/less than
% assume 4 for RDI workhorse instrument.
lenVar = length(sample_data.variables);
for i=1:lenVar
    paramName = sample_data.variables{i}.name;

    if strncmpi(paramName, 'UCUR', 4),  idUcur = i; end
    if strncmpi(paramName, 'VCUR', 4),  idVcur = i; end
    if strcmpi(paramName, 'WCUR'),      idWcur = i; end
    if strcmpi(paramName, 'CSPD'),      idCspd = i; end
    if strncmpi(paramName, 'CDIR', 4),  idCdir = i; end
end
idMandatory = (idUcur | idVcur | idWcur | idCspd | idCdir);
if ~idMandatory, return; end

% test for correlation magnitude
num_beams = sample_data.meta.adcp_info.number_of_beams;
idCMAG = cell(num_beams, 1);
for j=1:num_beams
    idCMAG{j}  = 0;
end
for i=1:lenVar
    paramName = sample_data.variables{i}.name;
    for j=1:num_beams
        cc = int2str(j);
        if strcmpi(paramName, ['CMAG' cc]), idCMAG{j} = i; end
    end
end
for j=1:num_beams
    idMandatory = idMandatory & idCMAG{j};
end
if ~idMandatory, return; end

Note further modifications required as RDI had CMAG range 0-255 with valid being >= 64, but RTI has CMAG range 0 - 1 with valid being >= 0.25; so had to read a cmag theshold dependent on instrument_make.

There are other hardcoded 4-beam references, but I just focused on correlation magnitude as it it the only one which I think is equivalent in a QA/QC sense. RTI does have a percent good variable which is not equivalent to RDI 4-beam ENU percent good in interpretation (more like 4-beam INSTRUMENT/EARTH reference frame i.e. percent good per beam); and has backscatter variable.

And have noticed that issue https://github.com/aodn/imos-toolbox/issues/751 was raised and if my googling is correct that is a 3-beam instrument and wondered if Mark ran into the same problems?