socib / glider_toolbox

MATLAB/Octave scripts to manage data collected by a glider fleet, including data download, data processing and product and figure generation, both in real time and delayed time.
GNU General Public License v3.0
31 stars 23 forks source link

Time lag definition? #10

Open kimmotikka opened 5 years ago

kimmotikka commented 5 years ago

I tried to define timelag for oxygen optodes calphase (23sec) like this:

In processGliderData.m: [..]
options.sensor_lag_list = ... struct('corrected', {'caphase'}, ... 'original', {'calphase'},... 'parameters', {23}); [..]

and in configDataProcessingSlocumG2.m: [..] processing_options.sensor_lag_list = ... struct('corrected', {'cphase'}, ... 'original', {'calphase'},... 'parameters', {23}); [..]

While running I get an error:

Error processing glider deployment data: Error using processGliderData (line 901) Invalid sensor lag settings 1: bad parameter specification.

It would be useful if examples given in comments where with some filled values and not just empty strings.

ctroupin commented 5 years ago

Hello Kimmo,

not an obvious question, especially since I've not opened the toolbox for several years. I've quickly checked the doc and indeed I would help to have an example with realistic values for the parameters.

Could you run the test with 'parameters' set to 'auto' and see if you get the same issue?

options.sensor_lag_list = ...
struct('corrected', {'caphase'}, ...
'original', {'calphase'},...
'parameters', {'auto'});
[..]
kimmotikka commented 5 years ago

Hi Charles,

I tried it, but got an error.

In processGliderData.m ca. row 863:

K>> ~isempty(sensor_lag_option.time) Reference to non-existent field 'time'.

And later: K>> isnumeric(sensor_lag_params) ans = logical 0

K>> numel(sensor_lag_params) ans = 4

—— And ——

K>> sensor_lag_num_params Undefined function or variable 'sensor_lag_num_params’.

—— And ——
strcmpi(sensor_lag_params, 'auto') ans = logical 1

Timelag are not calculated.

The time-sensor is a problem elsewhere also. I have noticed that when the oxygen optode writes values, there is not always ctd-timestamp and ctd values available. That results in problems when writing nc’s. All oxygen values are not included.

With the SeaOWL I set it to write measurements as fast as possible. As it measures with 1Hz frequency, there will be also rows with no ctd’s or oxygen values. In any case sci_m_present_time has a value and thus, I think, it should be the default reference time for all data manipulation.

Kimmo Tikka kimmo.tikka@fmi.fi

Charles Troupin notifications@github.com kirjoitti 12.9.2019 kello 10.21:

Hello Kimmo,

not an obvious question, especially since I've not opened the toolbox for several years. I've quickly checked the doc and indeed I would help to have an example with realistic values for the parameters.

Could you run the test with 'parameters' set to 'auto' and see if you get the same issue?

options.sensor_lag_list = ...

struct( 'corrected', {'caphase'}, ... 'original', {'calphase'},... 'parameters', {'auto' }); [..]

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

joanpau commented 5 years ago

First, this is just a guess, so excuse me for any errors here in advance. I have not been involved with that code in years either.

The original problem seems to come from a configuration error. With pumped CTD where constant flow can be assumed, the correction parameter is a single scalar. Otherwise, the correction parameter depends on the flow and requires two values.

Looking at m/processing_tools/processGliderData.m#L199 the option constant_flow defaults to false. If you are passing a single value precomputed parameter, you are assuming constant flow but this needs to be indicated setting the constant_flow option to true.

You can try in configDataProcessingSlocumG2.m:

processing_options.sensor_lag_list = ...
  struct('corrected', {'cphase'}, 'original', {'calphase'}, ...
            'parameters', {23}, 'constant_flow', {true})

Of course this requires a preexisting calphase original variable, and the default time and depth variables time and depth. You can choose alternative variables by setting the corresponding options in the struct too.