Error in label range in parquet file

CRIMAC-WP4-Machine-learning / CRIMAC-preprocessing

Preprocessing acoustic data from .raw to a gridded format

GNU Lesser General Public License v3.0

8 stars 6 forks source link

Error in label range in parquet file #6

Closed ingridut closed 3 years ago

ingridut commented 3 years ago

When plotting the annotations from the parquet files, they don't seem to match the data exactly. The labels are plotted a bit below the corresponding high frequency area in the data, as seen in the image below.

parqut_labels

Here is the annotations we have previously used (same raw file) for sake of comparison.

memmap_labels

I have only tested a couple of raw files (2017843-D20170426-T063457 and 2017843-D20170513-T081028). When comparing the depth at the upper left corner of annotations in the parquet files to the depth at the upper left corner in the annotations we have previously used, the difference is between 6 and 8 meters.

iambaim commented 3 years ago

Thanks for this, @ingridut . Me and @sindrevatnehol will have a look at it. Just a question, did any re-gridding happen when processing the raw files?

sindrevatnehol commented 3 years ago

Is the time OK, and is this only for 200 kHz or for all frequencies?

ingridut commented 3 years ago

We don't currently regrid when processing the raw files.
The time seems to be ok, at least in the two files I have checked
In a way, this happens for every frequency. The frequencies matches, in that the high frequency areas overlaps both in time and range, but they are all "out of sync" with the annotations. I only included the 200kHz frequency out of laziness, since this is the frequency closest to the annotation plot in my code and therefore easy to include when I used the snipping tool

sindrevatnehol commented 3 years ago

Since this is on every frequency i guess it is a range from transducer/depth from surface issue. I will see what i can do.

albao11 commented 3 years ago

Just to comment on what @ingridut wrote and regarding @iambaim comment: I am not sure that we do regridding when processing a single raw file.

iambaim commented 3 years ago

Since this is on every frequency i guess it is a range from transducer/depth from surface issue. I will see what i can do.

Perhaps we have to use the transducer_draft to correct the ranges?

sindrevatnehol commented 3 years ago

@iambaim Yes it could be, and perhaps heave. But I think we start with transducer draft and see If this correspond to the offseth

iambaim commented 3 years ago

@ingridut could you try using the transducer_draft and/or heave values stored in the output data object (Zarr) to correct the range errors?

Since you mentioned that the range difference varies and lower than it should be, it might be that the range correction inappropriately applied twice somewhere in the process.

ingridut commented 3 years ago

I have tried to correct the range errors using the heave values, this did not fix the problem. I will have a look at the transducer_draft

ingridut commented 3 years ago

Correcting with the transducer_draft seems to work. The files I have previously worked on were probably already corrected. Thanks, I'll close this issue :)