search

USGS-CMG / stglib

Routines used by the USGS Coastal/Marine Hazards & Resources Program to process oceanographic time-series data
Other
16 stars 14 forks source link

depth trimming using a surface following algorithm #36

Open mmartini-usgs opened 5 years ago

mmartini-usgs commented 5 years ago

This is something we have done in the past for up-looking ADCP profiles. Now that we are moving on to python, do we still want this? Or do we trim to a depth bin that guarantees all data are included? I'm asking because I don't see such a method here in stglib, maybe I missed it.

If so, there are two approaches:

  1. Use pressure to determine the surface location. assumes the pressure measurement is good. Note that older ADCP data will not have pressure. Maybe that data is old enough we don't care.
  2. Use backscatter peak, preferably from the center beam. This has been written in MATLAB already. Does it exist in python? Should it be written?
dnowacki-usgs commented 5 years ago

Hi Marinna, check out: https://github.com/dnowacki-usgs/stglib/blob/bd869605ea36a31ea1f28197d1b2915bb2d9f273/stglib/aqd/qaqc.py#L248 This implements approach 1 in your post. So far only implemented for Aquadopp data; should be able to use the same code for RDI data.

mmartini-usgs commented 5 years ago

Thanks, I did see this code. It uses pressure or water level, the issue of profilers without integral pressure data may come up later - or not. Ellyn says it's really preference, a straight trim at one bin that includes the highest tide would satisfy BBV requirements for us in Woods Hole. It should work on the V and Signature as those instruments do have pressure, if it doesn't I'll comment here.

Where pressure is not available, detecting a peak in acoustic backscatter has been the method to determine the location of the surface. We may end up writing this in python anyway for wave statistics from the center beam of the V and altimeter data from the Signature.

mmartini-usgs commented 5 years ago

It failed right off because the variable names are different for the Aquadopp, I'm working on an EPIC compliant file with u_1205, v_1206 etc.

dnowacki-usgs commented 5 years ago

Commit df821daded5d0669a00dccb3d5edaa6831ec684c should fix this issue and allow you to pass arbitrary values to trim using the data_vars argument. If you have code to detect a backscatter peak, I'd welcome a PR.

https://github.com/dnowacki-usgs/stglib/blob/df821daded5d0669a00dccb3d5edaa6831ec684c/stglib/aqd/qaqc.py#L248

mmartini-usgs commented 5 years ago

OK, while I was waiting I wrote a variant that simply looked up what variables had 'depth' coordinate. Using the statements in your method it trimmed nicely surface following. Now what is the trick for keeping NaNs out of the final written file when it is trimmed down to the highest bin that still contains data? Setting the _FillValues in the new Dataset object to be trimmed back to 1E35 before doing the trim operation did not do the trick. I think you are going to have to explicitly us the _FillValue in the trim operation here: ds = ds.isel(bindist=slice(0, lastbin))