GliderToolsCommunity / GliderTools

A toolkit for processing Seaglider base station NetCDF files: despiking, smoothing, outlier detection, backscatter, fluorescence quenching, calibration, gridding, interpolation.
https://glidertools.readthedocs.io
GNU Affero General Public License v3.0
68 stars 28 forks source link

Versatile profile grouping #175

Closed MartinMohrmann closed 1 year ago

MartinMohrmann commented 1 year ago

[x] Passes pre-commit run --all-files [x] New functions/methods are listed in api.rst

Hi, this change adds a versatile way of grouping the dataset quickly by profiles (using e.g. column dives). The reason I cast the dataset to pandas is a massive speed increase in the grouping (see screenshot), which makes a nice speedup for very small datasets (as shown) and makes it overall possible to group large datasets.

Screenshot from 2023-02-18 15-41-24

We can consider removing utils.time_average_per_dive eventually, because it is a special case of the newly added function. But I will take that later.