search

GEUS-Glaciology-and-Climate / pypromice

Process AWS data from L0 (raw logger) through Lx (end user)
https://pypromice.readthedocs.io
GNU General Public License v2.0
14 stars 4 forks source link

Resampling step in L2toL3 removed #86

Closed PennyHow closed 1 year ago

PennyHow commented 1 year ago

There was an unneeded resampling step included in L2toL3 that meant all outputs were hourly resolution (including L3 raw, which should be 10min). This was probably leftover from the IDL/GDL processing translation to Python.

This step has been removed, only using the resampling step in the aws object.

Sorry for using an odd branch to pull request this from, but given the data fixing is already being developed from the fixing-data branch, this branch was redundant.

PennyHow commented 1 year ago

@PennyHow this looks like a great catch. I wonder why the hourly resampling was done here originally? Presumably everything in toL3 doesn't care if the data is 10 minute or hourly?

It was in the IDL/GDL processing (v3) which was firstly translated and transferred to Python (by @mankoff). At that point, the aim was to effectively re-create the same output. When I came to adapting and developing it, I think I just missed it.

My only comment is a naming convention. In other modules, the convention of using ds for an xarray dataset is generally followed. Here, ds_h was used to represent hourly. (You can also see a commented line at the end of toL3 where daily is assigned to ds_d.)

Yes, it just looked a bit time-consuming as there are many instances of ds_h in this script. I'll change it to ds.