search

GEUS-Glaciology-and-Climate / pypromice

Process AWS data from L0 (raw logger) through Lx (end user)
https://pypromice.readthedocs.io
GNU General Public License v2.0
12 stars 4 forks source link

WMO positions & variable skipping #81

Closed patrickjwright closed 1 year ago

patrickjwright commented 1 year ago

This PR has two updates:

  1. Modify the fetch_old_positions function. This is used to grab positions for stations that were otherwise skipped or rejected from BUFR processing, so that we can still write their positions to the csv file (if available). A bug in the previous version of this was preventing some stations from flowing through the function.

  2. Implement a method to skip specific variables for specific stations, using the config file. This was prompted by bad values for p_i that were identified by DMI for CP1 and NAU.

As noted in the code, we need to be aware that if air temperature or pressure are not reporting at all (value is NaN), then the station is skipped entirely, as per min_data_check(). This new config setting therefore only applies to stations that are actually reporting a variable, but we have identified the variable as bad and wish to skip only that variable.

patrickjwright commented 1 year ago

UPDATE: Latest commit creates for more consistency (addressing last comment above). If we are going to allow partial variables through the config, we should allow partial variables in min_data_check also. Bjarne seems to be OK with just removing a single variable if it is missing or bad. With this update, if both air temp and pressure are missing, we will not submit. However, if only one or the other is present, we will proceed.