search

GEUS-Glaciology-and-Climate / pypromice

Process AWS data from L0 (raw logger) through Lx (end user)
https://pypromice.readthedocs.io
GNU General Public License v2.0
12 stars 4 forks source link

Data flagging/fixing .csv location #85

Closed PennyHow closed 1 year ago

PennyHow commented 1 year ago

Raised by @BaptisteVandecrux:

We need to find an appropriate place for the csv files containng flags and adjustments. It could be on the L0 dataset but then it is not publicly visible. It could also be on https://github.com/GEUS-Glaciology-and-Climate/PROMICE-AWS-data-issues. Then that repository should be cloned or updated everytime pypromice is being run. The overall goal is that whenever I upload a new flag or adjustment to the collection, then the cron job uses it for the next processing.

PennyHow commented 1 year ago

I see that flag files have initially been put in the fixing-data branch under the package source src, but this requires the user to routinely download the newest version of pypromice in order to get flagged data updates. Therefore this is fine for now but may not be great for the long term

My initial feeling is that https://github.com/GEUS-Glaciology-and-Climate/PROMICE-AWS-data-issues is a good home for them, but I haven't given it much thought.

patrickjwright commented 1 year ago

This issue in PROMICE-AWS-data-issues is related.

PennyHow commented 1 year ago

Flagging and fixing files now located under https://github.com/GEUS-Glaciology-and-Climate/PROMICE-AWS-data-issues