• Table of contents :toc_2:noexport:

• [[#introduction][Introduction]]

• [[#setup][Setup]]

  • [[#inputs][Inputs]]
  • [[#process][Process]]
  • [[#outputs][Outputs]]

• Introduction

This documents and demonstrates the PROMICE AWS v3 (IDL/GDL) processor

• Setup

** Inputs

• Raw data from https://github.com/GEUS-PROMICE/AWS-data

+BEGIN_SRC bash :results verbatim :exports results

ls ../data/L0/EGP/*.txt

+END_SRC

+RESULTS:

: ../data/L0/EGP/EGP_2016_raw.txt : ../data/L0/EGP/EGP_2017_raw_2.txt : ../data/L0/EGP/EGP_2017_raw_SlimTableMem.txt : ../data/L0/EGP/EGP_2018_raw_2.txt : ../data/L0/EGP/EGP_2018_raw.txt : ../data/L0/EGP/EGP_2019_raw1.txt : ../data/L0/EGP/EGP_2019_raw_transmitted.txt : ../data/L0/EGP/EGP_2019_raw.txt

+BEGIN_SRC bash :exports results

cat ../data/L0/metadata/EGP_metadata.csv | cut -d, -f-7

+END_SRC

+RESULTS:

| file_number | filename | slimtablemem? | transmitted? | header_lines | data_lines | columns | | 1 | EGP_2016_raw.txt | no | no | 3 | 10847 | 43 | | 2 | EGP_2017_raw_SlimTableMem.txt | yes | no | 4 | 7130 | 43 | | 3 | EGP_2017_raw_2.txt | no | no | 4 | 12678 | 43 | | 4 | EGP_2018_raw.txt | no | no | 4 | 42448 | 43 | | 5 | EGP_2018_raw_2.txt | no | no | 4 | 9780 | 43 | | 6 | EGP_2019_raw.txt | no | no | 4 | 43182 | 43 | | 7 | EGP_2019_raw1.txt | no | no | 4 | 8154 | 43 | | 8 | EGP_2019_raw_transmitted.txt | no | yes | 0 | 6350 | 38 |

** Process

+BEGIN_SRC bash :results verbatim :exports both

gdl # can also use IDL .com AWSdataprocessing_v3 ; compile AWSdataprocessing_v3 ; run

+END_SRC

+RESULTS:

+begin_example

% WARNING: your version of the GraphicsMagick library will truncate images to 16 bits per pixel

GDL - GNU Data Language, Version 0.9.9

• For basic information type HELP,/INFO
• Default library routine search path used (GDL_PATH/IDL_PATH env. var. not set): +/usr/share/gnudatalanguage/lib
• No startup file read (GDL_STARTUP/IDL_STARTUP env. var. not set).
• Please report bugs, feature or help requests and patches at: https://github.com/gnudatalanguage/gdl

GDL> .com AW AWSdataprocessing_v3.pro AWSdataprocessing_v3.pro.save
GDL> .com AWSdataprocessing_v3.pro % Compiled module: AWSDATAPROCESSING_V3. GDL> AW AWSdataprocessing_v3.pro AWSdataprocessing_v3.pro.save
GDL> AWSdataprocessing_v3 File: EGP_2016_raw.txt Date and time: 2016/ 5/ 1 14: 30 File: EGP_2017_raw_SlimTableMem.txt Date and time: 2016/ 7/ 19 16: 0 File: EGP_2017_raw_2.txt Date and time: 2017/ 5/ 12 17: 50 File: EGP_2018_raw.txt Date and time: 2017/ 8/ 8 18: 50 File: EGP_2018_raw_2.txt Date and time: 2018/ 5/ 30 20: 40 File: EGP_2019_raw.txt Date and time: 2018/ 8/ 6 18: 40 File: EGP_2019_raw1.txt Date and time: 2019/ 6/ 2 22: 0 File: EGP_2019_raw_transmitted.txt Date and time: 2019/ 7/ 29 12: 0 % Compiled module: INTERPOL. % Compiled module: FINDEX.

• Sun in view of upper sensor / workable albedos: 31038 22%
• Sun below horizon: 57270 40%
• Sun in view of lower sensor: 2178 1%
• Spikes removed using TOA criteria: 5500 3%

• Mean net SR change by corrections: 2.73046 W/m2 done writing instEGP total_lines: 140569 Done with this station. Run time (minutes) = 36.636442

  +end_example

** Outputs

The following files are generated:

+BEGIN_SRC bash :results verbatim :exports results

find ./out

+END_SRC

+RESULTS:

: ./out : ./out/EGP_day_v03.txt : ./out/EGP_inst_v03.txt : ./out/EGP_hour_v03.txt : ./out/EGP_month_v03.txt

As an example, the top few lines and first few columns of [[./data/out/EGP_day_v03.txt]] looks like this:

+BEGIN_SRC bash :exports results

head -n10 ./out/EGP_day_v03.txt | cut -c2- | sed 's/[[:space:]][[:space:]]*/,/g' | cut -d, -f1-7

+END_SRC

+RESULTS:

| Year | MonthOfYear | DayOfMonth | DayOfYear | DayOfCentury | AirPressure(hPa) | AirTemperature(C) | | 2016 | 5 | 1 | 122 | 5966 | -999.0 | -999.0 | | 2016 | 5 | 2 | 123 | 5967 | 723.74 | -27.47 | | 2016 | 5 | 3 | 124 | 5968 | 726.56 | -28.24 | | 2016 | 5 | 4 | 125 | 5969 | 726.98 | -28.79 | | 2016 | 5 | 5 | 126 | 5970 | 726.96 | -27.42 | | 2016 | 5 | 6 | 127 | 5971 | 724.3 | -27.75 | | 2016 | 5 | 7 | 128 | 5972 | 723.02 | -26.73 | | 2016 | 5 | 8 | 129 | 5973 | 724.55 | -25.91 | | 2016 | 5 | 9 | 130 | 5974 | 724.61 | -19.93 |