Daylight saving time

[GoogleCodeExporter]

Wenn du dir mal ein file (level 1) aus dem März anschaust, wird dir auffallen, 
dass in line 7513 die Zeit von "2011-03-27 01:55:00" auf "2011-03-27 03:00:00" 
springt. Das war der Zeitpunkt an dem dieses Jahr auf Sommerzeit umgestellt 
wurde. Daher ist die Zeitreihe eine Stunde kürzer als sie eigentlch sein 
sollte. Im original asc file finde ich 8928 Einträge für März 2011, im 
prozessierten ki_0000hom1_rug_201103010000_201103312355_eat_ca01_nai05_0010.dat 
nur 8916.
Das R-Modul ComputeLevel005File.R produziert die Zeitreihe korrekt (i.e. 8928 
rows). 

Original issue reported on code.google.com by nau...@googlemail.com on 10 Sep 2011 at 9:42

[GoogleCodeExporter]

Original comment by nau...@googlemail.com on 10 Sep 2011 at 9:44

• Changed state: New
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Bis Du sicher, dass R hier richtig produziert?

Ich habe als Beispiel die Dateien
ki_0000hom1_rug_201102180925_201104190925_mez_ra01_nai05_0000.asc
ki_0000hom1_rug_201102181125_201104191125_eat_ca01_nai05_0005.dat
ki_0000hom1_rug_201103010000_201103312355_eat_ca01_nai05_0010.dat

angeschaut.

In der Level-0-Datei sehen die Einträge wie folgt aus (d. h. komplett 
durchgehend von 00:00 bis 04:00 Uhr im 5-Minuten-Takt ohne Lücke):
27.03.11        00:00:00             12.3600         91.8600    
27.03.11        00:05:00             12.4000         92.2200    
27.03.11        00:10:00             12.3600         91.9100    
27.03.11        00:15:00             12.3600         92.1500    
27.03.11        00:20:00             12.2800         91.9300    
27.03.11        00:25:00             12.2400         92.2900    
27.03.11        00:30:00             12.2000         92.2800    
27.03.11        00:35:00             12.1600         92.2500    
27.03.11        00:40:00             12.1200         92.5700    
27.03.11        00:45:00             12.1200         92.4800    
27.03.11        00:50:00             12.0800         92.6400    
27.03.11        00:55:00             12.0400         92.4800    
27.03.11        01:00:00             12.0800         92.7700    
27.03.11        01:05:00             12.0800         92.8200    
27.03.11        01:10:00             12.1200         92.7400    
27.03.11        01:15:00             12.1200         92.8700    
27.03.11        01:20:00             12.1200         92.7400    
27.03.11        01:25:00             12.0400         92.2800    
27.03.11        01:30:00             12.0000         92.6200    
27.03.11        01:35:00             11.9600         92.8700    
27.03.11        01:40:00             11.9600         93.0000    
27.03.11        01:45:00             11.9600         93.3500    
27.03.11        01:50:00             12.0000         93.4900    
27.03.11        01:55:00             12.0000         93.4400    
27.03.11        02:00:00             11.9600         93.1300    
27.03.11        02:05:00             11.9600         93.6100    
27.03.11        02:10:00             11.9600         93.5000    
27.03.11        02:15:00             11.9600         93.6500    
27.03.11        02:20:00             12.0400         93.9100    
27.03.11        02:25:00             12.0400         93.8000    
27.03.11        02:30:00             12.0800         93.8500    
27.03.11        02:35:00             12.0800         93.6800    
27.03.11        02:40:00             12.1600         93.5100    
27.03.11        02:45:00             12.2000         92.8900    
27.03.11        02:50:00             12.1600         92.8800    
27.03.11        02:55:00             12.1200         92.8700    
27.03.11        03:00:00             12.1600         93.2300    
27.03.11        03:05:00             12.2800         93.3200    
27.03.11        03:10:00             12.3600         93.5000    
27.03.11        03:15:00             12.5200         93.1700    
27.03.11        03:20:00             12.7600         92.5800    
27.03.11        03:25:00             12.8800         92.5700    
27.03.11        03:30:00             13.0000         92.5400    
27.03.11        03:35:00             13.0900         92.1600    
27.03.11        03:40:00             13.0900         91.9200    
27.03.11        03:45:00             13.1300         91.8000    
27.03.11        03:50:00             13.0900         91.8700    
27.03.11        03:55:00             13.0900         92.2100    
27.03.11        04:00:00             13.0900         92.6400

In der Level-005-Datei (ich kopiere hier nur die ersten Spalten rein), kommt 
die Zeit zwischen 01:00 und 01:55 (also inkl. aller 5-Minuten-Schritte 
dazwischen) doppelt vor. Dafür fehlt der Zeitraum zwischen 02:00 und 02:55 
Uhr. Das führt dann im nächsten Prozessierungsschritt dazu, dass die Stunde 
zwischen 02:00 und 03:00 ebenfalls fehlt.
"2011-03-27 00:00:00","hom1","rug",12.92
"2011-03-27 00:05:00","hom1","rug",12.88
"2011-03-27 00:10:00","hom1","rug",12.88
"2011-03-27 00:15:00","hom1","rug",12.92
"2011-03-27 00:20:00","hom1","rug",12.92
"2011-03-27 00:25:00","hom1","rug",12.84
"2011-03-27 00:30:00","hom1","rug",12.8
"2011-03-27 00:35:00","hom1","rug",12.84
"2011-03-27 00:40:00","hom1","rug",12.76
"2011-03-27 00:45:00","hom1","rug",12.84
"2011-03-27 00:50:00","hom1","rug",12.84
"2011-03-27 00:55:00","hom1","rug",12.92
"2011-03-27 01:00:00","hom1","rug",12.92
"2011-03-27 01:05:00","hom1","rug",12.92
"2011-03-27 01:10:00","hom1","rug",12.92
"2011-03-27 01:15:00","hom1","rug",12.92
"2011-03-27 01:20:00","hom1","rug",12.84
"2011-03-27 01:25:00","hom1","rug",12.76
"2011-03-27 01:30:00","hom1","rug",12.72
"2011-03-27 01:35:00","hom1","rug",12.64
"2011-03-27 01:40:00","hom1","rug",12.6
"2011-03-27 01:45:00","hom1","rug",12.52
"2011-03-27 01:50:00","hom1","rug",12.52
"2011-03-27 01:55:00","hom1","rug",12.44
"2011-03-27 01:00:00","hom1","rug",12.36
"2011-03-27 01:05:00","hom1","rug",12.4
"2011-03-27 01:10:00","hom1","rug",12.36
"2011-03-27 01:15:00","hom1","rug",12.36
"2011-03-27 01:20:00","hom1","rug",12.28
"2011-03-27 01:25:00","hom1","rug",12.24
"2011-03-27 01:30:00","hom1","rug",12.2
"2011-03-27 01:35:00","hom1","rug",12.16
"2011-03-27 01:40:00","hom1","rug",12.12
"2011-03-27 01:45:00","hom1","rug",12.12
"2011-03-27 01:50:00","hom1","rug",12.08
"2011-03-27 01:55:00","hom1","rug",12.04
"2011-03-27 03:00:00","hom1","rug",12.08
"2011-03-27 03:05:00","hom1","rug",12.08
"2011-03-27 03:10:00","hom1","rug",12.12
"2011-03-27 03:15:00","hom1","rug",12.12
"2011-03-27 03:20:00","hom1","rug",12.12
"2011-03-27 03:25:00","hom1","rug",12.04
"2011-03-27 03:30:00","hom1","rug",12
"2011-03-27 03:35:00","hom1","rug",11.96
"2011-03-27 03:40:00","hom1","rug",11.96
"2011-03-27 03:45:00","hom1","rug",11.96
"2011-03-27 03:50:00","hom1","rug",12
"2011-03-27 03:55:00","hom1","rug",12
"2011-03-27 04:00:00","hom1","rug",11.96

Auf den ersten Blick würde ich hier sagen, dass R bei POSIXct eine 
automatische Umstellung auf Sommerzeit vornimmt, da ISODATE - wenn ich es 
richtig verstehe - GMT als Standart-Zeitzone mit übergibt.

Original comment by nau...@googlemail.com on 12 Sep 2011 at 4:30

• Changed state: Accepted
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Wenn ich das ComputeLvel005File über ein file vom März laufen lasse, sieht 
die dateCET nach Befehl  
dateCET <- as.POSIXct(ISOdate(year,month,day,hour,minute,second))
so aus:

 [9379] "2011-03-26 23:55:00 GMT" "2011-03-27 00:00:00 GMT"
 [9381] "2011-03-27 00:05:00 GMT" "2011-03-27 00:10:00 GMT"
 [9383] "2011-03-27 00:15:00 GMT" "2011-03-27 00:20:00 GMT"
 [9385] "2011-03-27 00:25:00 GMT" "2011-03-27 00:30:00 GMT"
 [9387] "2011-03-27 00:35:00 GMT" "2011-03-27 00:40:00 GMT"
 [9389] "2011-03-27 00:45:00 GMT" "2011-03-27 00:50:00 GMT"
 [9391] "2011-03-27 00:55:00 GMT" "2011-03-27 01:00:00 GMT"
 [9393] "2011-03-27 01:05:00 GMT" "2011-03-27 01:10:00 GMT"
 [9395] "2011-03-27 01:15:00 GMT" "2011-03-27 01:20:00 GMT"
 [9397] "2011-03-27 01:25:00 GMT" "2011-03-27 01:30:00 GMT"
 [9399] "2011-03-27 01:35:00 GMT" "2011-03-27 01:40:00 GMT"
 [9401] "2011-03-27 01:45:00 GMT" "2011-03-27 01:50:00 GMT"
 [9403] "2011-03-27 01:55:00 GMT" "2011-03-27 02:00:00 GMT"
 [9405] "2011-03-27 02:05:00 GMT" "2011-03-27 02:10:00 GMT"
 [9407] "2011-03-27 02:15:00 GMT" "2011-03-27 02:20:00 GMT"
 [9409] "2011-03-27 02:25:00 GMT" "2011-03-27 02:30:00 GMT"
 [9411] "2011-03-27 02:35:00 GMT" "2011-03-27 02:40:00 GMT"
 [9413] "2011-03-27 02:45:00 GMT" "2011-03-27 02:50:00 GMT"
 [9415] "2011-03-27 02:55:00 GMT" "2011-03-27 03:00:00 GMT"
 [9417] "2011-03-27 03:05:00 GMT" "2011-03-27 03:10:00 GMT"
 [9419] "2011-03-27 03:15:00 GMT" "2011-03-27 03:20:00 GMT"
 [9421] "2011-03-27 03:25:00 GMT" "2011-03-27 03:30:00 GMT"
 [9423] "2011-03-27 03:35:00 GMT" "2011-03-27 03:40:00 GMT"

also trotz GMT, keine Zeitumstellung... dann nach den folgenden 2 Befehlen
dateEAT <- dateCET + adjust_time
dateEAT <- as.POSIXct(format(dateEAT, usetz=F))
wo über usetz gesetzt wird dass keine Zeitzone verwendet werden soll:

 [9355] "2011-03-26 23:55:00 EAT" "2011-03-27 00:00:00 EAT"
 [9357] "2011-03-27 00:05:00 EAT" "2011-03-27 00:10:00 EAT"
 [9359] "2011-03-27 00:15:00 EAT" "2011-03-27 00:20:00 EAT"
 [9361] "2011-03-27 00:25:00 EAT" "2011-03-27 00:30:00 EAT"
 [9363] "2011-03-27 00:35:00 EAT" "2011-03-27 00:40:00 EAT"
 [9365] "2011-03-27 00:45:00 EAT" "2011-03-27 00:50:00 EAT"
 [9367] "2011-03-27 00:55:00 EAT" "2011-03-27 01:00:00 EAT"
 [9369] "2011-03-27 01:05:00 EAT" "2011-03-27 01:10:00 EAT"
 [9371] "2011-03-27 01:15:00 EAT" "2011-03-27 01:20:00 EAT"
 [9373] "2011-03-27 01:25:00 EAT" "2011-03-27 01:30:00 EAT"
 [9375] "2011-03-27 01:35:00 EAT" "2011-03-27 01:40:00 EAT"
 [9377] "2011-03-27 01:45:00 EAT" "2011-03-27 01:50:00 EAT"
 [9379] "2011-03-27 01:55:00 EAT" "2011-03-27 02:00:00 EAT"
 [9381] "2011-03-27 02:05:00 EAT" "2011-03-27 02:10:00 EAT"
 [9383] "2011-03-27 02:15:00 EAT" "2011-03-27 02:20:00 EAT"
 [9385] "2011-03-27 02:25:00 EAT" "2011-03-27 02:30:00 EAT"
 [9387] "2011-03-27 02:35:00 EAT" "2011-03-27 02:40:00 EAT"
 [9389] "2011-03-27 02:45:00 EAT" "2011-03-27 02:50:00 EAT"
 [9391] "2011-03-27 02:55:00 EAT" "2011-03-27 03:00:00 EAT"
 [9393] "2011-03-27 03:05:00 EAT" "2011-03-27 03:10:00 EAT"
 [9395] "2011-03-27 03:15:00 EAT" "2011-03-27 03:20:00 EAT"
 [9397] "2011-03-27 03:25:00 EAT" "2011-03-27 03:30:00 EAT"
 [9399] "2011-03-27 03:35:00 EAT" "2011-03-27 03:40:00 EAT"
 [9401] "2011-03-27 03:45:00 EAT" "2011-03-27 03:50:00 EAT"
 [9403] "2011-03-27 03:55:00 EAT" "2011-03-27 04:00:00 EAT"
 [9405] "2011-03-27 04:05:00 EAT" "2011-03-27 04:10:00 EAT"
 [9407] "2011-03-27 04:15:00 EAT" "2011-03-27 04:20:00 EAT"
 [9409] "2011-03-27 04:25:00 EAT" "2011-03-27 04:30:00 EAT"
 [9411] "2011-03-27 04:35:00 EAT" "2011-03-27 04:40:00 EAT"
 [9413] "2011-03-27 04:45:00 EAT" "2011-03-27 04:50:00 EAT"
 [9415] "2011-03-27 04:55:00 EAT" "2011-03-27 05:00:00 EAT"
 [9417] "2011-03-27 05:05:00 EAT" "2011-03-27 05:10:00 EAT"
 [9419] "2011-03-27 05:15:00 EAT" "2011-03-27 05:20:00 EAT"
 [9421] "2011-03-27 05:25:00 EAT" "2011-03-27 05:30:00 EAT"
 [9423] "2011-03-27 05:35:00 EAT" "2011-03-27 05:40:00 EAT"
 [9425] "2011-03-27 05:45:00 EAT" "2011-03-27 05:50:00 EAT"
 [9427] "2011-03-27 05:55:00 EAT" "2011-03-27 06:00:00 EAT"

Hier scheint der Fehler also nicht zu liegen.

Im InitLevel010File.R gehts auch nicht schief:

"dateEAT"
"2011-03-01 00:00:00"
"2011-03-01 00:05:00"
"2011-03-01 00:10:00"
"2011-03-01 00:15:00"
"2011-03-01 00:20:00"
"2011-03-01 00:25:00"
"2011-03-01 00:30:00"
"2011-03-01 00:35:00"
"2011-03-01 00:40:00"
"2011-03-01 00:45:00"
"2011-03-01 00:50:00"
"2011-03-01 00:55:00"
"2011-03-01 01:00:00"
"2011-03-01 01:05:00"
"2011-03-01 01:10:00"
"2011-03-01 01:15:00"
"2011-03-01 01:20:00"
"2011-03-01 01:25:00"
"2011-03-01 01:30:00"
"2011-03-01 01:35:00"
"2011-03-01 01:40:00"
"2011-03-01 01:45:00"
"2011-03-01 01:50:00"
"2011-03-01 01:55:00"
"2011-03-01 02:00:00"
"2011-03-01 02:05:00"
"2011-03-01 02:10:00"
"2011-03-01 02:15:00"
"2011-03-01 02:20:00"
"2011-03-01 02:25:00"
"2011-03-01 02:30:00"
"2011-03-01 02:35:00"
"2011-03-01 02:40:00"
"2011-03-01 02:45:00"
"2011-03-01 02:50:00"
"2011-03-01 02:55:00"
"2011-03-01 03:00:00"
"2011-03-01 03:05:00"
"2011-03-01 03:10:00"
"2011-03-01 03:15:00"
"2011-03-01 03:20:00"
"2011-03-01 03:25:00"
"2011-03-01 03:30:00"
"2011-03-01 03:35:00"
"2011-03-01 03:40:00"
"2011-03-01 03:45:00"
"2011-03-01 03:50:00"
"2011-03-01 03:55:00"
"2011-03-01 04:00:00"...

Original comment by tim.appe...@gmail.com on 12 Sep 2011 at 5:44

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Wenn ich die Datei compute_level_005_file.R, die durch die Python-Routinen 
erstellt wird (Inhalt der Datei ist ganz am Ende reinkopiert) mittels

R --no-save < compute_level_005_file.R

an R übergebe, erhalte ich die in meiner letzten Anmerkung aufgeführte 
Verdopplung von 01:00 bis 01:55 Uhr. Dabei wird dann auch die Datei 
ComputeLevel005File.R verwendet (Inhalt auch ganz unten).

Bei dem doppelten Zeitschritten werden auch nur die Zeiten verdoppelt, nicht 
die Werte; sprich: Beim zweiten 01:00 bis 01:55 werden die Werte zwischen 0:00 
und 00:55 aus der Original-Datei eingetragen. Links sind die Level-0-Werte, 
rechts Auszüge aus der Level-005-Datei (die Angabe hum1 und rug zwischen Datum 
und Werten habe ich weggelassen - die Level-005-Datei wird aber insgesamt 
komplett durch das R-Script mit allen NaN-Feldern etc. erstellt). Verwendetst 
Du eine andere R-Datei?

Verwendest Du ein anderes 

"26.03.11","22:00:00",12.92,91.37                   "2011-03-27 
00:00:00",12.92,91.37
"26.03.11","22:05:00",12.88,90.46                   "2011-03-27 
00:05:00",12.88,90.46
"26.03.11","22:10:00",12.88,90.28                   "2011-03-27 
00:10:00",12.88,90.28
"26.03.11","22:15:00",12.92,90.79                   "2011-03-27 
00:15:00",12.92,90.79
"26.03.11","22:20:00",12.92,90.83                   "2011-03-27 
00:20:00",12.92,90.83
"26.03.11","22:25:00",12.84,90.9                    "2011-03-27 
00:25:00",12.84,90.9
"26.03.11","22:30:00",12.8,91.11                    "2011-03-27 
00:30:00",12.8,91.11
"26.03.11","22:35:00",12.84,91.57                   "2011-03-27 
00:35:00",12.84,91.57
"26.03.11","22:40:00",12.76,91.1                    "2011-03-27 
00:40:00",12.76,91.1
"26.03.11","22:45:00",12.84,90.99                   "2011-03-27 
00:45:00",12.84,90.99
"26.03.11","22:50:00",12.84,90.54                   "2011-03-27 
00:50:00",12.84,90.54
"26.03.11","22:55:00",12.92,90.29                   "2011-03-27 
00:55:00",12.92,90.29
"26.03.11","23:00:00",12.92,90.27                   "2011-03-27 
01:00:00",12.92,90.27
"26.03.11","23:05:00",12.92,90.11                   "2011-03-27 
01:05:00",12.92,90.11
"26.03.11","23:10:00",12.92,90.11                   "2011-03-27 
01:10:00",12.92,90.11
"26.03.11","23:15:00",12.92,90.18                   "2011-03-27 
01:15:00",12.92,90.18
"26.03.11","23:20:00",12.84,90.63                   "2011-03-27 
01:20:00",12.84,90.63
"26.03.11","23:25:00",12.76,89.92                   "2011-03-27 
01:25:00",12.76,89.92
"26.03.11","23:30:00",12.72,90.33                   "2011-03-27 
01:30:00",12.72,90.33
"26.03.11","23:35:00",12.64,90.49                   "2011-03-27 
01:35:00",12.64,90.49
"26.03.11","23:40:00",12.6,91.33                    "2011-03-27 
01:40:00",12.6,91.33
"26.03.11","23:45:00",12.52,91.4                    "2011-03-27 
01:45:00",12.52,91.4
"26.03.11","23:50:00",12.52,91.68                   "2011-03-27 
01:50:00",12.52,91.68
"26.03.11","23:55:00",12.44,91.93                   "2011-03-27 
01:55:00",12.44,91.93
"27.03.11","00:00:00",12.36,91.86                   "2011-03-27 
01:00:00",12.36,91.86
"27.03.11","00:05:00",12.4,92.22                    "2011-03-27 
01:05:00",12.4,92.22
"27.03.11","00:10:00",12.36,91.91                   "2011-03-27 
01:10:00",12.36,91.91
"27.03.11","00:15:00",12.36,92.15                   "2011-03-27 
01:15:00",12.36,92.15
"27.03.11","00:20:00",12.28,91.93                   "2011-03-27 
01:20:00",12.28,91.93
"27.03.11","00:25:00",12.24,92.29                   "2011-03-27 
01:25:00",12.24,92.29
"27.03.11","00:30:00",12.2,92.28                    "2011-03-27 
01:30:00",12.2,92.28
"27.03.11","00:35:00",12.16,92.25                   "2011-03-27 
01:35:00",12.16,92.25
"27.03.11","00:40:00",12.12,92.57                   "2011-03-27 
01:40:00",12.12,92.57
"27.03.11","00:45:00",12.12,92.48                   "2011-03-27 
01:45:00",12.12,92.48
"27.03.11","00:50:00",12.08,92.64                   "2011-03-27 
01:50:00",12.08,92.64
"27.03.11","00:55:00",12.04,92.48                   "2011-03-27 
01:55:00",12.04,92.48
"27.03.11","01:00:00",12.08,92.77                   "2011-03-27 
03:00:00",12.08,92.77
"27.03.11","01:05:00",12.08,92.82                   "2011-03-27 
03:05:00",12.08,92.82
"27.03.11","01:10:00",12.12,92.74                   "2011-03-27 
03:10:00",12.12,92.74
"27.03.11","01:15:00",12.12,92.87                   "2011-03-27 
03:15:00",12.12,92.87
"27.03.11","01:20:00",12.12,92.74                   "2011-03-27 
03:20:00",12.12,92.74
"27.03.11","01:25:00",12.04,92.28                   "2011-03-27 
03:25:00",12.04,92.28
"27.03.11","01:30:00",12,92.62                      "2011-03-27 
03:30:00",12,92.62
"27.03.11","01:35:00",11.96,92.87                   "2011-03-27 
03:35:00",11.96,92.87
"27.03.11","01:40:00",11.96,93                      "2011-03-27 
03:40:00",11.96,93
"27.03.11","01:45:00",11.96,93.35                   "2011-03-27 
03:45:00",11.96,93.35
"27.03.11","01:50:00",12,93.49                      "2011-03-27 
03:50:00",12,93.49
"27.03.11","01:55:00",12,93.44                      "2011-03-27 
03:55:00",12,93.44
"27.03.11","02:00:00",11.96,93.13                   "2011-03-27 
04:00:00",11.96,93.13
"27.03.11","02:05:00",11.96,93.61                   "2011-03-27 
04:05:00",11.96,93.61
"27.03.11","02:10:00",11.96,93.5                    "2011-03-27 
04:10:00",11.96,93.5
"27.03.11","02:15:00",11.96,93.65                   "2011-03-27 
04:15:00",11.96,93.65
"27.03.11","02:20:00",12.04,93.91                   "2011-03-27 
04:20:00",12.04,93.91
"27.03.11","02:25:00",12.04,93.8                    "2011-03-27 
04:25:00",12.04,93.8
"27.03.11","02:30:00",12.08,93.85                   "2011-03-27 
04:30:00",12.08,93.85
"27.03.11","02:35:00",12.08,93.68                   "2011-03-27 
04:35:00",12.08,93.68
"27.03.11","02:40:00",12.16,93.51                   "2011-03-27 
04:40:00",12.16,93.51
"27.03.11","02:45:00",12.2,92.89                    "2011-03-27 
04:45:00",12.2,92.89
"27.03.11","02:50:00",12.16,92.88                   "2011-03-27 
04:50:00",12.16,92.88
"27.03.11","02:55:00",12.12,92.87                   "2011-03-27 
04:55:00",12.12,92.87 
"27.03.11","03:00:00",12.16,93.23                   "2011-03-27 
05:00:00",12.16,93.23

Hier der Inhalt von compute_level_005_file.R:
source("/media/permanent/development/released/julendat-processing-packages/src/k
i_stations/ComputeLevel005File.R")
compute_level_005_file (
asciipath="/media/permanent/development/test/kilimanjaro/metstations/plots/ki/00
00hom1/ra01_nai05_0000/ki_0000hom1_rug_201102180925_201104190925_mez_ra01_nai05_
0000.asc",
outpath="/media/permanent/development/test/kilimanjaro/metstations/plots/ki/0000
hom1/ca01_nai05_0005/ki_0000hom1_rug_201102181125_201104191125_eat_ca01_nai05_00
05.dat",
plotID="hom1",
loggertype="rug",
cf=c(NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN),
reorder=c(1,2,4,5),
skip=5,
quality=0005,
adjust_time=7200,
order_out=c("dateEAT", "plotID", "logger","t_rug", "rh_rug","precip_pu1", 
"p_precip_pu1", "c_precip_pu1","precip_pu2", 
"p_precip_pu2","c_precip_pu2","fog_pu2", "p_fog_pu2", "c_fog_pu2","t_wxt", 
"st_wxt", "rh_wxt","precip_wxt", "ws_wxt", "wd_wxt", "p_wxt","kd_w_wxt", 
"u_kd_wxt", "c_kd_wxt","ku_w_wxt", "u_ku_wxt", "c_ku_wxt","ld_w_wxt", 
"u_ld_wxt", "c_ld_wxt","lu_w_wxt", "u_lu_wxt", "c_lu_wxt", 
"t_cnr_wxt","kd_w_rad", "u_kd_rad", "c_kd_rad","par_m_rad", "u_par_rad", 
"c_par_rad", "t_rad", "rh_rad","chdateEAT", "year", "month", "day", "hour", 
"minute", "quality")) 

Hier der Inhalt von ComputeLevel005File.R:
## define function
compute_level_005_file <- function(asciipath, outpath, plotID, loggertype, 
                       cf, reorder, skip, tz, quality, adjust_time, order_out) {

  ## define order for reading in data and writing out data
  order_in <- c(1:length(order_out))

  ## read in logger data table (temp), create empty table (tab) of same size 
  ## (filled with NAs) and replace columns accoring to defined order
  temp <- read.table(asciipath, col.names=order_in, skip=skip, fill=T)
  tab <- matrix(ncol = length(order_out), nrow = length(temp[,1]))

  for (i in 1:length(reorder)) 
    for (j in 1:length(reorder))
      tab[,reorder[[i]]] <- temp[,i]    

  tab <- as.data.frame(tab)
  names(tab) <- order_out

  ## create date and time variables -- convert to local time (EAT)
  year <- substr(temp[,1],7,8)
  year <- paste("20",year,sep="")
  month <- substr(temp[,1],4,5)
  day <- substr(temp[,1],1,2)
  hour <- substr(temp[,2],1,2)
  minute <- substr(temp[,2],4,5)
  second <- substr(temp[,2],7,8)

  dateCET <- as.POSIXct(ISOdate(year,month,day,hour,minute,second))

  dateEAT <- dateCET + adjust_time

  dateEAT <- as.POSIXct(format(dateEAT, usetz=F))
  dateEAT <- as.character(dateEAT)

  chdateEAT <- strftime(dateEAT, "%Y%m%d%H%M%S")
  chdateEAT <- as.character(chdateEAT)

  year <- substr(chdateEAT,1,4)
  year <- as.character(year)

  month <- substr(chdateEAT,5,6)
  month<- as.character(month)

  day <- substr(chdateEAT,7,8)
  day <- as.character(day)

  hour <- substr(chdateEAT,9,10)
  hour <- as.character(hour)

  minute <- substr(chdateEAT,11,12)
  minute <- as.character(minute)

  second <- substr(chdateEAT,13,14)
  second <- as.character(second)

  tab$dateEAT <- dateEAT
  tab$year <- year
  tab$month <- month
  tab$day <- day
  tab$hour <- hour
  tab$minute <- minute
  tab$chdateEAT <- chdateEAT

  ## create vectors and fill table (tab) accordingly
  quality <- rep(quality, length(year))
  quality <- sprintf("%04d", quality)
  tab$quality <- quality

  plotid <- rep(plotID, length(year))
  plotid <- as.character(plotid)
  tab$plotID <- plotid

  logger <- rep(loggertype, length(year))
  logger <- as.character(logger)
  tab$logger <- logger

  c_precip_pu1 <- rep(cf[1], length(year))
  c_precip_pu1 <- as.numeric(c_precip_pu1)
  tab$precip_pu1 <- tab$p_precip_pu1 * c_precip_pu1
  tab$c_precip_pu1 <- c_precip_pu1

  c_precip_pu2 <- rep(cf[2], length(year))
  c_precip_pu2 <- as.numeric(c_precip_pu2)
  tab$precip_pu2 <- tab$p_precip_pu2 * c_precip_pu2
  tab$c_precip_pu2 <- c_precip_pu2

  c_fog_pu2 <- rep(cf[3], length(year))
  c_fog_pu2 <- as.numeric(c_fog_pu2)
  tab$fog_pu2 <- tab$p_fog_pu2 * c_fog_pu2
  tab$c_fog_pu2 <- c_fog_pu2

  c_kd_wxt <- rep(cf[4] / 1000, length(year))
  c_kd_wxt <- as.numeric(c_kd_wxt)
  tab$c_kd_wxt <- c_kd_wxt

  c_ku_wxt <- rep(cf[5] / 1000, length(year))
  c_ku_wxt <- as.numeric(c_ku_wxt)
  tab$c_ku_wxt <- c_ku_wxt

  c_ld_wxt <- rep(cf[6] / 1000, length(year))
  c_ld_wxt <- as.numeric(c_ld_wxt)
  tab$c_ld_wxt <- c_ld_wxt

  c_lu_wxt <- rep(cf[7] / 1000, length(year))
  c_lu_wxt <- as.numeric(c_lu_wxt)
  tab$c_lu_wxt <- c_lu_wxt

  c_kd_rad <- rep(cf[8], length(year))
  c_kd_rad <- as.numeric(c_kd_rad)
  tab$c_kd_rad <- c_kd_rad

  c_par_rad <- rep(cf[9], length(year))
  c_par_rad <- as.numeric(c_par_rad)
  tab$c_par_rad <- c_par_rad

  ## define Stefan-Boltzman calculations (needed for ld and lu - see below)
  kStefanBoltzman <- 5.672e-08 
  ZeroKelvin <- 273.15              
  SBlaw <- kStefanBoltzman * (tab$t_cnr_wxt + ZeroKelvin)^4

  tab$kd_w_wxt <- tab$u_kd_wxt / c_kd_wxt
  tab$kd_w_wxt <- as.numeric(tab$kd_w_wxt)
  tab$ku_w_wxt <- tab$u_ku_wxt / c_ku_wxt + SBlaw
  tab$ku_w_wxt <- as.numeric(tab$ku_w_wxt)
  tab$kd_w_wxt <- tab$u_kd_wxt / c_kd_wxt
  tab$kd_w_wxt <- as.numeric(tab$kd_w_wxt)
  tab$lu_w_wxt <- tab$u_lu_wxt / c_lu_wxt + SBlaw
  tab$lu_w_wxt <- as.numeric(tab$lu_w_wxt)

  for (i in 1:length(tab)) { 
    if (isTRUE(is.numeric(tab[,i]))) tab[,i] <- round(tab[,i], 4) }

  write.table(tab, outpath, append=F, col.names=T, row.names=F, sep=",", na="NaN")

}

#library(compiler)
#compute_level_005_file.c <- cmpfun(compute_level_005_file)

Original comment by nau...@googlemail.com on 13 Sep 2011 at 8:57

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Moin,
mein ComputeLevel005File.R file ist laut PSPad textdiff deckungsgleich.
Welche R version benutzen wir denn in MR
Hier auf meinem läuft R version 2.13.1 (2011-07-08)

Original comment by tim.appe...@gmail.com on 14 Sep 2011 at 6:06

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Hier ist der output (nur 1. Spalte) von meinem ComputeLevel005File.R mit den 
oben angegebenen Parametern (Comment 4, nur asciipath und outpath 
unterschiedlich) für das file rugged_cof3_201104181022.ASC (müsstest du auch 
haben):

"2011-03-26 23:55:00"
"2011-03-27 00:00:00"
"2011-03-27 00:05:00"
"2011-03-27 00:10:00"
"2011-03-27 00:15:00"
"2011-03-27 00:20:00"
"2011-03-27 00:25:00"
"2011-03-27 00:30:00"
"2011-03-27 00:35:00"
"2011-03-27 00:40:00"
"2011-03-27 00:45:00"
"2011-03-27 00:50:00"
"2011-03-27 00:55:00"
"2011-03-27 01:00:00"
"2011-03-27 01:05:00"
"2011-03-27 01:10:00"
"2011-03-27 01:15:00"
"2011-03-27 01:20:00"
"2011-03-27 01:25:00"
"2011-03-27 01:30:00"
"2011-03-27 01:35:00"
"2011-03-27 01:40:00"
"2011-03-27 01:45:00"
"2011-03-27 01:50:00"
"2011-03-27 01:55:00"
"2011-03-27 02:00:00"
"2011-03-27 02:05:00"
"2011-03-27 02:10:00"
"2011-03-27 02:15:00"
"2011-03-27 02:20:00"
"2011-03-27 02:25:00"
"2011-03-27 02:30:00"
"2011-03-27 02:35:00"
"2011-03-27 02:40:00"
"2011-03-27 02:45:00"
"2011-03-27 02:50:00"
"2011-03-27 02:55:00"
"2011-03-27 03:00:00"
"2011-03-27 03:05:00"
"2011-03-27 03:10:00"
"2011-03-27 03:15:00"
"2011-03-27 03:20:00"
"2011-03-27 03:25:00"
"2011-03-27 03:30:00"
"2011-03-27 03:35:00"
"2011-03-27 03:40:00"
"2011-03-27 03:45:00"
"2011-03-27 03:50:00"
"2011-03-27 03:55:00"
"2011-03-27 04:00:00"

Das von dir oben angebene file habe ich leider nicht als level 0 file.

Original comment by tim.appe...@gmail.com on 14 Sep 2011 at 6:34

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Ich verwende R version 2.13.1 (2011-07-08).

Anbei eine Eingabe- und Ausgabe-Datei inkl. der R-Dateien. Versuchs mal. 
Grundlage rugged_cof3_201104181022.ASC.

Gleiches Resultat: Verdopplung zwischen 1:00 und 1:55.

Wenn ich in der Datei ComputeLevel005File.R die Zeile

dateEAT <- as.POSIXct(format(dateEAT, usetz=F))

auskommentiere, dann erhalte ich ein richtiges Ergebnis.

Original comment by nau...@googlemail.com on 14 Sep 2011 at 8:12

• Added labels: ****
• Removed labels: ****

Attachments:

• ki_0000cof3_rug_201102221225_201104180420_eat_ca01_nai05_0005.dat.bu.zip

[GoogleCodeExporter]

Ok, dann liegt es wohl daran, dass ich Windows verwende und du Linux.
Hier der output von 2 durchläufen mit den von dir angehängten Daten (im 
Anhang)

1) *.ta vom manuellen durchlauf in der R-console

2) *.ta.cmd vom Aufruf R --no-save < compute_level_005_file.R über die 
DOS-console.

Beides mal passts bei mir wie vorher auch.

Probiers doch mal auf nem Windows Rechner im Lab und schau was passiert. Ich 
frag hier mal rum, ob jemand Linux hat und werde es dann hier auch unter Linux 
testen

Original comment by tim.appe...@gmail.com on 14 Sep 2011 at 1:48

• Added labels: ****
• Removed labels: ****

Attachments:

• ki_0000cof3_rug_201102221225_201104180420_eat_ca01_nai05_0005.dat.bu.ta.7z

[GoogleCodeExporter]

Hab grad noch was entdeckt. Mein Rechner hier lief die ganze Zeit auf Nairobi 
Zeitzone. Wenn ich ihn auf Berlin, Wien etc umstelle (automatisches Umstellen 
auf Somer/winterzeit aktiviert), sieht die erste Zeile so aus:

"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-26"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"
"2011-03-27"

also ohne Zeitangabe!! Wenn ich allerdings von dir oben genannte Zeile 
auskommentiere, funktionierts.
Würde fast sagen, dass die Zeile raus sollte aus dem script. Aber teste doch 
bitte nochmal im lab an nem Windows Rechner (hier ist leider niemand mit Linux 
im Moment).
Gute Nacht

Original comment by tim.appe...@gmail.com on 14 Sep 2011 at 6:11

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

ersetz mal

dateEAT <- as.POSIXct(format(dateEAT, usetz=F))

mit

dateEAT <- format(dateEAT, format =  "%Y-%m-%d %H:%M:%S", usetz=F)

dann funktionierts bei mir auch unter Zeitzone Berlin, Wien etc mit 
automatischer Umstellung.

Original comment by tim.appe...@gmail.com on 14 Sep 2011 at 6:34

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Mit dieser Veränderung geht es für Level 005 auch unter Linux (was das 
Wichtige ist - Windows ist für die GUI-Version entscheident, für die 
Level-1-Geschichte ist es Linux, da die Prozessierungsrechner alle Linux sind).

Aber bei der R-Routine zum initialisieren der Level-010-Daten kommt der gleiche 
Fehler zum tragen: Hier das Ergebnis der Initialisierung - es findet ein Sprung 
zwischen 01:55 und 03:00 Uhr statt (nachfolgend nur der Ausschnitt):
"2011-03-27 00:00:00"
"2011-03-27 00:05:00"
"2011-03-27 00:10:00"
"2011-03-27 00:15:00"
"2011-03-27 00:20:00"
"2011-03-27 00:25:00"
"2011-03-27 00:30:00"
"2011-03-27 00:35:00"
"2011-03-27 00:40:00"
"2011-03-27 00:45:00"
"2011-03-27 00:50:00"
"2011-03-27 00:55:00"
"2011-03-27 01:00:00"
"2011-03-27 01:05:00"
"2011-03-27 01:10:00"
"2011-03-27 01:15:00"
"2011-03-27 01:20:00"
"2011-03-27 01:25:00"
"2011-03-27 01:30:00"
"2011-03-27 01:35:00"
"2011-03-27 01:40:00"
"2011-03-27 01:45:00"
"2011-03-27 01:50:00"
"2011-03-27 01:55:00"
"2011-03-27 03:00:00"
"2011-03-27 03:05:00"
"2011-03-27 03:10:00"
"2011-03-27 03:15:00"
"2011-03-27 03:20:00"
"2011-03-27 03:25:00"
"2011-03-27 03:30:00"
"2011-03-27 03:35:00"
"2011-03-27 03:40:00"
"2011-03-27 03:45:00"
"2011-03-27 03:50:00"
"2011-03-27 03:55:00"
"2011-03-27 04:00:00"

Hier muss man wahrscheinlich auch noch was ändern - ich weiß aber nicht was.

Original comment by nau...@googlemail.com on 15 Sep 2011 at 8:37

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

also, hab jetzt den code ein wenig umgeschrieben. Der benutzt nun das 'chron' 
package, welches daylight saving und time zones gar nicht beinhaltet, sodass 
man alles manuell so machen kann wie es sein soll. Hier das upgedatete 
ComputeLevel005File.R. Probier mal ob das nun funktioniert. Dazu natürlich 
erst chron installieren --> install.packages("chron") 

## define function
compute_level_005_file <- function(asciipath, outpath, plotID, loggertype, 
                       cf, reorder, skip, tz, quality, adjust_time, order_out) {

  ## package 'chron' is needed for the handling of dates and times (as it
  ## ignores time zones and daylight saving)
  stopifnot(require("chron"))

  ## define order for reading in data and writing out data
  order_in <- c(1:length(order_out))

  ## read in logger data table (temp), create empty table (tab) of same size 
  ## (filled with NAs) and replace columns accoring to defined order
  temp <- read.table(asciipath, col.names=order_in, skip=skip, fill=T)
  tab <- matrix(ncol = length(order_out), nrow = length(temp[,1]))

  for (i in 1:length(reorder)) 
    for (j in 1:length(reorder))
      tab[,reorder[[i]]] <- temp[,i]    

  tab <- as.data.frame(tab)
  names(tab) <- order_out

  ## create date and time variables -- convert to local time (EAT)
  year <- substr(temp[,1],7,8)
  year <- paste("20",year,sep="")
  month <- substr(temp[,1],4,5)
  day <- substr(temp[,1],1,2)
  hour <- substr(temp[,2],1,2)
  minute <- substr(temp[,2],4,5)
  second <- substr(temp[,2],7,8)

  dateCET <- paste(year,month,day, sep="/")
  timeCET <- paste(hour,minute,second, sep=":")
  options(chron.year.abb = F)
  datetimeCET <- chron(dateCET, timeCET, format = c(dates = "y/m/d", 
                                                    times = "h:m:s"),
                       out.format = c(dates = "y-m-d", times = "h:m:s"))

  dateEAT <- datetimeCET + 0.08333333
  dateEAT <- as.character(dateEAT)

  dateEAT <- gsub("(", "", dateEAT, fixed=T)
  dateEAT <- gsub(")", "", dateEAT, fixed=T)

  chdateEAT <- gsub("-", "", dateEAT)
  chdateEAT <- gsub(" ", "", chdateEAT)
  chdateEAT <- gsub(":", "", chdateEAT)

  year <- substr(chdateEAT,1,4)
  year <- as.character(year)

  month <- substr(chdateEAT,5,6)
  month<- as.character(month)

  day <- substr(chdateEAT,7,8)
  day <- as.character(day)

  hour <- substr(chdateEAT,9,10)
  hour <- as.character(hour)

  minute <- substr(chdateEAT,11,12)
  minute <- as.character(minute)

  second <- substr(chdateEAT,13,14)
  second <- as.character(second)

  tab$dateEAT <- dateEAT
  tab$year <- year
  tab$month <- month
  tab$day <- day
  tab$hour <- hour
  tab$minute <- minute
  tab$chdateEAT <- chdateEAT

  ## create vectors and fill table (tab) accordingly
  quality <- rep(quality, length(year))
  quality <- sprintf("%04d", quality)
  tab$quality <- quality

  plotid <- rep(plotID, length(year))
  plotid <- as.character(plotid)
  tab$plotID <- plotid

  logger <- rep(loggertype, length(year))
  logger <- as.character(logger)
  tab$logger <- logger

  c_precip_pu1 <- rep(cf[1], length(year))
  c_precip_pu1 <- as.numeric(c_precip_pu1)
  tab$precip_pu1 <- tab$p_precip_pu1 * c_precip_pu1
  tab$c_precip_pu1 <- c_precip_pu1

  c_precip_pu2 <- rep(cf[2], length(year))
  c_precip_pu2 <- as.numeric(c_precip_pu2)
  tab$precip_pu2 <- tab$p_precip_pu2 * c_precip_pu2
  tab$c_precip_pu2 <- c_precip_pu2

  c_fog_pu2 <- rep(cf[3], length(year))
  c_fog_pu2 <- as.numeric(c_fog_pu2)
  tab$fog_pu2 <- tab$p_fog_pu2 * c_fog_pu2
  tab$c_fog_pu2 <- c_fog_pu2

  c_kd_wxt <- rep(cf[4] / 1000, length(year))
  c_kd_wxt <- as.numeric(c_kd_wxt)
  tab$c_kd_wxt <- c_kd_wxt

  c_ku_wxt <- rep(cf[5] / 1000, length(year))
  c_ku_wxt <- as.numeric(c_ku_wxt)
  tab$c_ku_wxt <- c_ku_wxt

  c_ld_wxt <- rep(cf[6] / 1000, length(year))
  c_ld_wxt <- as.numeric(c_ld_wxt)
  tab$c_ld_wxt <- c_ld_wxt

  c_lu_wxt <- rep(cf[7] / 1000, length(year))
  c_lu_wxt <- as.numeric(c_lu_wxt)
  tab$c_lu_wxt <- c_lu_wxt

  c_kd_rad <- rep(cf[8], length(year))
  c_kd_rad <- as.numeric(c_kd_rad)
  tab$c_kd_rad <- c_kd_rad

  c_par_rad <- rep(cf[9], length(year))
  c_par_rad <- as.numeric(c_par_rad)
  tab$c_par_rad <- c_par_rad

  ## define Stefan-Boltzman calculations (needed for ld and lu - see below)
  kStefanBoltzman <- 5.672e-08 
  ZeroKelvin <- 273.15              
  SBlaw <- kStefanBoltzman * (tab$t_cnr_wxt + ZeroKelvin)^4

  tab$kd_w_wxt <- tab$u_kd_wxt / c_kd_wxt
  tab$kd_w_wxt <- as.numeric(tab$kd_w_wxt)
  tab$ku_w_wxt <- tab$u_ku_wxt / c_ku_wxt + SBlaw
  tab$ku_w_wxt <- as.numeric(tab$ku_w_wxt)
  tab$kd_w_wxt <- tab$u_kd_wxt / c_kd_wxt
  tab$kd_w_wxt <- as.numeric(tab$kd_w_wxt)
  tab$lu_w_wxt <- tab$u_lu_wxt / c_lu_wxt + SBlaw
  tab$lu_w_wxt <- as.numeric(tab$lu_w_wxt)

  for (i in 1:length(tab)) { 
    if (isTRUE(is.numeric(tab[,i]))) tab[,i] <- round(tab[,i], 4) }

  write.table(tab, outpath, append=F, col.names=T, row.names=F, sep=",", na="NaN")

}

#library(compiler)
#compute_level_005_file.c <- cmpfun(compute_level_005_file)

Leider hatten wir heute den ganzen Tag keinen Strom, sodass ich nun nicht mehr 
weiter arbeiten kann. Ans InitLevel001File.R setze ich mich dann morgen (oder 
so).

Original comment by tim.appe...@gmail.com on 15 Sep 2011 at 4:51

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Funktioniert und die Aktualisierte Version von ComputeLevel005File.R ist 
bereits in der Versionierung.

Original comment by nau...@googlemail.com on 16 Sep 2011 at 8:34

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by nau...@googlemail.com on 16 Sep 2011 at 8:34

• Changed state: Started
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Halt! Bitte noch nicht damit rechnen, denn nun sind ja die neuen wxt logger 
installiert und wie zu erwarten war haben sich die Messstellen verändert. Will 
heissen dass die Reihenfolge Strahlungskomponenten im logger file nun anders 
ist. Muss mir das morgen erst nochmal genau anschauen und werde das dann noch 
anpassen. Wollte das script eh noch ein wnig flexibler machen. Werde mich dann 
auch ans InitLevel010File setzen.

Original comment by tim.appe...@gmail.com on 16 Sep 2011 at 6:11

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Ruhig Blut. Bevor InitLevel010File.R nicht verändert ist, rechnen wir sowieso 
nicht. Aber falls Du das neue 005-File brauchst, kannst Du es jetzt einfach von 
der Seite hier holen.

Original comment by nau...@googlemail.com on 16 Sep 2011 at 7:28

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Hier die neue Version von InitLevel010File.R. Bei der Übergabe von Python muss 
noch geändert werden dass start_time und end_time auch Sekunden haben. Also, 
anstatt:

init_level_010_file(outpath, start_time = "201103270000", end_time = 
"201103270355", time_step = 300)

dann so:

init_level_010_file(outpath, start_time = "20110327000000", end_time = 
"20110327035500", time_step = 300)

Dann müsste es funktionieren

init_level_010_file <- function(outpath, start_time, end_time, time_step) {

  ## we need 'chron' to handle time zone and daylight saving time independent data
  stopifnot(require("chron"))

  ## extract date and time components from start_time
  date_from <- substr(start_time, 1, 8)
  time_from <- substr(start_time, 9, 14)

  ## set 'chron' options to display years as yyyy istead of yy (default)
  options(chron.year.abb = F)

  ## create chron object for start of series
  datetime_from <- chron(date_from, time_from, format = c(dates = "ymd", 
                                                          times = "hms"),
                         out.format = c(dates = "y-m-d", times = "h:m:s"))

  ## extract date and time components from end_time
  date_to <- substr(end_time, 1, 8)
  time_to <- substr(end_time, 9, 14)

  ## create chron object for end of series
  datetime_to <- chron(date_to, time_to, format = c(dates = "ymd", 
                                                    times = "hms"),
                       out.format = c(dates = "y-m-d", times = "h:m:s"))

  ## create chron conform delta t value ("hh:mm:ss")
  ## from time_step (seconds as fractions of day)
  deltat <- times(time_step / 60 / 60 / 24)

  ## create regular time series
  tseries <- seq(from = datetime_from, to = datetime_to, 
                 by = deltat)

  ## tidy up chron object (i.e. delete enclosing brackets)
  tseries <- gsub("(", "", tseries, fixed=T)
  tseries <- gsub(")", "", tseries, fixed=T)

  ## convert time series to data frame for output
  tseries  <- data.frame("dateEAT"=tseries)  

  ## create output
  write.table(tseries, outpath, append=F, col.names=T, row.names=F, sep=",")

}

## EXAMPLE
#init_level_010_file("e:/kili_data/testing/level05/test_level010.dat", 
#                   "20110327000000", "20110327035500", 300)

und jetzt setze ich mich nochmal ans ComputeLevel005File.R und räum das ein 
wenig auf

Original comment by tim.appe...@gmail.com on 17 Sep 2011 at 7:14

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Also, so wie ich das sehe müsste es reichen, wenn wir im 
ki_level01_standards.cnf Zeilen 7+8 und 9+10 tauschen sodass es dann so 
aussieht:

"ld_w_wxt", "u_ld_wxt", "c_ld_wxt"
"lu_w_wxt", "u_lu_wxt", "c_lu_wxt"
"kd_w_wxt", "u_kd_wxt", "c_kd_wxt"
"ku_w_wxt", "u_ku_wxt", "c_ku_wxt", "t_cnr_wxt"

dann müsste ja eigentlich alles andere automatisch angepasst werden. Das 
einzige was sich dann ändert ist die Reihenfolge im output file, aber das ist 
ja prinzipiell egal.
Anbei noch ein file vom neuen wxt logger von der Station hier in Nkweeko 
(gra1b) zu ausprobieren

Original comment by tim.appe...@gmail.com on 17 Sep 2011 at 8:20

• Added labels: ****
• Removed labels: ****

Attachments:

• ki_000gra1b_wxt_201109011300_201109030730_mez_ra01_nai05_0000.asc

[GoogleCodeExporter]

Das Problem mit dem neuen init_level_010_file ist noch folgendes:

Das Level005-File wird mit R so erstellt, dass Mitternacht = 24 Uhr ist, also:
"2011-02-18 23:50:00",...
"2011-02-18 23:55:00",...
"2011-02-18 24:00:00",...
"2011-02-19 00:05:00",...
"2011-02-19 00:10:00",...

In der mittels InitLevel010File.R erstellten und zunächst leeren Datei, ist 
Mitternacht aber sinnvollerweise 0 Uhr, also:
"2011-02-18 23:50:00",...
"2011-02-18 23:55:00",...
"2011-02-18 00:00:00",...
"2011-02-19 00:05:00",...
"2011-02-19 00:10:00",...

Damit ist jeder Zeitslot um Mitternacht leer.

Bezüglich Logger-Umstellung: Das betrifft ja nur die neuen Dateien und wir 
könne es aufschieben, bis die alten prozessiert sind.

Original comment by nau...@googlemail.com on 22 Sep 2011 at 6:55

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

im ComputeLevel005File.R Zeile 40 bitte hiermit ersetzen

  dateEAT <- datetimeCET + adjust_time / 60 / 60 / 24

alternativ kann man sich überlegen, ob man die adjust_time in Stunden anstatt 
Sekunden übergibt, dann wäre die Zeile 

  dateEAT <- datetimeCET + adjust_time / 24

Ergebnis:

"2011-03-26 23:25:00"
"2011-03-26 23:30:00"
"2011-03-26 23:35:00"
"2011-03-26 23:40:00"
"2011-03-26 23:45:00"
"2011-03-26 23:50:00"
"2011-03-26 23:55:00"
"2011-03-27 00:00:00"
"2011-03-27 00:05:00"
"2011-03-27 00:10:00"
"2011-03-27 00:15:00"
"2011-03-27 00:20:00"
"2011-03-27 00:25:00"

Original comment by tim.appe...@gmail.com on 23 Sep 2011 at 6:00

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by nau...@googlemail.com on 24 Sep 2011 at 6:37

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Sommerzeit-Fehler scheint behoben zu sein, aber

InitLevel010File.R endet um 23:50 und nicht 23:55 Uhr im letzten Zeitschritt 
des Monats.Das ist erstmal korrigierbar, wenn man im Aufruf statt 00-Sekunden 
01-Sekunde hinter den letzten Zeitschrit macht.

Original comment by nau...@googlemail.com on 25 Sep 2011 at 11:02

• Changed state: Accepted
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

habe leider nichts gefunden wie man das Problem elegant lösen kann. Habe jetzt 
intern 1 Millisekunde zum letzten Zeitschritt dazugezählt. Damit scheints zu 
funktionieren.

init_level_010_file <- function(outpath, start_time, end_time, time_step) {

  ## we need 'chron' to handle time zone and 
  ## daylight saving time independent data
  stopifnot(require("chron"))

  ## extract date and time components from start_time
  date_from <- substr(start_time, 1, 8)
  time_from <- substr(start_time, 9, 14)

  ## set 'chron' options to display years as yyyy istead of yy (default)
  options(chron.year.abb = F)

  ## create chron object for start of series
  datetime_from <- chron(date_from, time_from, format = c(dates = "ymd", 
                                                          times = "hms"),
                         out.format = c(dates = "y-m-d", times = "h:m:s"))

  ## extract date and time components from end_time
  date_to <- substr(end_time, 1, 8)
  time_to <- substr(end_time, 9, 14)

  ## create chron object for end of series (+ 1 milli second added - otherwise
  ## series excludes last time step)
  datetime_to <- chron(date_to, time_to, format = c(dates = "ymd", 
                                                    times = "hms"),
                       out.format = c(dates = "y-m-d", times = "h:m:s"))
  datetime_to <- datetime_to + 0.001 / 24 / 60 / 60

  ## create chron conform delta t value ("hh:mm:ss")
  ## from time_step (seconds as fractions of day)
  deltat <- times(time_step / 60 / 60 / 24)

  ## create regular time series
  tseries <- seq(from = datetime_from, to = datetime_to, 
                 by = deltat, length.out = series.length)

  ## tidy up chron object (i.e. delete enclosing brackets)
  tseries <- gsub("(", "", tseries, fixed=T)
  tseries <- gsub(")", "", tseries, fixed=T)

  ## convert time series to data frame for output
  tseries  <- data.frame("dateEAT"=tseries)  

  ## create output
  write.table(tseries, outpath, append=F, col.names=T, row.names=F, sep=",")

}

## EXAMPLE
# init_level_010_file("e:/kili_data/testing/level05/test_level001.dat", 
#                   "20110327000000", "20110327235500", 300)

Original comment by tim.appe...@gmail.com on 28 Sep 2011 at 3:41

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by nau...@googlemail.com on 25 May 2012 at 5:46

• Changed state: Verified
• Added labels: ****
• Removed labels: ****