Create a map showing the annual differences between station and Spatial CIMIS estimations

[qjhart]

At the 2015-08-20 Meeting, Bekele expressed concern that the Spatial CIMIS data was introducing a Bias w.r.t. the CIMIS Station data. He had concerns about both the difference in calculation of Solar radiation, as well as changes to the grid values brought on by the spline fit of the data.

[qjhart]

In order to do this, we need to get a set of data comparing CIMIS station data to the raster data. Here is a script that captures the raster data for a list of station data. I had previously got the stations with the command r.in.et -s date=2015-09-10 output=stations

vars=ETo,Tn,Tx,Tdew,Rs,Rso,Rnl,U2,K; 
echo x,y,station,date,$vars > ~/station_raster_data.csv
for m in 20??-??-??; 
  do echo $m; 
  v.out.ascii fs=' ' input=stations | cut -d' ' -f 1,2,4 |\
  sed -e "s/$/,$m/" |\
  r.what input=$(echo $vars | sed -e "s/,/@$m,/g" -e "s/$/@$m/") fs=',' >> ~/station_raster_data.csv; 
done

[qjhart]

In addition, we need to get all the data, (Not just the things we interpolate) from the CIMIS program. This we do with their API. We could also use the ftp site, but this should be more consistent.

appKey=YOUR-APP-KEY
data_items=day-air-tmp-min,day-air-tmp-max,day-air-tmp-avg,day-dew-pnt,day-eto,day-asce-eto,day-precip,day-sol-rad-avg,day-sol-rad-net,day-wind-spd-avg,day-vap-pres-max,day-vap-pres-min;
targets=`seq -s ',' 1 240`;
for t in `seq 1 240`; do echo $t;  
  http "http://et.water.ca.gov/api/data?appKey=${appKey}&targets=$t&startDate=2003-10-01&endDate=2014-09-30&unitOfMeasure=M&dataItems=${data_items}" > cimis_station_$t.json; 
done

 json_items=`echo $data_items | sed -e "s/^\(.\)/.\u\1/" -e "s/,\(.\)/.Value,.\u\1/g" -e "s/\-\(.\)/\u\1/g" -e "s/$/.Value/"`
for t in `seq 1 240; do echo $t; 
  cat cimis_station_$t.json  |\
    jq -r ".Data.Providers[0].Records[] | [.Station,.Date,${json_items}] | @csv" >> cimis_station.csv
done

[qjhart]

Finally, we need to get all the data that we used to create the input files. This is sort of as a double check, but also allows us to see if there is missing data anywhere in our steps.

c='date,day_air_tmp_min,day_air_tmp_min_qc,day_air_tmp_max,day_air_tmp_max_qc,day_wind_spd_avg,day_wind_spd_avg_qc,day_rel_hum_max,day_rel_hum_max_qc,day_dew_pnt,day_dew_pnt_qc';
echo x,y,z,id,${c} > ~/input_station_data; 
for m in 20??-??; do 
  echo $m; 
  v.out.ascii columns=${c} input=et@$m fs=',' >> ~/input_station_data.csv;
done