pacificclimate / pdp

The PCIC Data Portal - Server software to run the entire web application
GNU General Public License v3.0
1 stars 2 forks source link

FLNRO_WMB data field empty #36

Closed doylejg closed 7 years ago

doylejg commented 8 years ago

Downloaded FLNRO_WMB data from 2011-02-01 -- 2011-02-28 in NetCDF format does not seem to contain data. The headers are fine, but data empty. Furthermore each file is 2.3K

Here is the link I used to download: link

Note, the FRBC data during that time also contains no data for the sites 08HE700 or 08KH029, and there seems to be erroneous data at 2011-02-01 06:00 at the station 08GA71 (the pcptotl=195.5)

Thanks again guys.

basilveerman commented 8 years ago

Hi,

I've just checked the database and we do not have any data available for this time period for any FLNRO-WMB stations. The 2.3K would be the file structure and headers. The same applies to station 08HE700. The reason that this is happening is the 'clip time series to filter date range' does not affect the station selection you see on the map. That is just based on the question 'could there potentially be data at these stations for the selected date range?' which is based on max/min observation at each station. Analysing the actual temporal distribution of data at each station in real time to update the selected stations would take just as long as downloading the file does. Sorry if that causes confusion; we should document this behaviour better.

Regarding station 08KH029, I'm not sure why that would be giving you no data, since it does have data during that period.

crmp=> select native_id, datum, count(*) from obs_raw natural join meta_history natural join meta_station where native_id = '08KH029' and obs_time >= '2011/02/01' and obs_time <= '2011/02/28' group by datum, native_id;
 native_id | datum | count 
-----------+-------+-------
 08KH029   |     0 |   649

Even though all observations are zero, this should still be delivered as zeros so I'll look into what is happening in this case.

Regarding station 08GA071, we do not have the capacity to do any significant quality control on delivered data. We simply mirror what we are delivered by the reporting agency. We do however have a method of flagging potentially incorrect observations, so thanks for letting us know about this one.

-Basil