Missing values and quality control

[rdstern]

One issue for the future is to be able to get reports on missing values in the climatic data. This could come through the R link or be programmed directly.

In R-Instat our inventory graphs are an attempt to be able to report on missing values in climatic data, currently particularly in daily data. They look quite good.

This has been particularly important in our general data checking, where we compare what has been entered to what still remains available on the paper records.

However CRAN now recently have a new comprehensive R package imaginatively called "narnia". (Neat, because missing in R is also denoted as NA!) This should be quite easy to add to R-Instat and potentially later to CLIMSOFT if that will be the CLIMSOFT route. This now introduces the issue of whether want to use (or make) routines that are specially developed for climatic data, or we do better by simply making use of facilities that can handle any data.

[isedwards]

@rdstern - we could benefit from your advice on this. We have discussed making using of the R-Instat missing values plot when we successfully link the two pieces of software, and could happily add the capabilites from narnia as well if they are successfully built into R-Instat.

In the meantime, for version 4.0, missing values plots are currently being tackled outside of VB using SQL scripts directly in the database: https://github.com/opencdms/Climsoft/issues/202

Version 4.0 also uses external software like WRPlot to plot wind roses.

The Climsoft draft roadmap aims to add the R products in our version 4.2

[mhabimana]

@rdstern - I remember during my visit at the University of Reading I had a chat with you on the possibility to downscale the inventory graph in near future by adding the options of inventory per station,per year, etc... I wonder if this is already implemented or is still in the pipe.

[rdstern]

@mhabimana this is working quite well. Here is an example of the dialogue for data from 2 stations from Ghana where we have many elements. You can see there are many options for the display of the information

And here is the result with the default display:

[mhabimana]

@rdstern -Thanks Roger, Can you please send through a similar inventory graph for only RR,One station for just one year? I want to see how this look like

[rdstern]

For a single element for one year, we wouldn't do an inventory. We are just finalising the dialogue called Display Daily, which gives a tabular and/or graphical display of the values for one (or more) years.

That should be completed before the visit to Lesotho in a week, so I'll show what it looks like then.

[mhabimana]

@rdstern -Thanks for your quick feedback.

[Fortunata]

This is very much desired if it can be implemented in Climsoft and if it can be scaled down to a single station. If the graph can also show data series for all years which can be implemented by counting the number of stations reporting pr having data each year. It can be shown to administrator or station Networks section to solicit for reopening of more stations.