rdstern
commented
4 years ago I therefore suggest we consider the following activities:
-
Following from the ideas above there are 1 (or 2 or 3) obvious packages to add to R-Instat, namely verification (we should add now) and probably hydroGOF. And, if we add hydroGOF we might also consider adding its partner, namely hydroTSM. a) Verification is comprehensive - as far as it goes - in comparing forecast and actual data. That's useful to have anyway. And (for us) the immediate use is to compare satellite and station data. By "comprehensive" I mean that it copes with data of different types. So it can compare continuous/continuous, e.g. temperature values, or categorical/categorical, e.g. heavy rain, rain or no rain - a 3 by 3 table, or binary (2 by 2 table). I have tried the key functions and we may later want to improve on them. But it is sensible to start with them in their current form. The main command is verify and this calls other functions from the package, depending on the type of the data. (It has graphs - not ggplot and I have not yet tried them. The last update was 2015.) b) hydroGOF titles itself as "Goodness-of-Fit Functions for Comparison of Simulated and Observed Hydrological Time Series". This is what @shadrackkibet states is used (or adapted) in the CDT package. Good to have some specialist hydrology packages in R-Instat anyway. Sends the right sort of message to users. c) So we might also consider adding its partner, hydroTSM as well it is "Time Series Management, Analysis and Interpolation for Hydrological Modelling" It includes some potentially interesting data sets at least.
-
If we just add a new package in this way is there anything we can do simply to be able to use functions from a new package reasonably easily. It may just be instructions, or perhaps a new simple dialogue. Here is what I did to use examples from the verify function from the verification package. a) File > New Data Frame and read the data from the example into 2 variables called obs and pred.
b) Then I used Model > Hypothesis Tests > Oneway (obs ~ pred)to be able to access the code. This was as follows:
data1 <- data_book$get_data_frame(data_name="data1")
attach(what=data1)
Last_Test <- oneway.test( obs~pred) Then I swapped the last line above for the lines from the package documentation (roughly), namely:
A<- verification::verify(obs, pred , frcst.type = "binary", obs.type = "binary")
summary(A)
I added the package name in the line above. It ran fine and (for reference) I give some of the results below.
I then slightly edited the remaining lines in the code that had been generated - to save A rather than Last_Test. The code was then:
data_book$add_model(model_name="A", model=A, data_name="data1")
data_book$get_models(data_name="data1", model_name="A")
detach(name=data1, unload=TRUE)
rm(list=c("A", "data1"))This saved the object, A into the data book and I could use Prepare > R Objects > View to look at the contents.
This was all satisfactory. Could we make the use of a new command, from a new package even easier?
a) There could be a dialogue to assist in installing a new package - see #5601.
b) Perhaps a dialogue adding comment lines to a version of the code above, so ready for a new line to be inserted?
- hydroTSM has some interesting sounding data sets. Not many can read into R-Instat, some because they are zoo objects. Could these be used to investigate how a few more structures can be read into a data book in R-Instat?
shadrackkibet
This is an important area to work on and also to add facilities to R-Instat. I collect here the initial e-mail correspondence by Shadrack and Danny. I then consider what we might do in R-Instat, starting with activities in the forthcoming code sprint.
"Dear Roger,
This is just to report some of my findings on the methods used to compare Satellite and in-situ data. Following up on my discussion with Rija (CDT developer) he shared some papers. One looks interesting for me and can be found here. However, I am still going to check others. There are various methods used in this paper which include;
Danny replies as follows:
"Hi Shadrack,
That list looks good. It's similar to what I've seen others using. I've separately sent some papers from Ghana doing comparisons, but I think they mainly use methods from your list.
This site is very comprehensive with methods . It's called methods for forecast verification but it mentions these are general for comparing two columns of data. What I like about the site is it describes what each method will tell you and what it is appropriate for and it's advantages and disadvantages.
Danny"
The site from Danny is excellent. I now use these two messages as background to propose some activities to make the verification easy - and useful - in R-Instat of course.