added separation plot and roc plot

[zmjones]

if this needs to be refactored to handle additional formats I'd be happy to do that.

[xfim]

I'm having a hard time trying to make it work with a reproducible example. Can you please provide one?

This is what I'm trying:

N <- 500
X <- runif(N)
z <- 3 + 5 * X 
pr <- 1/(1+exp(-z))
y <- rbinom(N, 1, pr)
D <- list(N=N, X=X, y=y)
md <- '
model {
  for (n in 1:N) {
    y[n] ~ dbern(mu[n])
    logit(mu[n]) <- theta[1] + theta[2] * X[n]
  }
  for (t in 1:2) {
    theta[t] ~ dnorm(0, 0.001)
  }
}'
write(md, file="model-logit.bug")
m <- jags.model("model-logit.bug", data=D,n.chains=3)
par <- c("y")
s <- coda.samples(m, par, n.iter=1E3, thin=5)

ggs_rocplot(s, data=y)
ggs_separation(s, data=y)

And it doesn't work even with some errors (response -> data) and deleting a '+' signal at the end of one line (see commit #0d83225).

[zmjones]

Sorry about the errors. I see you already figured out what was causing it. Again, sorry I didn't check this. That was rather silly of me. Maybe unit tests would be a good idea?

[xfim]

No worries @zmjones. As you have pointed out, I did another sampling 'y' instead of 'mu'.

I will start adapting your code to be more according the package, and trying to follow the new functions ggs_ppmean() and ggs_ppsd(), which use a ggs object as input, instead of a mcmc or mcmc.list. The idea is to make the ggs() function the one that does the dirty work of converting mcmc stuff into data frames.

[zmjones]

ok cool. thanks for the help fixing it.

[dmenne]

The real R-ish way would be to make the functions generic, and let the as.ggs()-magic run behind the screen. So one could explicitly use ggs for when multiple calls are made, or optionally skip it totally. This would not break old code.

[xfim]

I have done a major rewrite of the ggsrocplot() function. Please, @zmjones, take it a look (see commit #365e7f88 and previous ones). In addition to make it more similar to other ggs() functions, I have included the opportunity to make the ROC curve fully bayesian (not reducing the richness of Iteration \ observation a priori). But since it really takes an eternity, I have set by default the non-fully Bayesian version.

Also I have renamed the file to be more consistent with ggs_traceplot.R.

I have taken out the labels, which is something that the user can easily add afterwards with ggs_rocplot(...) + xlab("FPR") + ylab("TPR").

[xfim]

Another major rewrite, now of ggs_separation(). Please, check it again @zmjones (see commit #7321e92).

In both cases, what is missing is the possibility to compare models. In the ggs_separation(), in addition to that the fully Bayesian version has not been yet implemented (and I doubt about its feasibility).

[zmjones]

looks pretty cool. I didn't know about geom_step. I have a few ideas on it but I'll add those to the relevant commits. Thanks for working on this.