MCMC_roc_prc: generalize for other input

[jkarreth]

including JAGS/BUGS

[jkarreth]

Right now this function conveniently uses rstanarm::posterior_linpred(rstanarm_object, transform = TRUE) to generate pr(y = 1).

If we wanted to allow other input (JAGS/BUGS) we would need to add code to generate pr(y = 1) directly.

[jkarreth]

Rework this to be something like y.pred <- plogis(Xb)

[jkarreth]

i consider this completed and closed

[jkarreth]

Since the newest and super-fast implementation now requires an rjags object, I think we need to re-open this. Not super urgent, but I'd eventually like to make this a function that can take any MCMC input (from any of the packages we allow elsewhere).

I think the game plan would be to generalize the part that's currently up to line 65. The tricky part is likely to ensure that whatever grabs the pardraws is smart enough to ignore deviance and other ancillary parameters...

I'll be happy to tackle this down the road, hence reopening this issue.

[jkarreth]

(Note: since the current version is explicit about required input, this can remain an enhancement IMHO.)

[jkarreth]

43 is related to this, too - could be helpful.

[andybega]

Hello, I'll work on tackling this. TBC, I'll aim to make mcmcRocPrc() work with the four types of model objects that are currently in the package readme:

• [x] JAGS
• [ ] BUGS
• [x] MCMCpack
• [x] RStan
• [x] rstanarm
• [x] runjags
• [x] brms

Any other kinds of objects?

Also, is there a need to also have a version for the model matrix-style interface in mcmcRocPrcGen()?

[jayrobwilliams]

I think the only other one is brms. I don't see a need to keep the matrix interface if we can figure out a way to extract the samples and subset to the required ones. What are your thoughts @ShanaScogin @jkarreth?

[jkarreth]

I agree, @jayrobwilliams. We just need to be able to create predictions $(Pr (y = 1))$ from whatever the input is. From what I remember, that was a challenge - though not insurmountable.

Easy for rstanarm: rstanarm::posterior_linpred(rstanarm_object, transform = TRUE).

Thanks all!

[ShanaScogin]

I approved this merge but I don't want to close until @jkarreth can take a look. I'm going through it now to catch up - it all looks great. It's too bad about no method to accommodate R2WinBUGS, but the only time I have encountered R2WinBUGS was in coursework, so I think it might be best to leave it behind unless anyone else has thoughts.

[andybega]

Yeah. If one of you can add an example logit model from R2WinBUGS, I'm happy to add a method for it. I just didn't want to deal with trying to install OpenBUGS or WinBUGS.

[jkarreth]

I added a BUGS object representing output from a logit model: data/bugs_logit.rda

This was created by adapting data-raw/jags_logit.R to R2OpenBUGS.

(Note that the actual script to create bugs_logit.rda is not on Github, I asked someone else to run it on their machine. Hope this is OK for future CRAN uploads etc.)

@andybega @jayrobwilliams does this give you enough for the adjusting for BUGS objects as input?

(Apologies, I think I initially posted this comment in a different context elsewhere.)

[andybega]

(Note that the actual script to create bugs_logit.rda is not on Github, I asked someone else to run it on their machine. Hope this is OK for future CRAN uploads etc.)

data-raw/ is not a requirement for CRAN, so I think that's ok.

does this give you enough for the adjusting for BUGS objects as input?

After taking a quick look, I think so. As long as the data that went into it are equivalent to the jags logit example (looks like the bugs object doesn't contain a copy of the data, so it'll have to be manually passed to the function, like some of the other input methods).

Could one of you though merge the master branch where this commit is into develop though? The feature branch changes I had in the pull request are already on your develop--I don't think I can get this from master into a new feature branch for this update on my end without screwing something up.

[jkarreth]

As long as the data that went into it are equivalent to the jags logit example (looks like the bugs object doesn't contain a copy of the data, so it'll have to be manually passed to the function, like some of the other input methods).

Yes, that's correct - the call to bugs used the same data as the jags logit example