koalaverse / vip

Variable Importance Plots (VIPs)
https://koalaverse.github.io/vip/
186 stars 24 forks source link

Vignette code for pdp and ice method VI plots doesn't work #135

Closed jtr13 closed 1 year ago

jtr13 commented 1 year ago

Thanks for a very useful package!

FYI, these examples from the vignette don't work:

library(vip)
#> 
#> Attaching package: 'vip'
#> The following object is masked from 'package:utils':
#> 
#>     vi
set.seed(101)  # for reproducibility
trn <- as.data.frame(mlbench::mlbench.friedman1(500))  # ?mlbench.friedman1
pp <- ppr(y ~ ., data = trn, nterms = 11)  
vip(pp, method = "pdp") + ggtitle("PPR")
#> Error in match.arg(method): 'arg' should be one of "model", "firm", "permute", "shap"
vip(pp, method = "ice") + ggtitle("PPR")
#> Error in match.arg(method): 'arg' should be one of "model", "firm", "permute", "shap"

Created on 2023-04-14 with reprex v2.0.2

I can piece together from the help files what the code should be but it would be helpful to have the have the new syntax documented more clearly.

bgreenwell commented 1 year ago

Thanks for pointing this out @jtr13. I plan to dive back into this package in the next few weeks and the docs should be getting some much needed attention.

jtr13 commented 1 year ago

Great to hear. If I can help let me know. We've been using pdp, vip and fastshap in the Interpretable Machine Learning class I'm teaching so triple thank you. I plan to write up some notes for my students on sticky points such as writing prediction functions so I'll share that in case it's useful for the docs.

bgreenwell commented 1 year ago

I'm glad to hear you find them useful! You can probably tell all three were written at different stages of my R career. @bradleyboehmke and I are starting to work on a second edition of our book, so all three of those packages should be getting some much needed attention soon. Please let me know on any of the repos if you have suggestions on making them better! And yes, any thing you share will be more than useful.

bgreenwell commented 1 year ago

@jtr13 are the IML course materials by chance available online? Also, I recently gave a talk for an R-ladies group on pdp, vip, and fastshap (i.e., current state of packages, methods, drawbacks, etc.). Would be happy to give a virtual talk to any of your IML classes in the future.

jtr13 commented 1 year ago

Not yet... the plan is to organize them when the semester ends and then post publicly. That's a great offer... will take you up on that in the fall if you're available.

bgreenwell commented 1 year ago

Hi @jtr13, starting to fix up this package now that fastshap is in good shape (and on to CRAN hopefully soon). In any case, I cannot find where in the docs or vignette those examples are? All the example in GH (and seemingly on CRAN) seem to be up to date and working.

jtr13 commented 1 year ago

Good question.. I found it here: https://koalaverse.github.io/vip/articles/vip.html#pdp-method

bgreenwell commented 1 year ago

Ahh, thank you @jtr13. Long story short, when we updated the package, I had switched the URL in the repo to point to the R Journal article that we had just published which uses the correct code. But the old site still exists. In the next update the old pkg down site will be updated as well. Sorry for the confusion. Here’s the definitive guide for the current version: https://journal.r-project.org/archive/2020/RJ-2020-013/index.html

bgreenwell commented 1 year ago

I’ve unpublished the site for now, but will reactivate it when we make this next release; probably end of May given the number of open issues.