xrobin / pROC

Display and analyze ROC curves in R and S+
https://cran.r-project.org/web/packages/pROC/
GNU General Public License v3.0
118 stars 31 forks source link

ci.auc with method bootstrap that works in RStudio Cloud #51

Closed elb0 closed 5 years ago

elb0 commented 5 years ago

Hi,

I am trying to use pROC in RStudio Cloud. The data I'm dealing with can only be accessed in a secure "datalab" environment designed by Statistics New Zealand, so using R on my personal computer is not possible and I doubt that Stats NZ will be able to support a different implementation just for me.

Using the DeLong method to get a confidence interval works no problem, but when I try something like:

ci.auc(roc_object, method = "bootstrap")

I get the error:

Error in structure(.External(.C_dotTclObjv, objv), class = "tckObj") : [tcl] invalid comman name "toplevel".

This appears to me to be similar to this issue in the old RStudio community. It appears that the tcltkpackage is the reason it doesn't work? Is that the case for the pROC package too?

Thanks for the great package, would love any feedback on whether I'm mistaken or whether a work around is possible!

xrobin commented 5 years ago

Hi, thanks for your report. It looks like the tcltk package is installed but broken on the server. Can you please try to run the following commands to confirm it?

find.package("tcltk", quiet=TRUE)
library(tcltk)
pb <- tkProgressBar("test progress bar", "Some information",
                    0, 2, 1)
setTkProgressBar(pb, 2)
close(pb)

These are commands of the tcltk package itself, used by pROC if the package is present.

If tcltk is indeed present and broken, you should ask your administrator to either fix the package if possible, or more likely in your case to uninstall it completely. For a quicker and dirty fix you can manually set the progress bar type to text or disable it completely:

options(pROCProgress = list(name = "none"))

See ?pROC for more options.

elb0 commented 5 years ago

Thanks so much for this response. The quick and dirty fix has worked well! I also tried what you recommended above in my personal RStudio Cloud account and ran into the same errors. In a response to the question I linked to above, someone responded:

Hello,

Due to the nature of the tcltk package and the way RStudio is build to work over the web, the two don't work well together. Unfortunately this isn't really something that we'll be able to workaround or support down the road.

Josh

That was 5 years ago, and so I guess it may just be that tcltk and RStudio Cloud can't play nicely together. If this is the case, is there a way to give an informative error message?

My personal RStudio Cloud attempt:

> find.package("tcltk", quiet=TRUE)
[1] "/opt/R/3.5.2/lib/R/library/tcltk"
> library(tcltk)
Warning message:
In fun(libname, pkgname) : couldn't connect to display ":0"
> pb <- tkProgressBar("test progress bar", "Some information",
+                     0, 2, 1)
Error in structure(.External(.C_dotTclObjv, objv), class = "tclObj") : 
  [tcl] invalid command name "toplevel".
> setTkProgressBar(pb, 2)
Error in setTkProgressBar(pb, 2) : object 'pb' not found
xrobin commented 5 years ago

It's going to be hard to catch this error message specifically without risking to make other errors less useful. After some thought I disabled the tcltk progress bars by default. They have been causing trouble with MacOS before, and a text progress bar seems more appropriate for a command line R terminal anyway. I'm leaving the graphical progress bar for windows for now as I am not aware of specific issues with it, but I might change my mind and have consistent progress bar everywhere.