Cannot handle one level in the predicted variable against several levels in the observed variable

[cloudcell]

Describe the bug Having just one level in the predicted variable against several levels in the observed variable causes the following error message:

Error in utils::combn(levels, 2, function(X, response, predictor, percent, : n < m

To Reproduce Just run the following command:

multiclass.roc(c(1,1,1), c(1,2,1))

Expected behavior The function must treat this as valid predicted set of values

Additional context n/a

[cloudcell]

as a temporary fix, I've written this code:

# avoiding the error of pROC
safe_auc <- function(predicted,observed){
    if(length(levels(as.factor(predicted)))==1) {
        warning(paste0("Safe evaluation triggered! \n"))
        count=0
        total=length(predicted)
        for(i in 1:length(predicted)){
            if(predicted[i]==observed[i]){
                count <- count+1
            }
        }
        auc=count/total
    } else {
        roc <- multiclass.roc(predicted,observed)
        auc=as.numeric(roc$auc)
    }
    return(auc)
}

HTH

[xrobin]

You have the order wrong, the call is:

multiclass.roc(response, predictor, [...]

or in other terms

multiclass.roc(true, predicted)

So you probably want to write:

multiclass.roc(c(1,2,1), c(1,1,1))

I agree the error message could be improved. I realize maybe I can clarify the doc too.

[cloudcell]

Thank you for such a prompt response. I think the code you proposed at the end of your previous message would resolve the issue.

At the same time, if you could clarify the error message and the doc, that would benefit everyone. Thank you in advance for that.

[xrobin]

Done. Thanks for the report!