add handling of when reference column is not the best

[vpchung]

(Adding change for @allaway )

This change will handle the case for when the reference column is not the best set of predictions, i.e. baseline model. When this happens, the inversion of K (BF) will change, depending on which submission it is being compared to.

I did test this change with a sample bootstapped matrix to ensure the behavior will not change when the reference column is the best:

computeBayesFactorWhereRefIsNotBest <- function(bootstrapMetricMatrix,
                               refPredIndex,
                               invertBayes){

    M <- as.data.frame(bootstrapMetricMatrix - bootstrapMetricMatrix[,refPredIndex])
    K <- apply(M ,2, function(x) {
      k <- sum(x >= 0)/sum(x < 0)
      if(sum(x >= 0) > sum(x < 0)){
      return(k)
      }else{
      return(1/k)
      }
    })
    if(invertBayes == T){K <- 1/K}
    K[refPredIndex] <- 0

    return(K)
}

> computeBayesFactorWhereRefIsNotBest(boot, refPredIndex=1, invertBayes=F) %>%
+     as_tibble(rownames = "submission") %>%
+     rename(bayes = value)
# A tibble: 3 x 2
   submission                     bayes
   <chr>                          <dbl>
 1 team A                         0   
 2 team B                         8.09
 3 team C                         Inf   

> computeBayesFactor(boot, refPredIndex=1, invertBayes=F) %>%
+     as_tibble(rownames = "submission") %>%
+     rename(bayes = value) 
# A tibble: 3 x 2
   submission                     bayes
   <chr>                          <dbl>
 1 team A                         0   
 2 team B                         8.09
 3 team C                         Inf   

[thomasyu888]

I fixed the tests, if you pull in the most recent changes on develop, the CI will pass.

[vpchung]

Thanks for the fix, @thomasyu888 ! Look all good now.