We should check for consistency when scoring binary forecasts

[nikosbosse]

Transforming a vector of 0s and 1s into a factor can be tricky (at least if you're me).

Consider the following example:

set.seed(123)
test <- rbinom(n = 5, size = 1, prob = 0.7)
obs1 <- factor(test, levels = c(1, 0))
obs2 <- factor(test, levels = c(0, 1))

mean(brier_score(obs1, 0.7))
mean(brier_score(obs2, 0.7))

those two things give different results.

I suggest the following:

• we add some functionality which, at least in the simple case with 0s and 1s, checks whether the levels correspond to the numbers in the input vector. If that's not the case, we throw a warning.
• we create a helper function that allows users to convert their input into a factor and specify exactly which outcome their predicted value corresponds to.

[seabbs]

yes sounds like a good idea but is this really 2.0 or for a following version (i.e it seems nice to have vs must have)