tidyverts / fabletools

General fable features useful for extension packages
http://fabletools.tidyverts.org/
89 stars 31 forks source link

Suggestion: make signatures of accuracy measures consistent #153

Closed hongooi73 closed 3 years ago

hongooi73 commented 4 years ago

Currently, the functions MAE, MAPE, MSE et al have different arguments. Some take as input only the residuals, while others take both the residuals and the actuals. This makes it unnecessarily awkward to write generic code that allows switching between measures.

Consider having all these functions use the same arguments instead: predicted and actuals, rather than residuals (which can then be computed). So, for example, MAE would become

MAE <- function(.pred, .actual, na.rm=TRUE, ...)
{
    mean(abs(.pred - .actual), na.rm=na.rm)
}

and MAPE would become

MAPE <- function(.pred, .actual, na.rm=TRUE, ...)
{
    mean(abs((.pred - .actual)/.actual), na.rm=na.rm) * 100
}

Then it would be possible to write something like this, as a generic error function:

error <- function(..., measure="MAPE")
{
    func <- get(measure, getNamespace("fabletools"), inherits=FALSE)
    func(...)
}

and call it the same way, regardless of what measure was being used.

mitchelloharawild commented 4 years ago

I appreciate the value of having consistent formals for accuracy measures, however using both .pred and .actual requires more information than necessary for that measure (only .resid is required). Consider more complicated accuracy measures like MASE, which also require training data (.train) and the seasonal period (.period). It would not be appropriate for MAE to have these inputs as formals as the measure doesn't need them. The accuracy measures are constructed with consistent names of formals, and an unused .... If you wanted to use a generic error function as specified above, you could provide named arguments that may or may not be used by the specified measure.