report.compare.loo: minor edits

DominiqueMakowski commented 6 months ago

After reading a bit more and hearing inputs in https://github.com/easystats/performance/issues/716, I made some minor edits:

report IC by default as Brenton mentioned (ENP is optional as it seems like not entirely independent from the rest of the indices, so better to avoid verbosity by default I'd say - plus its interpretation remains somewhat complicated for more complex models)
emphasizing in the docs that the z-score/p-value is not really meaningful if the raw ELPD diff is small

> report(x)
The difference in predictive accuracy, as indexed by Expected Log Predictive Density (ELPD-LOO), suggests that 'm3' is the best model (LOOIC = 157.97), followed by 'm2'
(diff-ELPD = -13.41 +- 4.42, p = 0.002, LOOIC = 184.80) and 'm1' (diff-ELPD = -23.28 +- 4.41, p < .001, LOOIC = 204.53)

> report(x, include_IC = FALSE)
The difference in predictive accuracy, as indexed by Expected Log Predictive Density (ELPD-LOO), suggests that 'm3' is the best model (ELPD = -78.99), followed by 'm2'
(diff-ELPD = -13.41 +- 4.42, p = 0.002) and 'm1' (diff-ELPD = -23.28 +- 4.41, p < .001)

> report(x, include_ENP = TRUE)
The difference in predictive accuracy, as indexed by Expected Log Predictive Density (ELPD-LOO), suggests that 'm3' is the best model (LOOIC = 157.97, ENP = 4.92), followed
by 'm2' (diff-ELPD = -13.41 +- 4.42, p = 0.002, LOOIC = 184.80, ENP = 3.93) and 'm1' (diff-ELPD = -23.28 +- 4.41, p < .001, LOOIC = 204.53, ENP = 2.57)

rempsyc commented 6 months ago

Thanks, but we see:

Undefined global functions or variables:
    pnorm
  Consider adding
    importFrom("stats", "pnorm")

___________________________________________

ENP   report.compare.loo.Rd:14
Error:
! Spelling errors found. Either correct them or update `inst/WORDLIST`
  using `spelling::update_wordlist()`.

strengejacke commented 6 months ago

Thanks, but we see:

Undefined global functions or variables:
    pnorm
  Consider adding
    importFrom("stats", "pnorm")

___________________________________________

ENP   report.compare.loo.Rd:14
Error:
! Spelling errors found. Either correct them or update `inst/WORDLIST`
  using `spelling::update_wordlist()`.

And some "Place a space before left parenthesis, except in a function call." ;-)

rempsyc commented 6 months ago

(Dom I'm just secretly waiting for the meme signal to get to work 🤫)

easystats / report

report.compare.loo: minor edits #427