95 CI for rmse

[jorgemmteixeira]

Hi. Is it possible to get the 95 CI for rmse? Thank you

library(lme4)
library(performance)

m1 <- lmer(Distance_m  ~ Date + (1 | Name), data = dat)
summary(m1)

#icc
icc(m1, ci=0.95, ci_method = "boot")

# TEM/rmse
performance_rmse(m1, normalized = FALSE)

[strengejacke]

@bwiernik @mattansb do you know if there's an analytical solution? Else, I would just do bootstrapping.

[bwiernik]

Yeah take the CI for sigma for a gaussian model and multiple the bounds by (n - 1)/n

I think we have CI for sigma somewhere? I remember that being one of the earlier things I added

[strengejacke]

I think you added the code for r2(): https://github.com/easystats/performance/blob/main/R/r2_ci.R

[strengejacke]

Oh, and here for sigma: https://github.com/easystats/insight/blob/90906beca6c0f4508b548e61dc93345bfd4fe543/R/get_sigma.R#L295-L308

[jorgemmteixeira]

(n - 1)/n ... Is n the number of obs or IDs? Thank you

CI_high Does not cover sigma.

library(lme4)
library(performance)

dat <- read.csv("https://stats.idre.ucla.edu/stat/data/demo3.csv")

m1  <-  lmer(pulse ~  time + (1 | id), data = dat)

# TEM/rmse
performance_rmse(m1, normalized = FALSE)

# Calculate sigma with confidence interval
sigma_ci <- get_sigma(m1, ci=0.95)
print(sigma_ci)

# low_rsme
1.995201 * ((n - 1)/n) 

Like this?

[strengejacke]

library(lme4)
#> Loading required package: Matrix

library(performance)
library(datawizard)

dat <- data_read("https://stats.idre.ucla.edu/stat/data/demo3.csv")

m1  <-  lmer(pulse ~  time + (1 | id), data = dat)
rmse(m1, ci = 0.95)
#> RMSE |       95% CI
#> -------------------
#> 3.29 | [2.03, 4.38]

^{Created on 2024-07-05 with reprex v2.1.0}

[jorgemmteixeira]

I get an error:

rmse(m1, ci = 0.95) Error in rmse(m1, ci = 0.95) : unused argument (ci = 0.95)

[strengejacke]

You must install the latest GitHub version from performance, the new feature is currently only in the development version. See the README me for installation instructions: https://easystats.github.io/performance/

Run install.packages("performance", repos = "https://easystats.r-universe.dev").

[jorgemmteixeira]

Perfect. Thank you for all the great support.