pbreheny / visreg

Visualization of regression functions
http://pbreheny.github.io/visreg/
61 stars 18 forks source link

Lack of confidence band in multinomial models #111

Open MarcRieraDominguez opened 1 year ago

MarcRieraDominguez commented 1 year ago

Hi! I am using visreg to visualize a multinomial model, which appears to work just fine, except for a lack of confidence bands around factor levels, and numerical variables (both linear and non-linear with splines). The visreg website covers multinomial models explicitly, and does not show confidence bands either. Is this lack of confidence bands "by design"? Are there any plans to include confidence bands for multinomial models any time soon? (confidence bands are indeed provided by the effects package, for example). This is a reproducible example, building on the example found in Faraway (2016) Extending the linear model with R. I've fitted splines as I want to model non-linear trends (mgcv::gam() has a multinomial family, but I've been unable to make it work [Edit 30-VI-2023: error arose due to a conflict between the libraries nnet and mgcv]). Note that the lack of confidence bands appears to be related to the multinomial model rather than the splines themselves. Cheers! Congratulations on this rocking package :)

# Load libraries
library(faraway) # 1.0.8
#> Warning: package 'faraway' was built under R version 4.2.2
library(nnet) # 7.3.17
library(splines) # 4.2.0
library(visreg) # 2.7.0

# Assemble dataset
data(nes96, package="faraway")
party <- nes96$PID
levels(party) <- c("Democrat","Democrat","Independent","Independent",
                   "Independent","Republican","Republican")
rnes96 <- data.frame(party, tv = nes96$TVnews, education=nes96$educ, age=nes96$age)

# Inspect dataset
summary(rnes96)
#>          party           tv         education        age       
#>  Democrat   :380   Min.   :0.000   MS    : 13   Min.   :19.00  
#>  Independent:239   1st Qu.:1.000   HSdrop: 52   1st Qu.:34.00  
#>  Republican :325   Median :3.000   HS    :248   Median :44.00  
#>                    Mean   :3.728   Coll  :187   Mean   :47.04  
#>                    3rd Qu.:7.000   CCdeg : 90   3rd Qu.:58.00  
#>                    Max.   :7.000   BAdeg :227   Max.   :91.00  
#>                                    MAdeg :127
str(rnes96)
#> 'data.frame':    944 obs. of  4 variables:
#>  $ party    : Ord.factor w/ 3 levels "Democrat"<"Independent"<..: 3 1 1 1 1 1 1 2 2 1 ...
#>  $ tv       : int  7 1 7 4 7 3 7 1 7 0 ...
#>  $ education: Ord.factor w/ 7 levels "MS"<"HSdrop"<..: 3 4 6 6 6 4 4 4 4 3 ...
#>  $ age      : int  36 20 24 28 68 21 77 21 31 39 ...

# Fit multinomial model
mmod <- multinom(party ~ bs(age, df = 5) + education + tv, rnes96)
#> # weights:  42 (26 variable)
#> initial  value 1037.090001 
#> iter  10 value 1007.238444
#> iter  20 value 1004.174679
#> final  value 1004.002639 
#> converged

# Visualize model: no confidence bands for factors or continuous variables (regardless of splines)
visreg(mmod, "education", band = TRUE, collapse = TRUE, overlay = TRUE)

visreg(mmod, "age", band = TRUE, collapse = TRUE, overlay = TRUE)

visreg(mmod, "tv", band = TRUE, collapse = TRUE, overlay = TRUE)


# Visreg creates a confidence band for lm(), including splines
visreg(lm(tv ~ bs(age, df = 5), rnes96), "age", band = TRUE)

visreg(lm(tv ~ age, rnes96), "age", band = TRUE)


# Session info
sessionInfo()
#> R version 4.2.0 (2022-04-22 ucrt)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 10 x64 (build 22000)
#> 
#> Matrix products: default
#> 
#> locale:
#> [1] LC_COLLATE=Spanish_Spain.utf8  LC_CTYPE=Spanish_Spain.utf8   
#> [3] LC_MONETARY=Spanish_Spain.utf8 LC_NUMERIC=C                  
#> [5] LC_TIME=Spanish_Spain.utf8    
#> 
#> attached base packages:
#> [1] splines   stats     graphics  grDevices utils     datasets  methods  
#> [8] base     
#> 
#> other attached packages:
#> [1] visreg_2.7.0  nnet_7.3-17   faraway_1.0.8
#> 
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_1.0.10     rstudioapi_0.13 knitr_1.39      magrittr_2.0.3 
#>  [5] MASS_7.3-57     lattice_0.20-45 rlang_1.0.6     fastmap_1.1.0  
#>  [9] minqa_1.2.4     stringr_1.4.0   highr_0.9       tools_4.2.0    
#> [13] grid_4.2.0      nlme_3.1-157    xfun_0.31       cli_3.3.0      
#> [17] withr_2.5.0     htmltools_0.5.2 yaml_2.3.5      lme4_1.1-29    
#> [21] digest_0.6.29   lifecycle_1.0.3 Matrix_1.4-1    nloptr_2.0.1   
#> [25] fs_1.5.2        glue_1.6.2      evaluate_0.15   rmarkdown_2.14 
#> [29] reprex_2.0.2    stringi_1.7.6   compiler_4.2.0  boot_1.3-28

Created on 2023-06-28 with reprex v2.0.2