search

insightsengineering / teal.goshawk

Modules that produce web interfaces through which longitudinal visualizations can be dynamically
https://insightsengineering.github.io/teal.goshawk/
Other
3 stars 2 forks source link

UI elements do not refresh when filter removed #288

Open npaszty opened 3 months ago

npaszty commented 3 months ago

What happened?

"Select Biomarker" menu does not refresh do display all biomarkers available in data after filter panel filter on PARAMCD is removed.

Manifested in study data we are using to test upversioning and can replicate in teal.gallery app as well but there is different behavior as far as the modules producing a plot. none of the teal gallery app plots are produced if ALT is filtered out as a first step. however see below for what happens in the study data app we are using for testing.

Steps

  1. start app
  2. go to view data and filter PARAMCD to ALT and F2974907 for example
  3. got to boxplot plot where default PARAM is ALBUM. box plot throws validation error. this is actually the sample app behavior. the study app produces a boxplot of ALT
  4. got to correlation plot. produces a plot with ALT as x-axis biomarker and ALT as y-axis biomarker
  5. got to density dist plot. produces plot with ALT.
  6. got to line plot. produces plot with ALT.
  7. got to spaghetti plot. prodices plot with ALT.
  8. remove the PARAM filter. biomarker UI item does not reset to all biomarkers. just includes the two that were originally filtered

sessionInfo()

R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] tidyr_1.3.1                     teal.goshawk_0.1.15.9027        teal.modules.general_0.3.0.9038 ggmosaic_0.3.3                 
 [5] ggplot2_3.5.0                   teal.modules.clinical_0.9.0     tern_0.9.5.9007                 rtables_0.6.9.9002             
 [9] formatters_0.5.8.9000           teal.transform_0.5.0            teal.connectors.rice_0.2.0      teal.connectors.harbour_0.1.1  
[13] teal_0.15.2                     teal.slice_0.5.0                teal.data_0.5.0                 teal.code_0.5.0                
[17] stringr_1.5.1                   sparkline_2.0                   ricepass_1.1.4                  rice_3.0.1                     
[21] magrittr_2.0.3                  goshawk_0.1.17.9003             dplyr_1.1.4                     googlesheets4_1.0.1            
[25] DescTools_0.99.54               shiny_1.8.1.1                  

loaded via a namespace (and not attached):
  [1] rstudioapi_0.16.0      jsonlite_1.8.8         estimability_1.5       farver_2.1.1           rmarkdown_2.26         ragg_1.2.7            
  [7] geepack_1.3.10         fs_1.6.3               vctrs_0.6.5            memoise_2.0.1          askpass_1.2.0          htmltools_0.5.8.1     
 [13] forcats_1.0.0          curl_5.2.1             haven_2.5.4            broom_1.0.5            cellranger_1.1.0       sass_0.4.9            
 [19] bslib_0.7.0            htmlwidgets_1.6.4      fontawesome_0.5.2      emmeans_1.10.0         rootSolve_1.8.2.4      plotly_4.10.4         
 [25] cachem_1.0.8           mime_0.12              lifecycle_1.0.4        teal.widgets_0.4.2     pkgconfig_2.0.3        Matrix_1.6-1.1        
 [31] R6_2.5.1               fastmap_1.1.1          rbibutils_2.2.16       digest_0.6.35          Exact_3.2              colorspace_2.1-0      
 [37] shinycssloaders_1.0.0  textshaping_0.3.7      crosstalk_1.2.1        labeling_0.4.3         fansi_1.0.6            httr_1.4.7            
 [43] compiler_4.3.1         gargle_1.2.1           proxy_0.4-27           bit64_4.0.5            withr_3.0.0            backports_1.4.1       
 [49] logger_0.3.0           MASS_7.3-60.0.1        openssl_2.1.1          gld_2.6.6              tools_4.3.1            robslopes_1.1.3       
 [55] googledrive_2.0.0      httpuv_1.6.15          shinyvalidate_0.1.3    glue_1.7.0             nlme_3.1-163           promises_1.2.1        
 [61] grid_4.3.1             rsconnect_1.2.2        checkmate_2.3.1        generics_0.1.3         gtable_0.3.4           class_7.3-22          
 [67] data.table_1.15.4      lmom_3.0               hms_1.1.3              utf8_1.2.4             ggrepel_0.9.5          pillar_1.9.0          
 [73] later_1.3.2            splines_4.3.1          tern.gee_0.1.3         lattice_0.21-9         renv_1.0.3             survival_3.5-7        
 [79] bit_4.0.5              tidyselect_1.2.1       knitr_1.45             teal.logger_0.2.0.9005 xfun_0.43              expm_0.999-9          
 [85] DT_0.33                stringi_1.8.3          yaml_2.3.8             lazyeval_0.2.2         boot_1.3-29            shinyWidgets_0.8.4    
 [91] evaluate_0.23          tibble_3.2.1           cli_3.6.2              systemfonts_1.0.6      arrow_16.1.0           xtable_1.8-4          
 [97] Rdpack_2.6             munsell_0.5.1          jquerylib_0.1.4        Rcpp_1.0.12            teal.reporter_0.3.1    readxl_1.4.3          
[103] parallel_4.3.1         assertthat_0.2.1       mcr_1.3.3              viridisLite_0.4.2      mvtnorm_1.2-4          scales_1.3.0          
[109] e1071_1.7-14           crayon_1.5.2           purrr_1.0.2            rlang_1.1.3            cowplot_1.1.3          shinyjs_2.1.0

Relevant log output

INFO [2024-07-24 21:44:22] Shiny input change detected on submit: 0 -> 1
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Removed 6 rows containing missing values or values outside the scale range (`geom_line()`).
Warning: Unknown or uninitialised column: `AVALU`.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Removed 6 rows containing missing values or values outside the scale range (`geom_line()`).
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Removed 3 rows containing missing values or values outside the scale range (`geom_line()`).
`geom_line()`: Each group consists of only one observation.
ℹ Do you need to adjust the group aesthetic?
`geom_line()`: Each group consists of only one observation.
ℹ Do you need to adjust the group aesthetic?
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Groups with fewer than two data points have been dropped.
Warning: Removed 6 rows containing missing values or values outside the scale range (`geom_line()`).

Code of Conduct

Contribution Guidelines

Security Policy

m7pr commented 3 months ago

I think this is a bigger issue, not necesserilly just in modules from teal.goshawk but with teal in general. I think encoding panel gets created the moment when module is opened, and uses what was currently displayed in the filter panel, for the time being of the module till the end of the app.

Encodings do not get refreshed when filters are edited in the filter panel. I will open an issue in teal or teal.slice when I will create a reproducible example for developers to help.

m7pr commented 3 months ago

This happens for all plots for their all UI elements. I will change the name of the issue. For boxplot I think this part is responsible - it does not get updated when filter panel adds or remove filters.

https://github.com/insightsengineering/teal.goshawk/blob/main/R/tm_g_gh_boxplot.R#L330-L345 https://github.com/insightsengineering/teal.goshawk/blob/28735c89aa4eebaf3b4d38f2e40847ea3b2edb1c/R/tm_g_gh_boxplot.R#L330-L345

Other plots have similar code, so once we figure this out for a boxplot, we can figure this out for the rest of the plots in here.

m7pr commented 3 months ago

I was able to create a minimal reproducible example with an open data. I believe the main reason of this behavior is the value_choices function in param = choices_selected(choices = value_choices("ADLB", "PARAMCD", "PARAM")).

To reproduce the issue:

  1. Start the app.
  2. In View Data tab, create ADLB filter for PARAMCD variable. Pick 2 levels.
  3. Go to Box Plot tab, check choices under Select an X-Axis Biomarker - there are only 2.
  4. Remove filter you created for ADLB+PARAMCD - there are still 2 choices in Select an X-Axis Biomarker.
  5. Close the app.
  6. Start the app.
  7. Go to Box Plot tab without any filters created, check choices in Select an X-Axis Biomarker - there are 3.
  8. If you create a filter for ADLB+PARAMCD now, there will always be 3 choices, no matter how many levels were left in data.
Simple App ```r library(teal) library(teal.modules.general) library(teal.goshawk) data <- teal_data() data <- within(data, { library(dplyr) library(nestcolor) library(stringr) # use non-exported function from goshawk h_identify_loq_values <- getFromNamespace("h_identify_loq_values", "goshawk") # original ARM value = dose value arm_mapping <- list( "A: Drug X" = "150mg QD", "B: Placebo" = "Placebo", "C: Combination" = "Combination" ) set.seed(1) ADSL <- rADSL ADLB <- rADLB var_labels <- lapply(ADLB, function(x) attributes(x)$label) ADLB <- ADLB %>% mutate( AVISITCD = case_when( AVISIT == "SCREENING" ~ "SCR", AVISIT == "BASELINE" ~ "BL", grepl("WEEK", AVISIT) ~ paste("W", str_extract(AVISIT, "(?<=(WEEK ))[0-9]+")), TRUE ~ as.character(NA) ), AVISITCDN = case_when( AVISITCD == "SCR" ~ -2, AVISITCD == "BL" ~ 0, grepl("W", AVISITCD) ~ as.numeric(gsub("[^0-9]*", "", AVISITCD)), TRUE ~ as.numeric(NA) ), AVISITCD = factor(AVISITCD) %>% reorder(AVISITCDN), TRTORD = case_when( ARMCD == "ARM C" ~ 1, ARMCD == "ARM B" ~ 2, ARMCD == "ARM A" ~ 3 ), ARM = as.character(arm_mapping[match(ARM, names(arm_mapping))]), ARM = factor(ARM) %>% reorder(TRTORD), ACTARM = as.character(arm_mapping[match(ACTARM, names(arm_mapping))]), ACTARM = factor(ACTARM) %>% reorder(TRTORD), ANRLO = 50, ANRHI = 75 ) %>% rowwise() %>% group_by(PARAMCD) %>% mutate(LBSTRESC = ifelse( USUBJID %in% sample(USUBJID, 1, replace = TRUE), paste("<", round(runif(1, min = 25, max = 30))), LBSTRESC )) %>% mutate(LBSTRESC = ifelse( USUBJID %in% sample(USUBJID, 1, replace = TRUE), paste(">", round(runif(1, min = 70, max = 75))), LBSTRESC )) %>% ungroup() attr(ADLB[["ARM"]], "label") <- var_labels[["ARM"]] attr(ADLB[["ACTARM"]], "label") <- var_labels[["ACTARM"]] attr(ADLB[["ANRLO"]], "label") <- "Analysis Normal Range Lower Limit" attr(ADLB[["ANRHI"]], "label") <- "Analysis Normal Range Upper Limit" # add LLOQ and ULOQ variables ALB_LOQS <- h_identify_loq_values(ADLB, "LOQFL") ADLB <- left_join(ADLB, ALB_LOQS, by = "PARAM") }) datanames <- c("ADSL", "ADLB") datanames(data) <- datanames join_keys(data) <- default_cdisc_join_keys[datanames] app <- init( data = data, modules = modules( tm_data_table(label = "View Data"), tm_g_gh_boxplot( label = "Box Plot", dataname = "ADLB", param_var = "PARAMCD", param = choices_selected(choices = value_choices("ADLB", "PARAMCD", "PARAM")), yaxis_var = choices_selected(c("AVAL", "BASE", "CHG"), "AVAL"), xaxis_var = choices_selected(c("ACTARM", "ARM", "AVISITCD", "STUDYID"), "ARM"), facet_var = choices_selected(c("ACTARM", "ARM", "AVISITCD", "SEX"), "AVISITCD"), trt_group = choices_selected(c("ARM", "ACTARM"), "ARM"), loq_legend = TRUE, rotate_xlab = FALSE, hline_arb = c(60, 55), hline_arb_color = c("grey", "red"), hline_arb_label = c("default_hori_A", "default_hori_B"), hline_vars = c("ANRHI", "ANRLO", "ULOQN", "LLOQN"), hline_vars_colors = c("pink", "brown", "purple", "black"), ) ) ) if (interactive()) { shinyApp(app$ui, app$server) } ```
m7pr commented 3 months ago

@npaszty @gogonzo we are currently running a huge overhaul in teal and filter panael and the way we manage the data. I would recommend revisiting this issue once we are done with this refactor in teal https://github.com/insightsengineering/teal/pull/1253

npaszty commented 3 months ago

@m7pr @gogonzo

thanks for looking into this. sounds like a big deal. this issue can be replicated in the teal.gallery app so that's a working example.

what I noticed in the teal.gallery app is that all the modules fail to produce a plot because the "Select a Biomarker" UI item doesn't default to the first PARAMCD that is filtered to which is CRP. so a validation error is thrown

however in the study app the module behavior is different. for example, the first PARAMCD in the list is ALBUM so alphabetically before ALT. after filtering in the manner described "What happened" section above and navigating to boxplot, it produces a plot for ALT which is the first PARAMCD selected in the filtered PARAMCD. I updated the "What happened" section of this issue.

npaszty commented 3 months ago

@m7pr

removed priority since you think it's a bigger issue with teal itself and may be handled in a current refactoring focus area.

donyunardi commented 2 months ago

Acceptance Criteria

Possible solution from discussion

gogonzo commented 2 months ago

Note to consider when working on the issue: I think some encoding elements were build back then on unfiltered data (_raw today).

donyunardi commented 3 weeks ago

Note from discussion: