A method to check for page stability when shiny is not busy

vedhav commented 1 month ago

Introduces a private method for TealAppDriver called wait_for_page_stability which blocks the R process by sleeping until the page is unchanged till the stability period.

This is used in two places:

After a click is performed.
Before the visibility is checked.

This should potentially fix the CI errors in tmg and tmc.

Error in tmg:

  ══ Failed tests ════════════════════════════════════════════════════════════════
  ── Error ('test-shinytest2-tm_misssing_data.R:73:3'): e2e - tm_missing_data: Default settings and visibility of the summary graph ──
  Error in `app_wait_for_idle(self, private, duration = duration, timeout = timeout)`: An error occurred while waiting for Shiny to be stable
  Backtrace:
      ▆
   1. └─app_driver$click(selector = app_driver$active_module_element("iris-filter_na")) at test-shinytest2-tm_misssing_data.R:73:3
   2.   └─self$wait_for_idle()
   3.     └─shinytest2:::app_wait_for_idle(self, private, duration = duration, timeout = timeout)
   4.       └─shinytest2:::app_abort(self, private, "An error occurred while waiting for Shiny to be stable")
   5.         └─rlang::abort(..., app = self, call = call)
  ── Error ('test-shinytest2-tm_misssing_data.R:123:3'): e2e - tm_missing_data: Validate functionality and UI response for 'By Variable Levels' ──
  Error in `app_wait_for_idle(self, private, duration = duration, timeout = timeout)`: An error occurred while waiting for Shiny to be stable
  Backtrace:
      ▆
   1. └─app_driver$set_active_module_input("iris-summary_type", "By Variable Levels") at test-shinytest2-tm_misssing_data.R:123:3
   2.   └─self$wait_for_idle()
   3.     └─shinytest2:::app_wait_for_idle(self, private, duration = duration, timeout = timeout)
   4.       └─shinytest2:::app_abort(self, private, "An error occurred while waiting for Shiny to be stable")
   5.         └─rlang::abort(..., app = self, call = call)

  [ FAIL 2 | WARN 5 | SKIP 0 | PASS 464 ]

Error in tmc:

  ══ Failed tests ════════════════════════════════════════════════════════════════
  ── Failure ('test-shinytest2-tm_t_pp_basic_info.R:78:5'): e2e - tm_t_pp_basic_info: Deselection of patient_id throws validation error and table is not visible. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_basic_info.R:118:3'): e2e - tm_t_pp_basic_info: Deselection of cov_var throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:129:3'): e2e - tm_t_pp_laboratory: Deselection of patient_id throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:166:3'): e2e - tm_t_pp_laboratory: Deselection of paramcd throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:205:5'): e2e - tm_t_pp_laboratory: Deselection of param throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:243:3'): e2e - tm_t_pp_laboratory: Deselection of timepoints throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:282:3'): e2e - tm_t_pp_laboratory: Deselection of avalu throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:321:3'): e2e - tm_t_pp_laboratory: Deselection of aval_var throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE
  ── Failure ('test-shinytest2-tm_t_pp_laboratory.R:358:3'): e2e - tm_t_pp_laboratory: Deselection of arind throws validation error. ──
  app_driver$is_visible(...) is not FALSE

  `actual`:   TRUE 
  `expected`: FALSE

  [ FAIL 9 | WARN 8 | SKIP 0 | PASS 1966 ]

github-actions[bot] commented 1 month ago

Code Coverage Summary

Filename                          Stmts    Miss  Cover    Missing
------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------
R/dummy_functions.R                  36      25  30.56%   21-37, 40-47
R/get_rcode_utils.R                  31       1  96.77%   50
R/include_css_js.R                   22      17  22.73%   12-38, 76-82
R/init.R                             86      31  63.95%   108-115, 161-162, 164, 176-197, 228-229, 231
R/landing_popup_module.R             25      25  0.00%    61-87
R/module_bookmark_manager.R         158     125  20.89%   42-43, 57-59, 70-83, 93-143, 148-149, 189, 224-301
R/module_filter_manager.R            84      19  77.38%   38-42, 157, 162-175
R/module_nested_tabs.R              161      60  62.73%   39-112, 128, 180, 202, 224, 232, 236
R/module_snapshot_manager.R         241     178  26.14%   95-107, 136-139, 143-144, 159-169, 173-188, 190-198, 205-220, 224-228, 230-236, 239-252, 255-273, 282-298, 313-336, 339-350, 353-359, 373, 394-418
R/module_tabs_with_filters.R         76      33  56.58%   33-68, 100, 116
R/module_teal_with_splash.R         114      34  70.18%   60-95, 110, 131, 197-198
R/module_teal.R                     113      79  30.09%   52-119, 150-151, 157-160, 171, 184-215
R/module_wunder_bar.R                60      39  35.00%   23-41, 55-64, 68-77
R/modules.R                         159      26  83.65%   127-130, 147-151, 206-209, 291-292, 344, 356-364, 418-421
R/reporter_previewer_module.R        19       2  89.47%   30, 34
R/show_rcode_modal.R                 24      24  0.00%    17-42
R/tdata.R                            53       1  98.11%   154
R/teal_data_module-eval_code.R       27       0  100.00%
R/teal_data_module-within.R           7       0  100.00%
R/teal_data_module.R                  6       0  100.00%
R/teal_reporter.R                    68       6  91.18%   69, 77, 125-126, 129, 146
R/teal_slices-store.R                29       0  100.00%
R/teal_slices.R                      59      12  79.66%   137-150
R/TealAppDriver.R                   324     324  0.00%    43-671
R/utils.R                           173       1  99.42%   255
R/validate_inputs.R                  32       0  100.00%
R/validations.R                      58      37  36.21%   110-377
R/zzz.R                              12       8  33.33%   3-15
TOTAL                              2257    1107  50.95%

Diff against main

Filename             Stmts    Miss  Cover
-----------------  -------  ------  --------
R/TealAppDriver.R      +13     +13  +100.00%
TOTAL                  +13     +13  -0.30%

Results for commit: 6fc8dc24a0c5e68b3102451fd182e0cf25f6c908

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

github-actions[bot] commented 1 month ago

Unit Tests Summary

1 files 30 suites 3m 17s :stopwatch: 240 tests 240 :white_check_mark: 0 :zzz: 0 :x: 505 runs 505 :white_check_mark: 0 :zzz: 0 :x:

Results for commit 6fc8dc24.

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 1 month ago

Unit Test Performance Difference

Test Suite	$Status$	Time on `main`	$±Time$	$±Tests$	$±Skipped$	$±Failures$	$±Errors$
shinytest2-filter_panel	💔	$16.92$	$+9.48$	$0$	$0$	$0$	$0$
shinytest2-module_bookmark_manager	💔	$22.67$	$+1.88$	$0$	$0$	$0$	$0$
shinytest2-modules	💔	$16.93$	$+1.60$	$0$	$0$	$0$	$0$
shinytest2-reporter	💔	$32.49$	$+1.45$	$0$	$0$	$0$	$0$
shinytest2-teal_data_module	💔	$10.62$	$+1.68$	$0$	$0$	$0$	$0$
shinytest2-teal_slices	💔	$12.73$	$+11.62$	$0$	$0$	$0$	$0$

Additional test case details

| Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | shinytest2-filter_panel | 💔 | $5.84$ | $+4.16$ | e2e_filtering_a_module_specific_filter_is_not_refected_in_other_unshared_modules | | shinytest2-filter_panel | 💔 | $5.90$ | $+4.86$ | e2e_filtering_a_module_specific_filter_is_refected_in_other_shared_module | | shinytest2-reporter | 💔 | $25.87$ | $+1.29$ | e2e_adding_a_report_card_in_a_module_adds_it_in_the_report_previewer_tab | | shinytest2-teal_slices | 💔 | $5.66$ | $+3.38$ | e2e_teal_slices_filters_are_initialized_when_global_filters_are_created | | shinytest2-teal_slices | 💔 | $7.08$ | $+8.24$ | e2e_teal_slices_filters_are_initialized_when_module_specific_filters_are_created |

Results for commit f605ab1afbd04af87d7d648cc8412b88f4707c82

♻️ This comment has been updated with latest results.

insightsengineering / teal