Open yogat3ch opened 2 years ago
schloerke
commented
2 years ago Hi @yogat3ch !
There are some rough experiences given how {golem} is forcing a pkgload::load_all(). However, it should work.
The error is an IDE error. Does it work if you run it outside of the IDE (or in a terminal session)?
Side note: Instead of testthat::test_dir("tests/testthat"), you should be able to call shinytest2::test_app().
yogat3ch
commented
2 years ago Hi @schloerke, Thanks for the response and sorry about the extended delay on getting back to you! We had three sprint weeks on a different project so I have not had time to experiment with this until now.
I tried using shinytest2::test_app from the console and when doing so I encounter the following error:
The error is identical if hitting Run Tests from the Build pane.
I tried changing theapp_dir for shinytest2::AppDriver$new to "../../" and I think it started recursively loading the app:
I'm not sure how I would run shiny tests from a terminal session.
Still at a loss for how to get basic up & running with shinytest2
schloerke
commented
2 years ago Can I have you update to the latest RStudio IDE?
To run from the terminal, select the "Terminal" window pane in the IDE. Then enter R to start the R process. Given you're in your app folder, try shinytest2::record_test() from within the terminal.
yogat3ch
commented
2 years ago Hi @schloerke,
I just updated RStudio to 2022.07.2+576.
My setup-shinytest2.R looks as follows:
library(shinytest2)
test_that("Initial Shiny values are consistent", {
devtools::load_all(pkgload::pkg_path())
shinytest2::load_app_env()
app <-
shinytest2::AppDriver$new(
name = "Shiny_Driver1920x1080",
seed = 1,
height = 1920,
width = 1080,
options = options() |> purrr::list_modify(error = utils::recover, memoise = FALSE, use_local_crssDB = FALSE, use_db = TRUE)
)
app$expect_values()
})I ran shinytest2::record_test() from the terminal and it works better than expected as y'all seem to have added the ability to catch a great deal more browser events than previously! I recorded a test with the app and saved it.
I opened the file created by the tests and hit the "Run Tests" button.
The output is as follows:
The error:
── Error (setup-shinytest2.R:7:3): Initial Shiny values are consistent ─────────
Error in `app_initialize(self, private, app_dir = app_dir, ..., load_timeout = load_timeout,
timeout = timeout, wait = wait, expect_values_screenshot_args = expect_values_screenshot_args,
screenshot_args = screenshot_args, check_names = check_names,
name = name, variant = variant, view = view, height = height,
width = width, seed = seed, clean_logs = clean_logs, shiny_args = shiny_args,
render_args = render_args, options = options)`: Can't find `tests/testthat/` in current directory.
i You can inspect the failed AppDriver object via `rlang::last_error()$app`
i AppDriver logs:
{shinytest2} R info 11:51:32.98 Start AppDriver initialization
{shinytest2} R info 11:51:32.98 Error while initializing AppDriver:
Can't find `tests/testthat/` in current directory.
Caused by error in `testthat::test_path()`:
! Can't find `tests/testthat/` in current directory.is the same as when trying to run the setup-shinytest2.R file, so I think it's caused by the loading of the test app in that file.
Running shinytest2::test_app from the terminal has the same result. Thoughts?
yogat3ch
commented
2 years ago Hi @schloerke,
In attempting to debug, I modified setup-shinytest2.R as follows to print the absolute path of the directory since the error indicates that app_initialize cant find tests/testthat in the current directory:
library(shinytest2)
test_that("Initial Shiny values are consistent", {
devtools::load_all(pkgload::pkg_path())
load_app_env <- function (app_dir = "../../", renv = rlang::caller_env(), globalrenv = rlang::caller_env())
{
print(fs::path_abs(app_dir))
shiny::loadSupport(app_dir, renv = renv, globalrenv = globalrenv)
}
load_app_env()
app <-
shinytest2::AppDriver$new(
name = "Shiny_Driver1920x1080",
seed = 1,
height = 1920,
width = 1080,
options = options() |> purrr::list_modify(error = utils::recover, memoise = FALSE, use_local_crssDB = FALSE, use_db = TRUE)
)
app$expect_values()
})
# Load application support files into testing environment
shinytest2::load_app_env()
The location of setup-shinytest2.R can be seen in the subsection of this screenshot of the directory tree (the full tree is massive and would be confusing to display). The left most branch is the root folder of the golem package. It can be seen that tests/testthat is directly in the root folder.
When I run this file with the RStudio IDE, the print statement indicates that the app_dir argument evaluates to the root golem directory, so the load_app_env isn't the culprit.
The shinytest2::AppDriver was throwing the error. I modified the call to specify the app_dir as it uses testthat::test_path("../../") which resolves to "tests/testthat/../../" which doesn't look right. The app_dir now points to the package root.
debug(shinytest2:::app_start_shiny)
app <-
try(shinytest2::AppDriver$new(
app_dir = pkgload::pkg_path(),
name = "Shiny_Driver1920x1080",
seed = 1,
height = 1920,
width = 1080,
options = options() |> purrr::list_modify(error = rlang::trace_back, memoise = FALSE, use_local_crssDB = FALSE, use_db = TRUE)
))
print(app)I also added a debug and a try + print combination for more verbose logging.
When I run this via the RStudio IDE, I now encounter the error:
i You can inspect the failed AppDriver object via `rlang::last_error()$app`
i AppDriver logs:
{shinytest2} R info 15:07:41.00 Start AppDriver initialization
{shinytest2} R info 15:07:41.00 Starting Shiny app
{shinytest2} R info 15:07:42.07 Error while initializing AppDriver:
Error starting shiny application:
Loading required package: shiny
Warning in loadSupport(appDir, renv = sharedEnv, globalrenv = NULL) :
Loading R/ subdirectory for Shiny application, but this directory appears to contain an R package. Sourcing files in R/ may cause unexpected behavior.
i UU: `opts` & `toggle` functions were generated for the following options in '.Rprofile': use_db, max_policies, phase, use_local_crssDB, renv.config.auto.snapshot, memoise, use_reprex, use_debug, use_intro, shiny.autoload.r, shiny.testmode
Error in standardise_path(file) : object 'dirs' not found
{shiny} R stderr ----------- Loading required package: shiny
{shiny} R stderr ----------- Warning in loadSupport(appDir, renv = sharedEnv, globalrenv = NULL) :
{shiny} R stderr ----------- Loading R/ subdirectory for Shiny application, but this directory appears to contain an R package. Sourcing files in R/ may cause unexpected behavior.
{shiny} R stderr ----------- i UU: `opts` & `toggle` functions were generated for the following options in '.Rprofile': use_db, max_policies, phase, use_local_crssDB, renv.config.auto.snapshot, memoise, use_reprex, use_debug, use_intro, shiny.autoload.r, shiny.testmode
{shiny} R stderr ----------- Error in standardise_path(file) : object 'dirs' not found
Caused by error in `app_start_shiny()`:
! Error starting shiny application:
Loading required package: shiny
Warning in loadSupport(appDir, renv = sharedEnv, globalrenv = NULL) :
Loading R/ subdirectory for Shiny application, but this directory appears to contain an R package. Sourcing files in R/ may cause unexpected behavior.
i UU: `opts` & `toggle` functions were generated for the following options in '.Rprofile': use_db, max_policies, phase, use_local_crssDB, renv.config.auto.snapshot, memoise, use_reprex, use_debug, use_intro, shiny.autoload.r, shiny.testmode
Error in standardise_path(file) : object 'dirs' not found
---
Backtrace:
1. base::try(...)
at tests/testthat/setup-shinytest2.R:7:2
6. shinytest2::AppDriver$new(...)
7. shinytest2 (local) initialize(...)
8. shinytest2:::app_initialize(...)
10. shinytest2:::app_initialize_(self, private, ..., view = view)
11. shinytest2:::app_start_shiny(...)
── Warning (setup-shinytest2.R:4:3): Initial Shiny values are consistent ───────
replacing previous import 'ggplot2::last_plot' by 'plotly::last_plot' when loading 'dmdu'
Backtrace:
1. devtools::load_all(pkgload::pkg_path())
at tests/testthat/setup-shinytest2.R:4:2
2. pkgload::load_all(...)
3. pkgload:::load_imports(path)
4. pkgload:::process_imports(path)
9. base::namespaceImport(...)
10. base::namespaceImportFrom(...)dirs is a list object with functions that conveniently create file paths to various locations in the golem package. It executes before it's used due to roxygen2 @include statements in necessary locations.
When I try to run the code in this file at the console, I encounter the rs_shinyviewer error from earlier.
When I run shinytest2::test_app() from the terminal I am able to debug inside of app_start_shiny where I can see that the app starts successfully. The tests run, and when it gets to this setup file, I encounter the same error as when it runs via the RStudio IDE.
Two questions. The first implied by this error:
shinytest2 respect package collate order?
The second factors in when the app actually runs - we have database credentials in an .Renviron file that are necessary to the app running.shinytest2 load the project level ,Renviron file in the background task in which it runs the app?schloerke
commented
2 years ago
- Does
shinytest2respect package collate order?
Ah! Interesting. No it does not.
{shinytest2} uses ./tests/testthat/setup-shinytest2.R to call shinytest2::load_app_env(). That method does not respect collate.
Some changes can be made:
./tests/testthat/test-shinytest2.R (or a test-**.R) and not a setup-**.R file.shinytest2::load_app_env()
./tests/testthat/setup-shinytest2.R so that it does not call shinytest2::load_app_env(). ./tests/testthat.R to be shinytest2::test_app(check_setup = FALSE)../tests/testthat/setup-shinytest2.R to if (FALSE) shinytest2::load_app_env(){shinytest2} just calls shiny::runApp(). {golem} apps try to load the R folder for you via pkgload::load_all().
- Does
shinytest2load the project level ,Renviron file in the background task in which it runs the app?
Ah! Interesting. No it does not.
But it can!
I'll make a PR for the second bullet. Please let me know how the adjustments for the first bullet work for you!
yogat3ch
commented
2 years ago Hi @schloerke,
Thank you for the tips!
The app now boots successfully and passes the default "initial values" test.
I ran the test I recorded with record_test and am realizing that it did not pick up Datatable click actions, which are central to the interaction with our app.
When I was previously working on this, I developed some JS functions that use jQuery to reproduce these DT row clicks.
I was able to set up some javascript functions that checked for the presence of various DOM elements on the page.
Much of our testing requires this pattern: click some datatable rows, verify some DOM elements.
This was easy to do when I could just run the app loading code:
app <-
try(shinytest2::AppDriver$new(
app_dir = pkgload::pkg_path(),
variant = platform_variant(),
name = "Shiny_Driver1920x1080",
seed = 1,
height = 865,
width = 1211,
options = options() |> purrr::list_modify(error = rlang::trace_back, memoise = FALSE, use_local_crssDB = FALSE, use_db = TRUE)
))from the console, use app$view(), then proceed to use the app$run_js()/app$get_js() to execute tests of this nature.
However, recent updates to shinytest2 now prevent the app from loading at the console due to the rs_shinyviewer error:
Error in .Call("rs_shinyviewer", url, getwd(), "browser", NULL, PACKAGE = "(embedding)") :
"rs_shinyviewer" not available for .Call() for package "(embedding)"I am however, thanks to this conversation, now able to get the app running from the terminal, and use app$view to see the running app to develop jQuery based tests!
Do you have any ideas as to how to overcome the rs_shinyviewer bug? Should I open a new issue for that?
Thank you for all of your help and for the PR!
yogat3ch
commented
1 year ago Hi @schloerke, I've run into a wall with this workaround. I just updated the package to see any of these issues have been addressed but it doesn't appear so.
I tried running the app at the console and it encounters the same error as this issue was opened for:
Error in .Call("rs_shinyviewer", url, getwd(), "browser", NULL, PACKAGE = "(embedding)") :
"rs_shinyviewer" not available for .Call() for package "(embedding)"However, I'm also now unable to use the terminal session workaround, app$view() results in this error:
I've taken a look at the logs for the terminal session and there don't appear to be any R or Javascript errors that would cause it to fail.
I've reset chrome devtools to default settings to no avail. I'm running Chrome Version 111.0.5563.64 (Official Build) (arm64). My coworkers machine does not seem to experience this issue.
Any ideas on how to resolve this? I'm basically SOL on using shinytest2 at all at this point and our testing framework is failing because I can't maintain it due to the bugs with view
Any assistance will be much appreciated 🙏
schloerke
commented
1 year ago I can reprex {chromote} disconnecting by making a new AppDriver. This is unrelated to {golem} / RStudio IDE. Making a new issue for better tracking.
Hi shinytest2 devs, We're trying to set-up the shinytest2 package with a large golem app and running into issues each way we turn. The first of which is attempting to run the basic shinytest2 test.
The tests folder is organized as follows:
test-inst-apps.R was an attempt to get the app running from the
inst/test_appfolder, but that never worked properly so it's now commented out.The contents of test-shinytest2.R are as follows:
The options configure the app to run properly.
Trying to run the tests with:
testthat::test_dir("tests/testthat")Results in the following error:
This had previously not been a problem as of a few months ago when trying to set up shinytest2 and running them with
test_dir, but it's cropped up in the more recent version.We've been pushing off building tests for the app because we're unable to get shinytest2 to work, but the app is getting large enough where it's becoming very cumbersome and time-consuming for every code review to run through all the app functionality manually to ensure breaks aren't occurring.
I'm hoping that with some help from the team at RStudio (Posit?) we can figure out how to put some shiny tests in place!
Thanks in advance for the assistance, Stephen
sessionInfo()
R version 4.2.1 (2022-06-23) Platform: aarch64-apple-darwin20 (64-bit) Running under: macOS Monterey 12.6 Matrix products: default LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices datasets utils methods base other attached packages: [1] dmdu_0.0.0.9001 shinytest2_0.2.0 testthat_3.1.4 shiny_1.7.2 loaded via a namespace (and not attached): [1] backports_1.4.1 plyr_1.8.7 lazyeval_0.2.2 websocket_1.4.1 tippy_1.0.0 crosstalk_1.2.1 [7] usethis_2.1.6 ggplot2_3.3.6 digest_0.6.29 htmltools_0.5.3 parsnip_1.0.1 fansi_1.0.3 [13] magrittr_2.0.3 checkmate_2.1.0 memoise_2.0.1 googlesheets4_1.0.0 config_0.3.1 tzdb_0.3.0 [19] remotes_2.4.2 readr_2.1.2 vroom_1.5.7 hardhat_1.2.0 prettyunits_1.1.1 colorspace_2.0-3 [25] rvest_1.0.3 xfun_0.33 dplyr_1.0.10 callr_3.7.2 crayon_1.5.2 cicerone_1.0.5.9000 [31] BOR_0.0.0.9000 jsonlite_1.8.0 roxygen2_7.2.1 tidyrules_0.1.5 glue_1.6.2 gtable_0.3.1 [37] gargle_1.2.0 sortable_0.4.6 pkgbuild_1.3.1 crssDB_0.5.2 scales_1.2.1 pool_0.1.6 [43] DBI_1.1.3 miniUI_0.1.1.1 Rcpp_1.0.9 viridisLite_0.4.1 xtable_1.8-4 echarty_1.4.5 [49] UU_0.32.0 learnr_0.10.1 bit_4.0.4 DT_0.23 profvis_0.3.7 htmlwidgets_1.5.4 [55] httr_1.4.4 ellipsis_0.3.2 urlchecker_1.0.1 pkgconfig_2.0.3 sass_0.4.2 chromote_0.1.1 [61] utf8_1.2.2 tidyselect_1.1.2 rlang_1.0.6 later_1.3.0 munsell_0.5.0 cellranger_1.1.0 [67] tools_4.2.1 cachem_1.0.6 cli_3.4.1 generics_0.1.3 english_1.2-6 devtools_2.4.4 [73] evaluate_0.16 stringr_1.4.1 fastmap_1.1.0 yaml_2.3.5 processx_3.7.0 knitr_1.40 [79] bit64_4.0.5 fs_1.5.2 purrr_0.3.4 reactable_0.3.0 mime_0.12 xml2_1.3.3 [85] brio_1.1.3 compiler_4.2.1 rstudioapi_0.14 plotly_4.10.0 curl_4.3.2 tibble_3.1.8 [91] bslib_0.4.0 stringi_1.7.8 ps_1.7.1 desc_1.4.2 virgaUtils_0.2.0 markdown_1.1 [97] shinyjs_2.1.0 vctrs_0.4.2 pillar_1.8.1 lifecycle_1.0.2 jquerylib_0.1.4 data.table_1.14.2 [103] cowplot_1.1.1 httpuv_1.6.6 patchwork_1.1.1 R6_2.5.1 RMySQL_0.10.23 promises_1.2.0.1 [109] renv_0.16.0 rpart.plot_3.1.1 sessioninfo_1.2.2 assertthat_0.2.1 pkgload_1.3.0 golem_0.3.4 [115] bs4Dash_2.1.0 rprojroot_2.0.3 shinyWidgets_0.7.1 withr_2.5.0 parallel_4.2.1 hms_1.1.2 [121] shinyvalidate_0.1.2 grid_4.2.1 rpart_4.1.16 tidyr_1.2.1 waldo_0.4.0 rmarkdown_2.16 [127] attempt_0.3.1 snakecase_0.11.0 googledrive_2.0.0 lubridate_1.8.0