Support R packages - Githubissues

schloerke commented 1 year ago

Should not require ./tests/testthat/setup-shinytest2.R
Requires testthat file loading support
Listen to something in the DESCRIPTION file. Maybe Config/shinytest/package: true or is there a golem/rhino/etc key to listen to?
How to know the run method? Make a consistent run method?
- Ex: If package, then MYPKG::shinytest2_run_app(name=) must be supported.
  - MYPKG::run_app() currently clashes with golem's run_app().
  - Maybe it'll work out the box?
  - Maybe MYPKG::shinytest2_app(name=)?
- Must be a run-like method and not a get_shiny_app method.
  - Would fix #311

schloerke commented 1 year ago

Add support for "app directory only" driver. AppDirDriver(app_dir=)?

Add support for a package driver that wraps MYPKG::shinytest2_run_app(). PkgAppDriver$new(name=)?

If these two are being added, let's be explicit and add support for AppUrlDriver$new(url=) which handles a URL.

For all of these drivers, snaps will be saved in package snaps folder. No nested test structures!!!

schloerke commented 1 year ago

If AppDirDriver is implemented. There is a strong case for soft deprecating AppDriver and removing AppDriver from the public docs.

ColinFay commented 2 months ago

Hey,

TL;DR for anyone finding this comment:

Run usethis::use_testthat();shinytest2::use_shinytest2();shinytest2::use_shinytest2_test();golem::add_rstudioconnect_file()
Modify the test with:

test_that("Initial Shiny values are consistent", {
  skip_if(testthat::is_checking() || !interactive())

The shinytest will work with devtools::test() and be skipped otherwise.

Here are the current state of my explorations on the subject:

Rscript -e "golem::create_golem('testshinytest2')"
cd testshinytest2 
R -e "usethis::use_testthat();shinytest2::use_shinytest2();shinytest2::use_shinytest2_test()"

> fs::dir_tree()
.
├── DESCRIPTION
├── NAMESPACE
├── R
│   ├── app_config.R
│   ├── app_server.R
│   ├── app_ui.R
│   └── run_app.R
├── dev
│   ├── 01_start.R
│   ├── 02_dev.R
│   ├── 03_deploy.R
│   └── run_dev.R
├── inst
│   ├── app
│   │   └── www
│   │       └── favicon.ico
│   └── golem-config.yml
├── man
│   └── run_app.Rd
└── tests
    ├── testthat
    │   ├── setup-shinytest2.R
    │   └── test-shinytest2.R
    └── testthat.R

Out of the box (no app.R), the test suite does not work (yet still manage to try and launch the app):

library(shinytest2)

test_that("Initial Shiny values are consistent", {
  app <- AppDriver$new()

  app$expect_values()
})

# REDACTED for clarity
# [...]
{shiny}      R  stderr ----------- Listening on http://127.0.0.1:3899
{shiny}      R  stderr ----------- Warning: Error in golem_add_external_resources: could not find function "golem_add_external_resources"
# [...]
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]
Warning message:
In warn_if_app_dir_is_package(appDir) :
  Loading R/ subdirectory for Shiny application, but this directory appears to contain an R package. Sourcing files in R/ may cause unexpected behavior. See `?loadSupport` for more details.

golem_add_external_resources() is at app_ui.R#26.

> golem::add_rstudioconnect_file()

Will add the app.R & the _disable_autoload.R, and :

testthat::test_local() works
devtools::check() doesn't

══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-shinytest2.R:4:3'): Initial Shiny values are consistent ────────
Error in `app_initialize(self, private, app_dir = app_dir, ..., load_timeout = load_timeout, 
    timeout = timeout, wait = wait, expect_values_screenshot_args = expect_values_screenshot_args, 
    screenshot_args = screenshot_args, check_names = check_names, 
    name = name, variant = variant, view = view, height = height, 
    width = width, seed = seed, clean_logs = clean_logs, shiny_args = shiny_args, 
    render_args = render_args, options = options)`: Error starting shiny application:
Loading required package: shiny
Error in shinyAppDir(x) : App dir must contain either app.R or server.R.

i You can inspect the failed AppDriver object via `rlang::last_error()$app`
i AppDriver logs:
{shinytest2} R info   10:29:14.58 Start AppDriver initialization
{shinytest2} R info   10:29:14.58 Starting Shiny app
{shinytest2} R info   10:29:15.23 Error while initializing AppDriver:
                                  Error starting shiny application:
                                  Loading required package: shiny
                                  Error in shinyAppDir(x) : App dir must contain either app.R or server.R.
{shiny}      R stderr ----------- Loading required package: shiny
{shiny}      R stderr ----------- Error in shinyAppDir(x) : App dir must contain either app.R or server.R.

Which makes sense, as the package does not bundle the app.R.

The safest way would be to have:

test_that("Initial Shiny values are consistent", {
  app <- AppDriver$new(
    run_app()
  )

  app$expect_values()
})

as run_app() return a shiny.appobj

> class(run_app())
[1] "shiny.appobj"

which doesn't work either:

Error (test-shinytest2.R:4:3): Initial Shiny values are consistent

{shiny}      R  stderr ----------- Warning: Error in golem_add_external_resources: could not find function "golem_add_external_resources"

[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]
Error: Test failures

But

test_that("Initial Shiny values are consistent", {
  print(golem_add_external_resources)
  app <- AppDriver$new(
    run_app()
  )

  app$expect_values()
})

Does find the function:

> testthat::test_local()
✔ | F W  S  OK | Context
⠏ |          0 | shinytest2                                                                                                                                                                                       function() {
  add_resource_path(
    "www",
    app_sys("app/www")
  )

  tags$head(
    favicon(),
    bundle_resources(
      path = app_sys("app/www"),
      app_title = "testshinytest2"
    )
    # Add here other external resources
    # for example, you can add shinyalert::useShinyalert()
  )
}
<environment: namespace:testshinytest2>

Digging into the shinytest2 code, I've found that when passing an obj, a template app.R is copied and paste, then the globals are saved and re-read, but as the comments says here https://github.com/rstudio/shinytest2/blob/main/R/save-app.R#L17 "what happen if app uses non-exported function?", which is the current case for golem_add_external_resources.

Another note here, the app_data seems to record only the values from server, not the ui.

Will keep exploring and come back here later.

schloerke commented 2 months ago

@ColinFay For package support, I'd like to avoid the save-app approach if possible

I'd like to add / leverage the load_package= argument of testthat::test_dir() ( sent in to here: https://github.com/rstudio/shinytest2/blob/436535d0ec563abeccb6b6a9323d30c5b20199c0/R/test-app.R#L254 ) and pass it along to somewhere near https://github.com/rstudio/shinytest2/blob/436535d0ec563abeccb6b6a9323d30c5b20199c0/R/app-driver-start.R#L50 so that the background R session can call pkgload::load_all(), just like {testthat} does.

By using the same variable, we can have predictable testthat-like behavior.

While I'm leaning away from save_app(), it wouldn't hurt to also have the loaded environment added to the searchable environments when saving the app.

rstudio / shinytest2

Support R packages #308