Open nick-youngblut opened 2 years ago
I created a simple snapshot test:
library(shinytest2) test_that("{shinytest2} recording: startup", { app <- AppDriver$new(variant = platform_variant(), name = "startup", seed = 3812, height = 976, width = 1619) app$expect_screenshot() })
...which passes when I run shinytest2::test_app() (mac-OS 12.6). However, my GitHub action on macos-latest fails:
shinytest2::test_app()
macos-latest
✔ | F W S OK | Context date, intersect, setdiff, union ⠏ | 0 | shinytest2 ⠋ | 1 0 | shinytest2 ✖ | 1 0 | shinytest2 [11.0s] ──────────────────────────────────────────────────────────────────────────────── Failure (test-shinytest2.R:7): {shinytest2} recording: startup Snapshot of `file` to 'shinytest2/startup-001.png' has changed Error: Error: Test failures Run `testthat::snapshot_review('shinytest2/')` to review changes Execution halted Backtrace: 1. app$expect_screenshot() at test-shinytest2.R:7:2 2. shinytest2:::app_expect_screenshot_and_variant(...) 3. shinytest2:::app_expect_screenshot(self, private, ...) 4. shinytest2:::app__expect_snapshot_file(...) 6. testthat::expect_snapshot_file(...)
My check-app.yaml file:
check-app.yaml
on: push: branches: [main, master] pull_request: branches: [main, master] name: my_shiny_app jobs: test-app: runs-on: ${{ matrix.os }} name: ${{ matrix.os }} (${{ matrix.r }}) strategy: fail-fast: false matrix: os: [ubuntu-latest, macos-latest] r: [4.2.1] env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} R_KEEP_PKG_SOURCE: yes steps: - uses: actions/checkout@v2 - uses: r-lib/actions/setup-pandoc@v2 - uses: r-lib/actions/setup-r@v2 with: r-version: ${{ matrix.r }} http-user-agent: ${{ matrix.config.http-user-agent }} use-public-rspm: true - uses: r-lib/actions/setup-renv@v2 - uses: rstudio/shinytest2/actions/test-app@v1 with: path: "."
The CI test OS: macOS 12.6.1
macOS 12.6.1
If I had to guess on what the problem is, I would bet on the png width slightly changing. It appears that the png width doesn't stay constant in all conditions, even though width= is set via AppDriver$new()
width=
AppDriver$new()
Related: https://github.com/rstudio/shinytest2/issues/171
I created a simple snapshot test:
...which passes when I run
shinytest2::test_app()
(mac-OS 12.6). However, my GitHub action onmacos-latest
fails:My
check-app.yaml
file:The CI test OS:
macOS 12.6.1
If I had to guess on what the problem is, I would bet on the png width slightly changing. It appears that the png width doesn't stay constant in all conditions, even though
width=
is set viaAppDriver$new()