r-lib / testthat

An R 📦 to make testing 😀
https://testthat.r-lib.org
Other
888 stars 318 forks source link

test_that desc with line breaks causes expect_snapshot to always be overwritten. #1900

Open LDSamson opened 10 months ago

LDSamson commented 10 months ago

Using test_that descriptions with line breaks can be quite useful, as described here. However I encountered an issue using this format with expect_snapshot(): The snapshots are saved correctly, but will always be overwritten when a test is run again.

Example code:

test_that(
  "Descriptions
  with multiple lines
  will cause snapshots to always be overwritten.
  ", {
    expect_snapshot(1*1)
  }
)

Outcome, irrespective of how often I run the test:

── Warning (test-hello.R:6:5): Descriptions
  with multiple lines
  will cause snapshots to always be overwritten.
   ──
Adding new snapshot:
Code
  1 * 1
Output
  [1] 1
[ FAIL 0 | WARN 1 | SKIP 0 | PASS 1 ]

The snapshot output is always the same, see below, but is overwritten every time I run the test.

# Descriptions
  with multiple lines
  will cause snapshots to always be overwritten.

    Code
      1 * 1
    Output
      [1] 1
Session Info ─ Session info ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── setting value version R version 4.3.1 (2023-06-16 ucrt) os Windows 11 x64 (build 22621) system x86_64, mingw32 ui RStudio language EN collate German_Germany.utf8 ctype German_Germany.utf8 tz Europe/Berlin date 2023-11-15 rstudio 2023.06.2+561 Mountain Hydrangea (desktop) pandoc NA ─ Packages ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ! package * version date (UTC) lib source brio 1.1.3 2021-11-30 [1] CRAN (R 4.3.1) cachem 1.0.8 2023-05-01 [1] CRAN (R 4.3.1) callr 3.7.3 2022-11-02 [1] CRAN (R 4.3.1) cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.1) crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.1) desc 1.4.2 2022-09-08 [1] CRAN (R 4.3.1) devtools * 2.4.5 2022-10-11 [1] CRAN (R 4.3.1) digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.1) ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.3.1) evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.1) fansi 1.0.4 2023-01-22 [1] CRAN (R 4.3.1) fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.1) fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.1) glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.1) htmltools 0.5.6 2023-08-10 [1] CRAN (R 4.3.1) htmlwidgets 1.6.2 2023-03-17 [1] CRAN (R 4.3.1) httpuv 1.6.11 2023-05-11 [1] CRAN (R 4.3.1) later 1.3.1 2023-05-02 [1] CRAN (R 4.3.1) lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.1) magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.1) memoise 2.0.1 2021-11-26 [1] CRAN (R 4.3.1) mime 0.12 2021-09-28 [1] CRAN (R 4.3.0) miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.3.1) pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.1) pkgbuild 1.4.2 2023-06-26 [1] CRAN (R 4.3.1) pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.1) pkgload 1.3.2.1 2023-07-08 [1] CRAN (R 4.3.1) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.3.1) processx 3.8.2 2023-06-30 [1] CRAN (R 4.3.1) profvis 0.3.8 2023-05-02 [1] CRAN (R 4.3.1) promises 1.2.1 2023-08-10 [1] CRAN (R 4.3.1) ps 1.7.5 2023-04-18 [1] CRAN (R 4.3.1) purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.1) R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.1) Rcpp 1.0.11 2023-07-06 [1] CRAN (R 4.3.1) remotes 2.4.2 2021-11-30 [1] CRAN (R 4.2.1) rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.1) rprojroot 2.0.3 2022-04-02 [1] CRAN (R 4.2.1) rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.1) sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.1) shiny 1.7.4.1 2023-07-06 [1] CRAN (R 4.3.1) stringi 1.7.12 2023-01-11 [1] CRAN (R 4.2.2) stringr 1.5.0 2022-12-02 [1] CRAN (R 4.2.3) testthat * 3.2.0 2023-10-06 [1] CRAN (R 4.3.2) tibble 3.2.1 2023-03-20 [1] CRAN (R 4.2.3) urlchecker 1.0.1 2021-11-30 [1] CRAN (R 4.2.1) usethis * 2.2.2 2023-07-06 [1] CRAN (R 4.3.1) utf8 1.2.3 2023-01-31 [1] CRAN (R 4.2.3) vctrs 0.6.3 2023-06-14 [1] CRAN (R 4.3.1) waldo 0.5.1 2023-05-08 [1] CRAN (R 4.3.1) withr 2.5.0 2022-03-03 [1] CRAN (R 4.2.1) xtable 1.8-4 2019-04-21 [1] CRAN (R 4.2.1)
LDSamson commented 9 months ago

Some context about this issue: I discovered it when I tried to implement Behaviour Driven Development processes in my unit tests, using longer descriptions in the functions testthat::describe() and testthat::it(). I use the longer descriptions so that I can scrape them with the unit tests using a ListReporter to create a traceability matrix that can be reported in an automated manner. I found that such a longer description greatly helps to create a clear traceability matrix for not only the developers but also stakeholders.

Although there are of course workarounds, better support for snapshots in this format would be more than welcome! If I can help with anything, I am happy to do so.