Tests Not Idempotent - Githubissues

Tests fail if run multiple times in the same R session.

Session Info

```r r$> sessionInfo() R version 4.0.0 (2020-04-24) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Catalina 10.15.4 Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] vcr_0.5.4.91 httr_1.4.1 crul_0.9.0 testthat_2.3.2 loaded via a namespace (and not attached): [1] Rcpp_1.0.4.6 compiler_4.0.0 base64enc_0.1-3 prettyunits_1.1.1 remotes_2.1.1 tools_4.0.0 digest_0.6.25 pkgbuild_1.0.8 pkgload_1.1.0 jsonlite_1.6.1 [11] memoise_1.1.0.9000 rlang_0.4.6 cli_2.0.2 rstudioapi_0.11 curl_4.3 yaml_2.2.1 xml2_1.3.2 withr_2.2.0 desc_1.2.0 fauxpas_0.5.0 [21] fs_1.4.1 devtools_2.3.0 triebeard_0.3.0 rprojroot_1.3-2 glue_1.4.1 httpcode_0.3.0 R6_2.4.1 processx_3.4.2 fansi_0.4.1 sessioninfo_1.1.1 [31] callr_3.4.3 webmockr_0.6.2 magrittr_1.5 whisker_0.4 backports_1.1.7 urltools_1.7.3 ps_1.3.3 ellipsis_0.3.1 usethis_1.6.1 assertthat_0.2.1 [41] mime_0.9 lazyeval_0.2.2 crayon_1.3.4.9000 ```

Example

```r r$> devtools::test() Loading vcr Testing vcr ✔ | OK F W S | Context ✔ | 15 | Cassette [0.2 s] ✔ | 12 | FileSystem ✔ | 11 | HttpInteraction [0.4 s] ✔ | 13 | HTTPInteractionList [0.1 s] ✔ | 6 | Persisters ✔ | 35 | Request ✔ | 7 | RequestHandler [0.5 s] ✔ | 25 1 | RequestIgnorer [1.2 s] ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-RequestIgnorer.R:114: skip: RequestIgnorer usage: w/ vcr_configure() usage Reason: port 8000 not available ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 51 | RequestMatcherRegistry ✔ | 14 | use_cassette: works as expected [0.4 s] ✔ | 8 | use_cassette fails well ✔ | 6 | ause_cassette_match_body_empty_body [0.5 s] ✔ | 32 | ause_cassette_match_requests_on [1.2 s] ✔ | 15 | ause_cassette_match_requests_on_json [0.5 s] ✔ | 3 | use_cassette options: re_record_interval [10.6 s] ✔ | 34 | use_cassette: record modes work as expected [2.4 s] ✔ | 21 | use_cassette: write to disk [0.6 s] ✔ | 30 | use_cassette: handle binary files on disk [3.0 s] ✔ | 0 1 | handling images: httr ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-binary_images.R:8: skip: use_cassette w/ with images: httr Reason: jpeg cannot be loaded ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 10 | handling images: crul [0.5 s] ✔ | 6 | cassette_options ✔ | 4 | cassettes ✔ | 5 | current_cassette ✔ | 3 | cassette_path ✔ | 4 | check_cassette_names ✔ | 14 | configuration-inheritance ✔ | 12 | Configuration [0.2 s] ✔ | 27 | adapter-crul: POST requests works [0.7 s] ✔ | 4 | eject_cassette ✔ | 8 | UnhandledHTTPRequestError ✔ | 20 | UnhandledHTTPRequestError: secret handling [0.1 s] ✔ | 8 | adapter-httr: status code works [0.2 s] ✔ | 12 | adapter-httr: use_cassette works [0.1 s] ✔ | 10 | adapter-httr: use_cassette w/ preserve_exact_body_bytes [0.1 s] ✔ | 17 | adapter-httr: use_cassette w/ >1 request per cassette [0.4 s] ✔ | 3 | adapter-httr: use_cassette w/ simple auth [0.1 s] ✔ | 22 | adapter-httr: POST requests works [0.8 s] ✔ | 7 | insert_cassette fails well ✔ | 25 | insert_cassette works [0.1 s] ✔ | 39 | lightswitch functions [0.5 s] ✔ | 3 | logger: vcr_log_file fails well ✔ | 17 | logger: vcr_log_file works ✔ | 0 | no_cassette_in_use ✔ | 23 | request_summary [0.1 s] ✔ | 15 | response_summary [0.1 s] ✔ | 23 | serializers: YAML ✔ | 16 | use_vcr works ✔ | 36 | utilities ✔ | 10 | vcr_last_error ✔ | 2 | write_disk_path: package context ✔ | 13 | write_interactions ✔ | 14 | write: utils ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ Duration: 27.0 s OK: 770 Failed: 0 Warnings: 0 Skipped: 2 r$> devtools::test() Loading vcr Testing vcr ✔ | OK F W S | Context ✔ | 15 | Cassette ✔ | 12 | FileSystem ✔ | 11 | HttpInteraction [0.4 s] ✔ | 13 | HTTPInteractionList [0.1 s] ✔ | 6 | Persisters ✔ | 35 | Request ✔ | 7 | RequestHandler [0.4 s] ✔ | 25 1 | RequestIgnorer [1.2 s] ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-RequestIgnorer.R:114: skip: RequestIgnorer usage: w/ vcr_configure() usage Reason: port 8000 not available ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 51 | RequestMatcherRegistry ✔ | 14 | use_cassette: works as expected [0.4 s] ✔ | 8 | use_cassette fails well ✔ | 6 | ause_cassette_match_body_empty_body [0.5 s] ✔ | 32 | ause_cassette_match_requests_on [1.0 s] ✔ | 15 | ause_cassette_match_requests_on_json [0.6 s] ✔ | 3 | use_cassette options: re_record_interval [10.5 s] ✔ | 34 | use_cassette: record modes work as expected [2.4 s] ✔ | 21 | use_cassette: write to disk [0.6 s] ✔ | 30 | use_cassette: handle binary files on disk [3.3 s] ✔ | 0 1 | handling images: httr ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-binary_images.R:8: skip: use_cassette w/ with images: httr Reason: jpeg cannot be loaded ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 10 | handling images: crul [0.5 s] ✔ | 6 | cassette_options ✖ | 2 2 | cassettes ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-cassettes.R:8: failure: cassettes works length(aa) not equal to 0. 1/1 mismatches [1] 3 - 0 == 3 test-cassettes.R:14: failure: cassettes works Names of `bb` ('abcd', 'crul_post_upload_file_no_list', 'foobar24', 'httr_test1') don't match 'foobar24' ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 5 | current_cassette ✔ | 3 | cassette_path ✔ | 4 | check_cassette_names ✔ | 14 | configuration-inheritance ✔ | 12 | Configuration ✔ | 27 | adapter-crul: POST requests works [0.8 s] ✔ | 4 | eject_cassette ✔ | 8 | UnhandledHTTPRequestError ✔ | 20 | UnhandledHTTPRequestError: secret handling [0.1 s] ✔ | 8 | adapter-httr: status code works [0.2 s] ✔ | 12 | adapter-httr: use_cassette works [0.1 s] ✔ | 10 | adapter-httr: use_cassette w/ preserve_exact_body_bytes [0.1 s] ✔ | 17 | adapter-httr: use_cassette w/ >1 request per cassette [0.4 s] ✔ | 3 | adapter-httr: use_cassette w/ simple auth [0.1 s] ✔ | 22 | adapter-httr: POST requests works [0.8 s] ✔ | 7 | insert_cassette fails well ✔ | 25 | insert_cassette works [0.1 s] ✔ | 39 | lightswitch functions [0.5 s] ✔ | 3 | logger: vcr_log_file fails well ✔ | 17 | logger: vcr_log_file works ✔ | 0 | no_cassette_in_use ✔ | 23 | request_summary [0.1 s] ✔ | 15 | response_summary [0.1 s] ✔ | 23 | serializers: YAML ✖ | 15 1 1 | use_vcr works ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── test-use_vcr.R:36: failure: use_vcr fails well `use_vcr(dir)` threw an error with unexpected message. Expected match: "'dir' does not exist" Actual message: "'DESCRIPTION' not found; are you sure it's an R package?" Backtrace: 1. testthat::expect_error(use_vcr(dir), "'dir' does not exist") tests/testthat/test-use_vcr.R:36:2 6. vcr::use_vcr(dir) 7. vcr::pkg_name(dir) R/vcr_setup.R:76:2 test-use_vcr.R:40: warning: use_vcr fails well '/var/folders/74/yl_jm18j02z_s2_m11v203k40000gn/T//RtmpckmCGf/foobar2' already exists ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔ | 36 | utilities ✔ | 10 | vcr_last_error ✔ | 2 | write_disk_path: package context ✔ | 13 | write_interactions [0.1 s] ✔ | 14 | write: utils ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ Duration: 26.8 s OK: 767 Failed: 3 Warnings: 1 Skipped: 2 ```

ropensci / vcr

Tests Not Idempotent #174