Is this the correct way to test that an api function fails?

[DaveParr]

In basic test_that the following is a super simplified example of testing that an error is generated:

x <- 12
attempt::stop_if_not(x, is.character)
testthat::expect_error(attempt::stop_if_not(x, is.character))

x is 12, Error: Testis.characteronxreturned an error., testthat silently passes.

A more complex example in my package is:

check_status <- function(response, operation = "", expected = 200) {
  code <- httr::status_code(response)
  attempt::stop_if_not(
    .x = code,
    .p = ~ .x == expected,
    msg = glue::glue("The API returned an error: {code}\nSee https://docs.dev.to/api/#operation/{operation} for error codes", code = code)
  )
  attempt::message_if(
    .x = code,
    .p = ~ .x == expected,
    msg = glue::glue("The API returned the expected success: {code}", code = code)
  )
}

get_my_user <- function(key = NA) {

  check_internet()

  response <- httr::GET(url = "https://dev.to/api/users/me",
                        httr::add_headers("api-key" = api_key(key = key)),
                        user_agent)

  check_json(response)

  check_status(response, operation = "getUserArticles", expected = 200)

  response
}

vcr::use_cassette("get_my_user_auth_failure", {
  test_that("doesn't get a user", {
    my_fail_user <- dev.to.ol::get_my_user("fail")
    expect_error(my_fail_user)
  })
})

check_status expects a 200, but because the API key is going to fail, the function will actually return a 401, which it's the stop_if_not.

get_user calls this function with the response from the api where it has supplied the bad apikey: "fail". The cassette does get recorded seemingly correctly. The test however fails:

Loading dev.to.ol
✓ |  OK F W S | Context
x |   2 1     | get_my_user [0.4 s]
test-get_my_user.R:12: error: doesn't get a user
The API returned an error: 401
See https://docs.dev.to/api/#operation/getUserArticles for error codes
Backtrace:
 1. dev.to.ol::get_my_user("fail") ~Documentsdev.to.olteststestthattest-get_my_user.R:12:4
 2. dev.to.ol::check_status(...) UsersdavidparrDocumentsdev.to.olRget_my_user.R:28:2
 3. attempt::stop_if_not(...) UsersdavidparrDocumentsdev.to.olRutils.R:11:2

Shouldn't this test succeed? What have I missed?

session info

``` R version 3.6.0 (2019-04-26) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS 10.15.5 Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib locale: [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] testthat_2.3.2 printr_0.1 forcats_0.4.0 stringr_1.4.0 dplyr_0.8.5 [6] purrr_0.3.3 readr_1.3.1 tidyr_1.0.2 tibble_3.0.1 ggplot2_3.3.0.9000 [11] tidyverse_1.3.0 dev.to.ol_0.0.0.9000 loaded via a namespace (and not attached): [1] Rcpp_1.0.4 lubridate_1.7.4 lattice_0.20-38 assertthat_0.2.1 digest_0.6.25 packrat_0.5.0 R6_2.4.1 [8] cellranger_1.1.0 backports_1.1.6 reprex_0.3.0 evaluate_0.14 httr_1.4.1 pillar_1.4.3 rlang_0.4.5 [15] lazyeval_0.2.2 curl_3.3 readxl_1.3.1 rstudioapi_0.11 whisker_0.3-2 vcr_0.5.4.91 rmarkdown_2.1 [22] urltools_1.7.3 triebeard_0.3.0 munsell_0.5.0 broom_0.5.2 compiler_3.6.0 modelr_0.1.5 xfun_0.13 [29] pkgconfig_2.0.3 base64enc_0.1-3 htmltools_0.3.6 tidyselect_1.0.0 httpcode_0.3.0 attempt_0.3.1 fansi_0.4.1 [36] crayon_1.3.4 dbplyr_1.4.2 withr_2.2.0 crul_0.9.0 grid_3.6.0 nlme_3.1-140 jsonlite_1.6 [43] gtable_0.3.0 lifecycle_0.2.0 DBI_1.0.0 magrittr_1.5 scales_1.1.0 cli_2.0.2 stringi_1.4.6 [50] fauxpas_0.5.0 fs_1.3.1 xml2_1.3.2 ellipsis_0.3.0 generics_0.0.2 vctrs_0.2.4 webmockr_0.6.2 [57] tools_3.6.0 glue_1.4.0 hms_0.5.3 yaml_2.2.0 colorspace_1.4-1 rvest_0.3.5 knitr_1.26 [64] haven_2.2.0 ```

[sckott]

thanks @DaveParr I'll have a look

[sckott]

Doesn't the error happen during the get_my_user() call? if so, the expect_error call needs to be immediately wrapped around the get_my_user() call. like

vcr::use_cassette("get_my_user_auth_failure", {
  test_that("doesn't get a user", {
    expect_error(get_my_user("fail"))
  })
})

[DaveParr]

It probably is that. Sorry, musta had a mind fart when I was doing it the first time :)

[sckott]

no worries, glad it works