sboysel / fredr

An R client for the Federal Reserve Economic Data (FRED) API
https://sboysel.github.io/fredr/
Other
92 stars 21 forks source link

Better test environment for API requests #73

Closed sboysel closed 4 years ago

sboysel commented 4 years ago

Following up on the discussion in #72 . Running the full test suite leads to the package build time running in excess of 20 mins, mostly due to latency and/or connection issues with the FRED server. Using a more elegant web testing framework such as nealrichardson/httptest would handle test calls more efficiently.

sboysel commented 4 years ago

Working on mocking API requests with nealrichardson/httptest. Running into error in basic setup so posting here for reference. Captured requests with httptest::capture_request() stored in tests/testthat but then wrapping requests with httptest::with_mock_api() throws an error

> getwd()
[1] "/Users/sboysel/repos/fredr/tests/testthat"
> 
> devtools::load_all(here::here())
Loading fredr
> library(httptest)
> options(httptest.verbose=TRUE)
> 
> skip_if_no_key <- function() {
+     testthat::skip_if(
+         condition = identical(Sys.getenv("FRED_API_KEY"), ""),
+         message = "FRED API key is not set."
+     )
+ }
> 
> # mock paths
> httptest::.mockPaths()
[1] "."
> 
> # capture
> httptest::start_capturing()
> readRenviron(here::here(".Renviron"))
> fredr::fredr_set_key(Sys.getenv("FRED_API_KEY"))
> fredr_category(category_id = 10L)
Writing /Users/sboysel/repos/fredr/tests/testthat/api.stlouisfed.org/fred/category-af4199.json
# A tibble: 1 x 3
     id name                                    parent_id
  <int> <chr>                                       <int>
1    10 Population, Employment, & Labor Markets         0
> httptest::stop_capturing()
> 
> # test with mock api
> httptest::with_mock_api({
+     test_that("fredr_category()", {
+         skip_if_no_key()
+         expect_silent(ctg <- fredr_category(category_id = 10L))
+     })
+ })
Tracing function "form_file" as seen from package "httr"
Error: Test failed: 'fredr_category()'
* `ctg <- fredr_category(category_id = 10L)` produced messages.
Untracing function "form_file" in package "curl"
> 

So httptest::.mockPaths() are default (relative to tests/testthat). Can't see if I'm missing something fundamental from nealrichardson/httptest docs.

Edit: From the error message, perhaps with_mock_api() is generating messages causing expect_silent() to raise an error.

sboysel commented 4 years ago

Another complication to testing: switching from httr::GET to httr::RETRY in fredr::fredr_request() introduced some lag in the testing suite given some tests expect failure for bad endpoints. By default, fredr::fredr_request() will run the request up to 10 times for any failure. I will expose a retry_times argument in fredr::fredr_request(), passed directly to the times argument of internal httr::RETRY. The default will be 10 but we can reduce the number of tries in expect_error() tests for which immediate failure is desired (e.g. bad endpoints, malformed requests).

sboysel commented 4 years ago

First stab at mocking the FRED API with httptest merged to master with #78