when comparing the three packages

[maelle]

• http://veekaybee.github.io/2019/02/10/lockin/

• In presser docs https://r-lib.github.io/presser/articles/introduction.html#presser-vs-mocking " Better test writing experience. This is subjective, and your mileage may vary."

[maelle]

Topics for the comparison

• What HTTP package can you use (curl, httr, crul)
• Sustainability (All 3 packages are actively maintained)
• Workflows for saving fixtures (with vcr, it's built-in as you're not really expected to edit fixtures; with the two other packages you have more flexibility/work to do)
• How to test for errors (with httptest testing for errors is the same as testing normal behavior but you'll have written the fixture yourself; with vcr it's a different thing involving webmockr; with presser it's like testing normal behavior).
• How to test for behavior in the case of a sequence of calls (e.g. 502 502 200) -- not shown in the whole games. webmockr then, presser docs links, for httptest link to an example.
• How to not share secrets. With presser you'd need to add authentication yourself so harder to leak a secret; with vcr see vcr_configure; with httptest configuration needed only if secret in the response.
• How to "turn off fake tests"

[maelle]

• Something about debugging test failures? with presser you can open your fake app in a browser / play with it with curl. that said with webmockr and httptest you have less needs for such debugging anyway.

[maelle]

Reg recording by hand https://github.com/maelle/exemplighratia/pull/3#discussion_r509861774

[maelle]

Neal Richardson's quote about mock files "Good APIs don't change that often so I tend to be less worried about "what if they change the API and I don't notice?" or maybe I weigh the tradeoffs differently. I'd personally prefer to have clearer, more expressive tests using mocks that I've massaged rather than have tests I can run unchanged against the live API but that can't assert much about the resulting behavior."

So the trade-off is

• mock files recorded automatically: easy to record, easy to re-record, less intentional testing (<- not really good phrasing)
• mock files created by hand or modified by hand: less easy to record or re-record (you could script the modifications), more expressive tests. In particular you can assert for actual values even when the API output would have changed (think of searching tweets). Also, smaller mock files (lighter to store, easier to read).

vcr has more tooling around the first possibility even a re-recording intervals. httptest serves both workflows, but with less features around re-recording. presser doesn't even allow any sort of automatic recording.

So your view on these things can help you choose a package.

[maelle]

if you use vcr when testing for errors you use something that's entirely different, webmockr.

With httptest and presser the process for faking 502 errors is much more similar.

(on the topic of error, not that sequence of responses e.g. 502 502 200 are smooth with webmockr and presser but not httptest yet)

[sckott]

Interesting on the differences, thanks @maelle

[maelle]

reg debugging. how does one run code in the tests interactively whilst using the recorded response? I'm not too sure how I'd achieve that with vcr/httptest. With presser one would "just" start the app before running the code.

[maelle]

with httptest for debugging you'd use https://enpiar.com/r/httptest/reference/use_mock_api.html I suppose

@sckott how would one run code outside of tests, while still using a given cassette?

[maelle]

another point of interest, can you POST data and GET it.

• with presser yes, documented
• with webmockr how would you do that @sckott?
• with httptest cf what's done for vignettes, maybe other examples.

[sckott]

how does one run code in the tests interactively whilst using the recorded response?

I do this by running the vcr_configuration() code block in the test helper file, and just change the paths to fixtures/files to reflect the directory you're in, e.,g.,

if this is what's in helper-pkg.R

vcr::vcr_configure(dir = "../fixtures", write_disk_path = "../files")

and if you're current directory is the root of the package and tests are are in tests/, then run

vcr::vcr_configure(dir = "tests/fixtures", write_disk_path = "tests/files")

and then when you run http requests wrapped in use_cassette you'll use the recorded fixtures

[sckott]

answer about post then get in slack

[maelle]

Oooh could one improve this by

• switching back to helper not setup. Helper is no longer recommended but still supported and loaded by load_all(). Maybe not worth the switch though.
• have tests live in testthat/fixtures instead this way one would use testthat::test_path() to refer to the path which works in interactive mode too. Or some equivalent function. This way no need to fix the configuration 🤔

[sckott]

have tests live in testthat/fixtures

I don't like this idea.

We can keep thinking though

[maelle]

Yeah one would just need a function that returns the right path to tests/fixtures no matter where it is called from (from tests/testthat or from the root)

[sckott]

Right