add the notion of "contract test"? + turn off fake tests

ropensci-books / http-testing

HTTP testing for R

https://books.ropensci.org/http-testing/

Creative Commons Zero v1.0 Universal

53 stars 17 forks source link

add the notion of "contract test"? + turn off fake tests #43

Closed maelle closed 3 years ago

maelle commented 4 years ago

h/t https://www.martinfowler.com/bliki/ContractTest.html

sckott commented 4 years ago

nice. one way maybe one could achieve the goal here is to run tests that use vcr, but turning vcr off with those env vars so that the same tests that use fixtures do real HTTP requests instead. but maybe there is some other approach that would be better, or at least better for some use cases

maelle commented 4 years ago

yes I think that's the simple contract test.

I wonder whether with vcr, there could be some sort of tool for more easily exploring the difference between new and previous versions of cassettes when one re-records (similar to what one gets with testthat's snapshot testing).

sckott commented 3 years ago

Possibly. Probably only if they go through a vcr function that we make so we can move the old cassette to a temporary folder, then after creating the new one, be able to compare the two.

maelle commented 3 years ago

notes about httptest and presser. if you write your mock files / response content by hand, not that easy to re-record/see if the API output changed. :thinking:

maelle commented 3 years ago

with vcr turn vcr off to have real interactions
with presser&httptest either have a logic with an environment variable, or why not an entirely different test folder e.g. inst/real-tests (if the tests using mocks/presser-stuff are too specific)