discuss tradeoffs for mocking

[maxheld83]

🙏 so much for pulling all of this together for testing API wrapper packages -- this is a godsend.

I randomly came across arguments against mocking (or rather, there's tradeoffs involved) and some of them struck a chord:

• https://medium.com/javascript-scene/mocking-is-a-code-smell-944a70c90a6a
• https://www.solutionsiq.com/resource/blog-post/to-mock-or-not-to-mock-is-that-even-a-question/

My summary would be:

1. there's a cost in complexity to mocking
2. some pkgs need more, other need less (none) mocking, roughly depending on how central I/O is to test coverage.
3. the need for mocking can mask a lack of uncoupling in the overall design.

I'd be happy to write this up (worded very carefully) in a 2-3 paragraphs and add a PR if you're interested in featuring this angle at all.

[maelle]

Thank you!

What would be the alternative to mocking in practice for an API package? We e.g. have https://books.ropensci.org/http-testing/pkgs-comparison.html but with webfakes you might actually write more complex code than with vcr/httptest unless you can use the built-in httpbin app.

The tradeoff discussion would need some examples, I think. Do you think it'd work as a new advanced topic chapter? (those can be very short!).

The item 3 is very important, it could also somehow be added to the intro or conclusion. :thinking:

@sckott what do you think?

[maxheld83]

What would be the alternative to mocking in practice for an API package?

If I understand these "critiques" (if that's what they are?) correctly, I think the alternative to mocking in API packages would be neither mocking nor faking, but uncoupling the API calls as much as possible from the rest of the pkg, and then testing those few functions against the live API (with the limitations that entails).

[sckott]

To what extent do those two blog posts include mocking http requests specifically vs. test mocks in general? Does it matter?

We used to have a short chapter discussing some tradeoffs https://github.com/ropensci-books/http-testing/blob/65c84a111aea43c894306199c2468317b8048819/17-testing-considerations.Rmd

Maybe it would go in this chapter https://books.ropensci.org/http-testing/pkgs-comparison.html that compares the various tools?

I think mocking http requests has its important use cases. e.g,. you often can't get a server to return a specific error response (e..g, a 503 with a particular response body) -

[maxheld83]

To what extent do those two blog posts include mocking http requests specifically vs. test mocks in general? Does it matter?

They mostly cover mocks in general, and mostly in the context of OO I think. This actually recommends mocking for integration tests of web services.

I think mocking http requests has its important use cases. e.g,. you often can't get a server to return a specific error response (e..g, a 503 with a particular response body)

🙏

What I take away from these pieces and experiences is that, mocking, as all other test code, should guide you to write uncoupled, composable code. If you need a lot of hairy fixtures, tests or mocks, something may be amiss.

I'd be happy to write a PR to that effect.

But I'll close this for now as I fear I'm creating noise here. Apologies.

[sckott]

This actually recommends mocking for integration tests of web services.

Thx for pointing that out, i didn't read both posts before I commented.

I'm still interested in a PR. I think most people will benefit from more help on deciding when to use mocking. Thoughts @maelle ?

[maelle]

Definitely, it does not sound like noise to me! I was just wondering about where to put it and my request of examples was a genuine question. 🙂