Testing External Services

[osahyoun]

@SumOfUs/developers

TL; DR Tests for external services are brittle and slow. Instead, use http mocking libraries like webmock (with VCR).

---

In champaign we make an HTTP POST request to AWS when we push JSON to our SQS queue. The HTTP API has been brilliantly abstracted with AWS' aws-sdk gem. Making a correctly signed HTTP POST request is as simple as:

Aws::SQS::Client.new.send_message({
          queue_url:    'http://my-aws-queue.com',
          message_body: { data: 'for', my: 'queue' }
        })

Presently, this is how it's being spec'd:

expect_any_instance_of(Aws::SQS::Client).to(
  receive(:send_message).with( expected_arguments )
)

ChampaignQueue::Clients::Sqs.push({foo: :bar})

ChampaignQueue::Clients::Sqs.push is just a wrapper around AWS SDK gem, which shields us from changes to their gem.

The spec is setting a message expectation (:send_message) on any instance of Aws::SQS::Client. When we call our wrapper method ChampaignQueue::Clients::Sqs.push the expectation is met.

The expectation is also a stub so :send_message is blocked from being called on the gem, so no HTTP request is ever made. However, if the AWS gem should change, our spec will NOT break. But this is unlikely to happen due to the stability of Aws::SQS::Client.

If I hadn't been feeling so lazy, and confident about amazon's SDK, I would have have mocked the http request...

Mocking External HTTP Requests

This is the standard way of testing integration with external services across languages. With Ruby there are a couple of libraries that make http mocking trivial. Webmock is my favourite:

https://github.com/bblimke/webmock

Library for stubbing and setting expectations on HTTP requests in Ruby

With webmock we can set our expectations at a much lower level, making our spec far more valuable, but just as fast and resilient. If there are changes to AWS's SDK our spec will catch them.

Here's my updated version of this spec using Webmock: https://github.com/SumOfUs/Champaign/pull/113

VCR - Recording Real Requests

For pushing to SQS we don't really care too much about the API's response. Often though, we care very much about the data we get back. @NealJMD, I imagine this will be the case when testing your API wrapper for shareprogress.

VCR https://github.com/vcr/vcr sits on top of webmock. When the spec is first run, it'll allow a real request to be made. It then stores the response in a yaml file, and sets the response as fixture data. Any subsequent spec runs are then blocked, with a message expectation set on the http request. It'll then respond with the fixture data for your specs to use as necessary. Occasionally running VCR in live mode means our requests remain up-to-date with the live service.

@Tuuleh my concern with fake_sqs was that it meant we were testing against a different service. Can we trust the gem's author will keep it up to date with the real thing. Also, the spec is more powerful if we can test the detail of HTTP request being sent.

Apologies if this is all very apparent and obvious. It certainly wasn't for me when I first started testing API service integrations. This is a post I wish had been sent to me a few years back.

[osahyoun]

There's a lot written on this subject. Here's a fairly good post by thoughtbot: https://robots.thoughtbot.com/how-to-stub-external-services-in-tests

[Tuuleh]

For SQS, there were already tests running locally, using the fake_sqs gem. What's the benefit of mocking external HTTP requests vs. using the local queue?

On Thu, Aug 20, 2015 at 4:23 AM, Omar Sahyoun notifications@github.com wrote:

There's a lot written on this subject. Here's a fairly good post by thoughtbot: https://robots.thoughtbot.com/how-to-stub-external-services-in-tests

— Reply to this email directly or view it on GitHub https://github.com/SumOfUs/sumofus/issues/14#issuecomment-132979648.

[osahyoun]

fake_sqs is a black box, and we can't know how how well it follows AWS' SQS api. If it was maintained by AWS I'd probably have more confidence in it.

Short of actually testing against a working SQS queue, mocking an external HTTP request is the closest we can get to testing our integration with their SQS service (or any other external service we might chose to integrate with).

If our spec can validate that we're making a signed HTTP Post request as per their API documentation, then we'll here about it as soon as their API spec changes. But also, without fake_sqs we're no longer dependent on another service for our specs to run - that means faster, less brittle specs.

[osahyoun]

https://github.com/SumOfUs/sumofus/wiki/Testing-External-Services