A clean, middleware-based mocking interface

[indrora]

The current mocking mechanism isn't so much a mocking mechanism as it is "use this language feature to do the work".

Currently, mocking is achieved by

• Adding an interface that covers the specific use cases of the client that you use
• Designing your calls such that any interaction with the service is a function call that takes that interface
• Possibly creating a singleton that produces either the concrete or mocked versions of the client, depending on context
• implementing the logic of the call so that you can return the appropriate response
• making sure your errors match the errors that the real service would respond with
• additionally requiring unit tests of the mocked calls, to verify that your re-implementation of the service is correct
• doing this for every service that you use, possibly multiple times depending on the usage of the service

Bringing a codebase that was not designed for this up to this level is tiresome and may not be available in some cases (such as when doing integration testing in a Kubernetes cluster based application).

Describe the solution you'd like

A clean, straightforward mechanism to provide a mocked API: Provide some JSON that represents responses, those become the "read" responses.

Botocore has this with the Stubber API: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/stubber.html

This means that in boto, you can do the following:

import datetime
import botocore.session
from botocore.stub import Stubber

s3 = botocore.session.get_session().create_client('s3')
stubber = Stubber(s3)

response = {
    'IsTruncated': False,
    'Name': 'test-bucket',
    'MaxKeys': 1000, 'Prefix': '',
    'Contents': [{
        'Key': 'test.txt',
        'ETag': '"abc123"',
        'StorageClass': 'STANDARD',
        'LastModified': datetime.datetime(2016, 1, 20, 22, 9),
        'Owner': {'ID': 'abc123', 'DisplayName': 'myname'},
        'Size': 14814
    }],
    'EncodingType': 'url',
    'ResponseMetadata': {
        'RequestId': 'abc123',
        'HTTPStatusCode': 200,
        'HostId': 'abc123'
    },
    'Marker': ''
}

expected_params = {'Bucket': 'test-bucket'}

stubber.add_response('list_objects', response, expected_params)
stubber.activate()

service_response = s3.list_objects(Bucket='test-bucket')
assert service_response == response

Ideally, the Go implementation would look something like this:

// ...

mockMiddleware := mocking.NewMocker()
mockMiddleware.AddResponse("s3", "getBucket", `{"name":"test-bucket"}`, 
   ` {
    'IsTruncated': False,
    'Name': 'test-bucket',
    'MaxKeys': 1000, 'Prefix': '',
    'Contents': [{
        'Key': 'test.txt',
        'ETag': '"abc123"',
        'StorageClass': 'STANDARD',
        'LastModified': '2021-01-02 12:34:56Z',
        'Owner': {'ID': 'abc123', 'DisplayName': 'myname'},
        'Size': 14814
    }],
    'EncodingType': 'url',
    'ResponseMetadata': {
        'RequestId': 'abc123',
        'HTTPStatusCode': 200,
        'HostId': 'abc123'
    },
    'Marker': ''
}`)

cfg := config.WithMocks(mockMiddleware)

This, additionally, allows for some services to be fully mocked, such as Secrets Manager, S3, and others (which are functionally key-value stores with bells and whistles) to have more complex functionality.

Something between the Stubber interface and the HTTP client replacement would be ideal, where I can glue responses to requests.

Boto3 has a community-supported Moto project: https://github.com/spulec/moto -- one that even includes a testing server for network operations. Folks looking for a mocking interface for boto often find Moto through stackoverflow.

Digging through Google and other repo searches, it seems this isn't the first time this issue has come up:

• https://github.com/aws/aws-sdk-go-v2/issues/70
• https://github.com/aws/aws-sdk-go-v2/issues/786
• https://stackoverflow.com/questions/50377674/how-do-i-write-unit-test-aws-sdk-go-v2-dynamodb-implementation
• https://github.com/aws/aws-sdk-go-v2/issues/116

Obviously, this indicates that more straightforward mocking interfaces have been a topic of desire.

Describe alternatives you've considered

• Removing discussion of mocking entirely from the guide -- Doable, admission of failure. The page itself feels like a PM wanted it there "because mocking is important" and then it just got written.
• Rewriting the mocking page as a serving suggestion -- Doesn't help the problem.
• slotting in mock versions of the API at build time -- Possible, but not scalable at all (needs build tags, has lots of sharp corners)

[indrora]

Updated the ticket to have more context.

[vudh1]

I have discussed with the team and we decided that we would not consider implementing this feature. However, we will create a backlog item for alternate solution like another second documentation that show how customer can use the middleware to do mocking.

[andreip]

Thank you! is https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/gov2/testtools/docs.go the outcome of that @vudh1 ? Is using middleware a good approach or would you recommend to use a DIY interface approach to make it easier to migrate to a future SDK V3?