einride / protoc-gen-go-aip-test

MIT License
8 stars 0 forks source link

protoc-gen-go-aip-test

Generate test suites for protobuf services implementing standard AIP methods.

The generated test suites are based on guidance for standard methods, and experience from implementing these methods in practice. See Suites for a list of the generated tests.

Experimental: This plugin is experimental, and breaking changes with regard to the generated tests suites should be expected.

Usage

Step 1: Declare a service with AIP standard methods

service FreightService {
  // Get a shipper.
  // See: https://google.aip.dev/131 (Standard methods: Get).
  rpc GetShipper(GetShipperRequest) returns (Shipper) {
    option (google.api.http) = {
      get: "/v1/{name=shippers/*}"
    };
    option (google.api.method_signature) = "name";
  }

  // ...
}

Step 2: Install the generator

Download a prebuilt binary from releases and put it in your PATH.

The generator can also be built from source using Go.

Step 3: Generate test suites

Include the plugin in protoc invocation

protoc
  --go-aip-test_out=[OUTPUT DIR] \
  --go-aip-test_opt=module=[OUTPUT MODULE] \
  [.proto files ...]

This can also be done via a buf generate template. See buf.gen.yaml for an example.

Step 4: Run tests

There are two alternative ways of bootstrapping the tests.

Alternative 1:

Instantiate the generated test suites and call the methods you want to test.

package example

func Test_FreightService(t *testing.T) {
    t.Skip("this is just an example, the service is not implemented.")
    // setup server before test
    server := examplefreightv1.UnimplementedFreightServiceServer{}
    // setup test suite
    suite := examplefreightv1.FreightServiceTestSuite{
        T:      t,
        Server: server,
    }

    // run tests for each resource in the service
    ctx := context.Background()
    suite.TestShipper(ctx, examplefreightv1.ShipperTestSuiteConfig{
        // Create should return a resource which is valid to create, i.e.
        // all required fields set.
        Create: func() *examplefreightv1.Shipper {
            return &examplefreightv1.Shipper{
                DisplayName:    "Example shipper",
                BillingAccount: "billingAccounts/12345",
            }
        },
        // Update should return a resource which is valid to update, i.e.
        // all required fields set.
        Update: func() *examplefreightv1.Shipper {
            return &examplefreightv1.Shipper{
                DisplayName:    "Updated example shipper",
                BillingAccount: "billingAccounts/54321",
            }
        },
    })
}

Alternative 2:

Implement the generated configure provider interface (FreightServiceTestSuiteConfigProvider) and pass the implementation to TestServices to start the tests.

A benefit of using TestServices (over alternative 1) is that as new services or resources are added to the API the test code won't compile until the required inputs are also added (or explicitly ignored). This makes it harder to forget to add the test implementations for new services/resources.

package example

import "testing"

func Test_FreightService(t *testing.T) {
    // Even though no implementation exists, the tests will pass but be skipped.
    examplefreightv1.TestServices(t, &aipTests{})
}

type aipTests struct{}

var _ examplefreightv1.FreightServiceTestSuiteConfigProvider = &aipTests{}

func (a aipTests) FreightServiceShipper(_ *testing.T) *examplefreightv1.FreightServiceShipperTestSuiteConfig {
    // Returns nil to indicate that it's not ready to be tested.
    return nil
}

func (a aipTests) FreightServiceSite(_ *testing.T) *examplefreightv1.FreightServiceSiteTestSuiteConfig {
    // Returns nil to indicate that it's not ready to be tested.
    return nil
}

Skipping tests

There may be multiple reasons for an API to deviate from the guidance for standard methods (for examples see AIP-200). This plugin supports skipping individual or groups of tests using the Skip field generated for each test suite config.

Each test are compared, using strings.Contains, against a list of skipped test patterns. The full name of each test will follow the format [resource]/[method type]/[test_name].

Sample skips:

Suites

Create

Name Description Only if
missing parent Method should fail with InvalidArgument if no parent is provided. Generated only if all are true:
  • has Create method
  • resource has a parent
invalid parent Method should fail with InvalidArgument if provided parent is invalid. Generated only if all are true:
  • has Create method
  • resource has a parent
create time Field create_time should be populated when the resource is created. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has field 'create_time'
persisted The created resource should be persisted and reachable with Get. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has Get method
user settable id If method support user settable IDs, when set the resource should be returned with the provided ID. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has user settable ID
invalid user settable id Method should fail with InvalidArgument if the user settable id doesn't conform to RFC-1034, see doc. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has user settable ID
invalid user settable id - uuid Method should fail with InvalidArgument if the user settable ID appears to be a UUID, see doc. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has user settable ID
already exists If method support user settable IDs and the same ID is reused the method should return AlreadyExists. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has user settable ID
required fields The method should fail with InvalidArgument if the resource has any required fields and they are not provided. Generated only if all are true:
  • has Create method
  • resource has any required fields
resource references The method should fail with InvalidArgument if the resource has any resource references and they are invalid. Generated only if all are true:
  • has Create method
  • resource has any mutable resource references
etag populated Field etag should be populated when the resource is created. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has field 'etag'

Get

Name Description Only if
missing name Method should fail with InvalidArgument if no name is provided. Generated only if all are true:
  • has Get method
invalid name Method should fail with InvalidArgument if the provided name is not valid. Generated only if all are true:
  • has Get method
exists Resource should be returned without errors if it exists. Generated only if all are true:
  • has Get method
not found Method should fail with NotFound if the resource does not exist. Generated only if all are true:
  • has Get method
only wildcards Method should fail with InvalidArgument if the provided name only contains wildcards ('-') Generated only if all are true:
  • has Get method
soft-deleted A soft-deleted resource should be returned without errors. Generated only if all are true:
  • has Get method
  • has Delete method
  • has field 'delete_time'

BatchGet

Name Description Only if
invalid parent Method should fail with InvalidArgument if provided parent is invalid. Generated only if all are true:
  • resource has a parent
  • has BatchGet method
  • is not alternative batch request message
names missing Method should fail with InvalidArgument if no names are provided. Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
invalid names Method should fail with InvalidArgument if a provided name is not valid. Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
wildcard name Method should fail with InvalidArgument if a provided name only contains wildcards (-) Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
all exists Resources should be returned without errors if they exist. Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
atomic The method must be atomic; it must fail for all resources or succeed for all resources (no partial success). Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
parent mismatch If a caller sets the "parent", and the parent collection in the name of any resource being retrieved does not match, the request must fail. Generated only if all are true:
  • resource has a parent
  • has BatchGet method
  • is not alternative batch request message
ordered The order of resources in the response must be the same as the names in the request. Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message
duplicate names If a caller provides duplicate names, the service should return duplicate resources. Generated only if all are true:
  • has BatchGet method
  • is not alternative batch request message

Update

Name Description Only if
missing name Method should fail with InvalidArgument if no name is provided. Generated only if all are true:
  • has Update method
invalid name Method should fail with InvalidArgument if provided name is not valid. Generated only if all are true:
  • has Update method
update time Field update_time should be updated when the resource is updated. Generated only if all are true:
  • has Create method
  • Create method does not return long-running operation
  • has Update method
  • Update method does not return long-running operation
  • has field 'update_time'
persisted The updated resource should be persisted and reachable with Get. Generated only if all are true:
  • has Update method
  • Update method does not return long-running operation
  • has Get method
preserve create_time The field create_time should be preserved when a '*'-update mask is used. Generated only if all are true:
  • has Update method
  • Update method does not return long-running operation
  • has field 'create_time'
  • resource has any required fields
etag mismatch Method should fail with Aborted if the supplied etag doesnt match the current etag value. Generated only if all are true:
  • has Update method
  • request has etag field
  • has field 'etag'
etag updated Field etag should have a new value when the resource is successfully updated. Generated only if all are true:
  • has Update method
  • request has etag field
  • has field 'etag'
not found Method should fail with NotFound if the resource does not exist. Generated only if all are true:
  • has Update method
invalid update mask The method should fail with InvalidArgument if the update_mask is invalid. Generated only if all are true:
  • has Update method
  • Update method has update_mask
required fields Method should fail with InvalidArgument if any required field is missing when called with '*' update_mask. Generated only if all are true:
  • has Update method
  • resource has any required fields

List

Name Description Only if
invalid parent Method should fail with InvalidArgument if provided parent is invalid. Generated only if all are true:
  • has List method
  • resource has a parent
invalid page token Method should fail with InvalidArgument is provided page token is not valid. Generated only if all are true:
  • has List method
negative page size Method should fail with InvalidArgument is provided page size is negative. Generated only if all are true:
  • has List method
isolation If parent is provided the method must only return resources under that parent. Generated only if all are true:
  • resource has a parent
  • has List method
  • resource has a parent
last page If there are no more resources, next_page_token should not be set. Generated only if all are true:
  • resource has a parent
  • has List method
  • resource has a parent
more pages If there are more resources, next_page_token should be set. Generated only if all are true:
  • resource has a parent
  • has List method
  • resource has a parent
one by one Listing resource one by one should eventually return all resources. Generated only if all are true:
  • resource has a parent
  • has List method
  • resource has a parent
deleted Method should not return deleted resources. Generated only if all are true:
  • resource has a parent
  • has List method
  • has Delete method
  • resource has a parent

Search

Name Description Only if
invalid parent Method should fail with InvalidArgument if provided parent is invalid. Generated only if all are true:
  • has Search method
  • resource has a parent
invalid page token Method should fail with InvalidArgument is provided page token is not valid. Generated only if all are true:
  • has Search method
negative page size Method should fail with InvalidArgument is provided page size is negative. Generated only if all are true:
  • has Search method
isolation If parent is provided the method must only return resources under that parent. Generated only if all are true:
  • resource has a parent
  • has Search method
  • resource has a parent
last page If there are no more resources, next_page_token should not be set. Generated only if all are true:
  • resource has a parent
  • has Search method
  • resource has a parent
more pages If there are more resources, next_page_token should be set. Generated only if all are true:
  • resource has a parent
  • has Search method
  • resource has a parent
one by one Searching resource one by one should eventually return all resources. Generated only if all are true:
  • resource has a parent
  • has Search method
  • resource has a parent
deleted Method should not return deleted resources. Generated only if all are true:
  • resource has a parent
  • has Search method
  • has Delete method
  • resource has a parent

Delete

Name Description Only if
missing name Method should fail with InvalidArgument if no name is provided. Generated only if all are true:
  • has Delete method
invalid name Method should fail with InvalidArgument if the provided name is not valid. Generated only if all are true:
  • has Delete method
exists Resource should be deleted without errors if it exists. Generated only if all are true:
  • has Delete method
not found Method should fail with NotFound if the resource does not exist. Generated only if all are true:
  • has Delete method
already deleted Method should fail with NotFound if the resource was already deleted. This also applies to soft-deletion. Generated only if all are true:
  • has Delete method
only wildcards Method should fail with InvalidArgument if the provided name only contains wildcards ('-') Generated only if all are true:
  • has Delete method
etag mismatch Method should fail with Aborted if the supplied etag doesnt match the current etag value. Generated only if all are true:
  • has Delete method
  • request has etag field
  • has field 'etag'