earldouglas / swagger-test

Specification-driven REST API testing
MIT License
113 stars 19 forks source link

Setup and teardown fixtures for the tests #4

Open wking opened 9 years ago

wking commented 9 years ago

The example x-amples usage requests and receives a pet with ID fido4. How do you know that ID exists in the tested implementation? It seems like you'd need a setup block where you could push data into the tested implementation, and a teardown block where you could remove any data that might have been left by the test. For systems that autogenerate IDs after POSTs, preserving those IDs between requests would take some state preservation on the swagger-test side, and I'm not sure what that would look like in the x-amples definition.

earldouglas commented 9 years ago

How do you know that ID exists in the tested implementation?

That's a great point. As currently written, the example specifies that there must be a fido4. This is probably unrealistic for the reasons you point out, but it's still a valid specification; the imaginary company publishing this imaginary specification is asserting that fido4 is expected to exist.

In other incarnations of this, experiments have been tested with support for "prerequisites" -- requests that are sent before the x-ample request/response is handled. There are several options for how to address this problem, so we'll want to think about how to do it in the simplest way for the most users.

Trenrod commented 7 years ago

Hi, i came across the same requirement. At the moment I only have only a proof of concept. The requirement in my case was. To create a dynamic map/cache during test cases execution to reuse a response value in another test case.

The below example is far from perfect. But if its a worth a discussion I can think about an integration to this project and create a PR.

First I needed to create a fixed execution order. Also used create more complex test szenarios later. Therefore create an array with the description of the testcase as reference.

I changed the parseXample function to store the raw http parts as 'httpReq' to recreate a preq object later.

  function parseXample(spec, uri, method, xample) {
      var uriTemplate = template.parse(uri);
      var expandedUri = uriTemplate.expand(xample.request.params);
      xample.request.method = method;
      xample.request.uri = spec.host + spec.basePath + expandedUri;

      var httpReq = {};
      httpReq = {};
      httpReq.paths = {};
      httpReq.paths[uri] = {};
      httpReq.paths[uri][method] = {};
      httpReq.paths[uri][method]['x-amples'] = [xample];
      httpReq.host = spec.host;
      httpReq.basePath = spec.basePath;

      return {
          description: xample.description || method + ' ' + uri,
          request: xample.request,
          responses: xample.responses,
          httpReq: httpReq
      };
  }

To be able to parse the test case like : swaggerTest.parse(parseStoreRestore(xample.test.httpReq));

To be able to store and restore object e.g. the objectid of the created account. I added "$store" in the response and "$restore_" in the request x-amples doc Where $store is resolved right before validating


 *         responses:
 *           status: 200
 *           $store: ROOT_TRADER1_DOCUMENT
  // Check to store data
  let storeResTo = null;
  if (request.test.responses["$store"]) {
    storeResTo = request.test.responses["$store"];
    delete request.test.responses["$store"];
  }
  // Check
  assert.shallowDeepEqual(resp, request.test.responses, "Deep shallow compare of response failed.");
  if (storeResTo) {
    // If exists merge
    if (testRefMapping[storeResTo]) {
      let org = JSON.parse(JSON.stringify(testRefMapping[storeResTo]));
      testRefMapping[storeResTo] = Object.assign(org, resp.body.data);
    } else {
      testRefMapping[storeResTo] = resp.body.data;
    }
    console.log("Stored", testRefMapping[storeResTo]);
  }

And restore is replaced with right before generating a preq object.

 *           params:
 *             traderid: $restore_ROOT_TRADER1_DOCUMENT.trader._id

I used "object-resolve-path" to be able to reference specific values from an object