emanualjade / test-openapi

Automatic API integration testing
MIT License
14 stars 4 forks source link

Travis Node Gitter

Automatic API integration testing.

Features

Installation

$ npm install -D test-openapi

Usage (shell)

$ test-openapi

If a task failed, exit code will be 1.

Options are passed as CLI flags.

$ test-openapi --merge.spec.definition openapi_schema.yml

Tasks are passed as positional argument.

$ test-openapi **/*.tasks.yml

Usage (Node.js)

const { run } = require('test-openapi')

const promise = run(options)

If a task failed, run() will reject the promise with a TestOpenApiError.

Options are passed as arguments. Tasks are passed as a tasks argument.

const promise = run({
  tasks: ['**/*.tasks.yml'],
  merge: { spec: { definition: 'openapi_schema.yml' } },
})

Tasks

Tasks are specified in YAML or JSON files.

By default tasks at **/*.tasks.yml|json will be used.

Each task file contains an array of tasks definitions.

A single task performs the following:

Each task must specify a name unique within its file.

Example input

- name: exampleTask
  call:
    method: GET
    server: http://localhost:8081
    path: /tags
    query.onlyPublic: false
  validate:
    status: 200
    body:
      type: array
      # Each tag object
      items:
        type: object
        required: [tag, isPublic]
        properties:
          tag:
            type: string
            maxLength: 32
          isPublic:
            type: boolean

# And so on
- name: anotherTask

This task calls:

GET http://localhost:8081/icoTagNames?onlyPublic=false

It then validates that:

Example output

This screenshot shows a typical task run with few task failures.

Screenshot

The failed task is called getLists.success and performs the following HTTP request:

GET http://127.0.0.1:8081/lists?accessToken=8ac7e235-3ad2-4b9a-8a22

It expects a status code of 200 but receives 500 instead.

Other tasks are shown failing at the end. A final summary is also present.

HTTP requests

HTTP requests are specified with the call task property.

- name: taskName
  call:
    method: PUT
    server: https://localhost:8081
    path: /tags/:tagName
    url.tagName: exampleTagName
    query.accessToken: 1e42f0e1
    headers.content-type: application/json
    body:
      _id: 1
      name: exampleTagName
      color: red
    https:
      rejectUnauthorized: false

url.NAME, query.NAME, headers.NAME and body can be either a string or any other JSON type:

Response validation

The HTTP response is validated against the validate task property.

- name: taskName
  validate:
    status: 201
    headers.content-type: application/json
    body:
      type: array

Validation can also vary according to the response's status code by using the following notation.

- name: taskName
  validate:
    201:
      body:
        type: array
    400:
      body:
        type: object

OpenAPI

The call and validate tasks properties can be pre-filled if you have described your API endpoints with OpenAPI.

- name: taskName
  spec:
    operation: getTags
    definition: ../openapi_document.yml

The following OpenAPI properties are currently used:

OpenAPI schemas can use the following extensions:

Shared properties

To specify properties shared by all tasks, use the merge option:

$ test-openapi --merge.spec.definition ../openapi_document.yml

To specify properties shared by a few tasks, create a task with a merge property.

- name: sharedTask
  merge: invalidCheck/.*
  validate:
    status: 400

The merge property should be a regular expression (or an array of them) targeting other tasks by name. The shared task will not be run. Instead it will be deeply merged to the target tasks.

The target tasks can override the shared task by using undefined inside task properties.

Template variables

Template variables can be used using the $$name notation.

Template variables are specified using the template task property.

- name: exampleTask
  template:
    $$exampleVariable: true
  call:
    query.isPublic: $$exampleVariable

The example above will be compiled to:

- name: exampleTask
  call:
    query.isPublic: true

Template variables can:

The following template variables are always available:

- name: exampleTask
  call:
    server: $$env.SERVER
    query.password:
      $$random:
        type: string
        minLength: 12
        pattern: '[a-zA-Z0-9]'
    body:
      name: $$faker.name.firstName

Sequences of requests

A request can save its response using variables. Other requests will be able to re-use it as template variables. This creates sequences of requests.

- name: createAccessToken
  variables:
    $$accessToken: call.response.body.accessToken

- name: taskName
  call:
    query.accessToken: $$accessToken

The call.request and call.response are available to re-use the HTTP request and response.

The task will fail if the variable is undefined unless you append the word optional to its value.

- name: createAccessToken
  variables:
    $$accessToken: call.response.body.accessToken optional

Tasks selection

By default all tasks are run in parallel at the same time.

To only run a few tasks use the only option.

$ test-openapi --only 'taskNameRegularExpression/.*'

Or the only task property.

- name: taskName
  only: true

The skip option and task property can be used to do the opposite.

Reporting

The following reporters are available:

Specify the --report.REPORTER option to select which reporter to use

$ test-openapi --report.notify --report.pretty

Use the --report.REPORTER.output to redirect the output of a reporter to a file:

$ test-openapi --report.pretty.output path/to/file.txt

Use the --report.REPORTER.level to modify the verbosity:

$ test-openapi --report.pretty.level info

The available levels are:

Data-driven testing

With the repeat.data task property, tasks are repeated by iterating over an array of inputs.

- name: exampleTask
  repeat:
    data:
      - name: apples
        quantity: 1
      - name: oranges
        quantity: 10
      - name: plums
        quantity: 100
  call:
    method: GET
    path: /fruits/:fruitName
    url.fruitName: $$data.name
    query:
      quantity: $$data.quantity

The task above will be run three times: GET /fruits/apples?quantity=1, GET /fruits/oranges?quantity=10 and GET /fruits/plums?quantity=100.

With the repeat.times task property, tasks are simply repeated as is. This can be useful when used with the $$random template function.

- name: exampleTask
  repeat:
    times: 10

repeat.data and repeat.times can be combined.