Automatic API integration testing.
$ npm install -D test-openapi
$ test-openapi
If a task failed, exit code will be 1
.
Options are passed as CLI flags.
$ test-openapi --merge.spec.definition openapi_schema.yml
Tasks are passed as positional argument.
$ test-openapi **/*.tasks.yml
const { run } = require('test-openapi')
const promise = run(options)
If a task failed, run()
will reject the promise with a TestOpenApiError
.
Options are passed as arguments. Tasks are passed as a tasks
argument.
const promise = run({
tasks: ['**/*.tasks.yml'],
merge: { spec: { definition: 'openapi_schema.yml' } },
})
Tasks are specified in YAML or JSON files.
By default tasks at **/*.tasks.yml|json
will be used.
Each task file contains an array of tasks definitions.
A single task performs the following:
call
property.validate
property.Each task must specify a name
unique within its file.
- name: exampleTask
call:
method: GET
server: http://localhost:8081
path: /tags
query.onlyPublic: false
validate:
status: 200
body:
type: array
# Each tag object
items:
type: object
required: [tag, isPublic]
properties:
tag:
type: string
maxLength: 32
isPublic:
type: boolean
# And so on
- name: anotherTask
This task calls:
GET http://localhost:8081/icoTagNames?onlyPublic=false
It then validates that:
200
{ tag: string, isPublic: boolean }
This screenshot shows a typical task run with few task failures.
The failed task is called getLists.success
and performs the following HTTP
request:
GET http://127.0.0.1:8081/lists?accessToken=8ac7e235-3ad2-4b9a-8a22
It expects a status code of 200 but receives 500 instead.
Other tasks are shown failing at the end. A final summary is also present.
HTTP requests are specified with the call
task property.
- name: taskName
call:
method: PUT
server: https://localhost:8081
path: /tags/:tagName
url.tagName: exampleTagName
query.accessToken: 1e42f0e1
headers.content-type: application/json
body:
_id: 1
name: exampleTagName
color: red
https:
rejectUnauthorized: false
method
{string}
(default: GET
): HTTP methodserver
{string}
:
http://
HOST
or (if absent) localhost
PORT
(if present)path
{string}
: URL's pathurl.NAME
{any}
:
server
or path
using the :NAME
notationpath
is /tags/:tagName
it can be url.tagName
:NAME
: required parameter:NAME?
: optional parameter:NAME*
: several optional parameters:NAME+
: several required parametersquery.NAME
{any}
:
&=
to use NAME
several timesquery.name: "a&=b&=c"
becomes the query variables
?name=a&name=b&name=c
headers.NAME
{any}
:
headers.content-type
defaults to:application/json
if body
is an object or an arrayapplication/octet-stream
otherwisebody
{any}
: request bodyhttps
{object}
:
ca
, cert
, ciphers
, clientCertEngine
, crl
, dhparam
,
ecdhCurve
, honorCipherOrder
, key
, passphrase
, pfx
,
rejectUnauthorized
, secureOptions
, secureProtocol
, servername
,
sessionIdContext
.url.NAME
, query.NAME
, headers.NAME
and body
can be either a string or
any other JSON type:
Content-Type
multipart/form-data
and x-www-form-urlencoded
are not supported yet.The HTTP response is validated against the validate
task property.
- name: taskName
validate:
status: 201
headers.content-type: application/json
body:
type: array
status
{string|integer}
:
201
1xx
, 2xx
, 3xx
, 4xx
or 5xx
201 202 3xx
2xx
headers.NAME
{any|jsonSchema}
:
NAME
is case-insensitivex-optional
{boolean}
(default: true
): if false
, validate that
the HTTP header is present in the responsex-forbidden
{boolean}
(default: false
): if true
, validate that
the HTTP header is not present in the responsebody
{any|jsonSchema}
:
headers.NAME
)Validation can also vary according to the response's status code by using the following notation.
- name: taskName
validate:
201:
body:
type: array
400:
body:
type: object
The call
and validate
tasks properties can be pre-filled if you have
described your API endpoints with OpenAPI.
- name: taskName
spec:
operation: getTags
definition: ../openapi_document.yml
operation
{string}
: OpenAPI's
operationId
definition
{string}
:
merge
task property can be usedThe following OpenAPI properties are currently used:
consumes
OpenAPI property
sets the request Content-Type
header (call['headers.content-type']
)produces
OpenAPI property
sets the request Accept
header (call['headers.accept']
) and validate the
response's Content-Type
header (validate['headers.content-type']
)host
and basePath
OpenAPI properties
set the call.server
task property. At the moment the protocol is always
http://
.call.method
and call.path
is taken from the
OpenAPI definitionparameters
OpenAPI property:
required
will only be used (and merged)
if they are explicitly present in the call
task propertycall
task property:valid
: re-use the OpenAPI parameter definition. Useful if the OpenAPI
parameter is not marked as required
. Redundant otherwise.undefined
: do not use the OpenAPI parameter definitionschema
and headers
OpenAPI properties are used to validate the HTTP
response (validate.status|body|headers
)OpenAPI schemas can use the following extensions:
schema.x-nullable|oneOf|anyOf|not
: behaves like OpenAPI 3.0
nullable|oneOf|anyOf|not
schema.x-additionalItems|dependencies
: behaves like JSON schemas
additionalItems|dependencies
To specify properties shared by all tasks, use the merge
option:
$ test-openapi --merge.spec.definition ../openapi_document.yml
To specify properties shared by a few tasks, create a task with a merge
property.
- name: sharedTask
merge: invalidCheck/.*
validate:
status: 400
The merge
property should be a regular expression (or an array of them)
targeting other tasks by name
. The shared task will not be run. Instead it
will be deeply merged to the target tasks.
The target tasks can override the shared task by using undefined
inside task
properties.
Template variables can be used using the $$name
notation.
Template variables are specified using the template
task property.
- name: exampleTask
template:
$$exampleVariable: true
call:
query.isPublic: $$exampleVariable
The example above will be compiled to:
- name: exampleTask
call:
query.isPublic: true
Template variables can:
$$exampleVariable --- $$anotherVariable
$$exampleArray[0].propertyName
{ $$exampleFunction: [firstArg, secondArg] }
The following template variables are always available:
$$env
: use environment variables (case-sensitive)$$random
: generate fake data using a
JSON schema version 4$$faker
: generate fake data using
Faker.js- name: exampleTask
call:
server: $$env.SERVER
query.password:
$$random:
type: string
minLength: 12
pattern: '[a-zA-Z0-9]'
body:
name: $$faker.name.firstName
A request can save its response using variables
. Other requests will be able
to re-use it as template variables. This creates sequences of requests.
- name: createAccessToken
variables:
$$accessToken: call.response.body.accessToken
- name: taskName
call:
query.accessToken: $$accessToken
The call.request
and call.response
are available to re-use the HTTP request
and response.
The task will fail if the variable is undefined
unless you append the word
optional
to its value.
- name: createAccessToken
variables:
$$accessToken: call.response.body.accessToken optional
By default all tasks are run in parallel at the same time.
To only run a few tasks use the only
option.
$ test-openapi --only 'taskNameRegularExpression/.*'
Or the only
task property.
- name: taskName
only: true
The skip
option and task property can be used to do the opposite.
The following reporters are available:
pretty
: default reportertap
: Test Anything Protocolnotify
: desktop notificationdata
: JSON outputSpecify the --report.REPORTER
option to select which reporter to use
$ test-openapi --report.notify --report.pretty
Use the --report.REPORTER.output
to redirect the output of a reporter to a
file:
$ test-openapi --report.pretty.output path/to/file.txt
Use the --report.REPORTER.level
to modify the verbosity:
$ test-openapi --report.pretty.level info
The available levels are:
silent
error
warn
(default for pretty
)info
(default for tap
and notify
)debug
(default for data
)With the repeat.data
task property, tasks are repeated by iterating over an
array of inputs.
- name: exampleTask
repeat:
data:
- name: apples
quantity: 1
- name: oranges
quantity: 10
- name: plums
quantity: 100
call:
method: GET
path: /fruits/:fruitName
url.fruitName: $$data.name
query:
quantity: $$data.quantity
The task above will be run three times: GET /fruits/apples?quantity=1
,
GET /fruits/oranges?quantity=10
and GET /fruits/plums?quantity=100
.
With the repeat.times
task property, tasks are simply repeated as is. This can
be useful when used with the $$random
template function.
- name: exampleTask
repeat:
times: 10
repeat.data
and repeat.times
can be combined.