API-scenario
Scenario API testing from the command line.
API-scenario is a simple command line tool that allow you to execute easily a scenario to test your APIs.
It is perfect to make end to end tests, you could use in your CI workflow to validate your deployment or just to test locally your development.
Why this project?
Our goal is to have a simple command line tool that can run scenario to test your apis directly from the command line. You can create a scenario and start using it during development, but also use it in your CI/CD to validate your deployment.
You can use variables in your scenario, so you are sure to run the same test suite on each environment and be sure of your release.
Installation
All the binaries are available in the release.
Install using Homebrew (mac and linux)
If you are using Homebrew package manager, you can install api-scenario with Homebrew.
brew tap thomaspoignant/homebrew-tap
brew install api-scenarioInstall using Scoop (windows)
If you are on Windows and using scoop package manager, you can install api-scenario with Scoop.
scoop bucket add org https://github.com/thomaspoignant/scoop.git
scoop install api-scenarioInstall using .deb file (linux)
If you prefer to use directly the .deb file to install in your debian like linux distribution.
Don't forget to set the correct version number.
wget https://github.com/thomaspoignant/api-scenario/releases/download/vX.X.X/api-scenario_X.X.X_Tux_64-bit.deb && dpkg -i api-scenario_*.debInstall using .rpm file (linux)
If you prefer to use directly the .rpm file to install in your centos/fedora like linux distribution.
Don't forget to set the correct version number.
wget https://github.com/thomaspoignant/api-scenario/releases/download/vX.X.X/api-scenario_X.X.X_Tux_64-bit.rpm && rpm -i api-scenario_*.rpmUse docker image
To use the last version of our docker image you can pull thomaspoignant/api-scenario:latest.
docker pull thomaspoignant/api-scenario:latestYou can also pull all the version directly but also directly a major version.
Available images are:
docker pull thomaspoignant/api-scenario:latest(target the latest version)docker pull thomaspoignant/api-scenario:vX(target the major version)docker pull thomaspoignant/api-scenario:vX.X(target the minor version)docker pull thomaspoignant/api-scenario:vX.X.X(target a specific version)
See docker hub page for usage.
Command line usage
Version
api-scenario version
# 0.1.0Execute your scenario
To execute your scenario please use the run options and specify your scenario file.
api-scenario run --scenario="./scenario.json"There are several options you can use with this command:
| Option | Short version | Required | Description |
|---|---|---|---|
--scenario |
-s |
✓ | Input file for the scenario. |
--authorization-token |
-t |
Authorization token send in the Authorization headers. | |
--header |
-h |
Header you want to override (format should be "header_name:value"). You can have multiple values of this options |
|
--variable |
-h |
Value for a variable used in your scenario (format should be "variable_name:value"). You can have multiple values of this options |
|
--verbose |
-s |
Run your scenario with debug information. | |
--quiet |
-s |
Run your scenario in quiet mode. | |
--no-color |
Do not display color on the output. | ||
--output-file |
-f |
Output file where to save the result (use --output-format to specify if you want JSON or YAML output). |
|
--output-format |
Format of the output file, available values are JSON and YAML (ignored if --output-file is not set, default value is JSON). |
Save result into file
To keep history of your scenario execution you can export the results into a file.
You just have to add the option --output-file="<your file location>" and it will save the result into a JSON file
(If you prefer YAML result add --output-format=YAML).
Example:
api-scenario run --scenario="./scenario.json" --output-file="<your file location>" --output-format=YAMLCreating Your First Test
Creating a test is simple, you just have to write json or yaml file to describe you api calls, and describe assertions.
The basic structure of the file
YAML
```yaml name: Simple API Test Example description: A full description ... version: '1.0' steps: - ... ```JSON
```json { "name": "Simple API Test Example", "description": "A full description of your test scenario", "version": "1.0", "steps": [ ... ] } ```This global fields allow to describe your scenario:
- name: The name of your scenario
- description: A complete description of what your scenario is doing
- version: The version of your scenario
- steps: Array of steps, it will describe all the steps of your scenario (see steps for more details).
Our first step
For our first step we will create a basic call who verify that an API answer with http code 200 when calling it.
YAML
```yaml - step_type: request url: {{baseUrl}}/api/users method: GET headers: Content-Type: - application/json assertions: - comparison: equal_number value: '200' source: response_status ```JSON
```json { "step_type": "request", "url": "{{baseUrl}}/api/users", "method": "GET", "headers": { "Content-Type": ["application/json"] }, "assertions": [ { "comparison": "equal_number", "value": "200", "source": "response_status" } ] } ```We manipulate different concepts here.
- step_type: The type of the step, here we are using request because we want to test a rest API (see steps to see the list of available step types).
- url: The URL of our request,
{{baseUrl}}will be replaced before the call (see Using Variables in Requests for details) - method: The HTTP verb of our request.
- headers: The list of headers we sent with the request.
- assertions: This is the list of checks we are doing when we have received the response.
- comparison: The type of check we are doing, here we are testing that number are equals.
- value: The expected value
- source: On what part of the response we are looking, here we are checking the response status.
Now the first scenario is build, we can run it (see complete scenario: YAML / JSON).
api-scenario run --scenario="examples/first-test.json" --variable="baseUrl:https://reqres.in"What we are doing is here is running our scenario file, and we ask to replace every occurrence
of {{baseUrl}} by https://reqres.in.
Steps
There are different types of steps who allow performs different types of actions.
To specify the type of a step we are using the property step_type is the step object.
_If there is no steptype property we ignore the step.
Pause
pause is simple, it is a step that wait X seconds.
This is useful when you have asynchronous API and allows waiting before calling the next API in the scenario.
| Parameters | Description |
|---|---|
| step_type | pause |
| duration | Number of seconds to wait. |
| skipped | If true the step is skipped and nothing is running. |
Example: Wait for 5 seconds
YAML
```yaml - step_type: pause duration: 5 ```JSON
```json { "step_type": "pause", "duration": 5 } ```Request
request is the step who can call a REST Api.
| Parameters | Description |
|---|---|
| step_type | request |
| url | URL of your endpoint |
| method | HTTP verb of your request (GET, POST, PUT, DELETE, OPTIONS, PATCH) |
| body | A string with the body of the request |
| variables | Array of variables to extract from the response (see Using Variables to Pass Data Between Steps for details) |
| headers | Object who contains all the headers attach to the request (see how to add headers) |
| assertions | Array of assertions, this is the acceptance tests (see how to create assertion tests) |
| skipped | If true the step is skipped and nothing is running. |
Headers
Headers are represented by an object containing all the headers to send.
Each header is has the name of the header for key and an array of strings as value.
You can use variables in the headers, they will be replaced before sending the request (see Using Variables to Pass Data Between Steps or Global Variables).
Example:
YAML
```yaml headers: Accept-Charset: - utf-8 Accept: - application/scim+json Authorization: - {{auth}} ```JSON
```json { "headers": { "Accept-Charset": [ "utf-8" ], "Accept": [ "application/scim+json" ], "Authorization": [ "{{auth}}" ] } } ```Assertions
Assertions are a big part of api-scenario, this is the acceptance tests of your request, it will allow you to simply write test to verify that you endpoint is doing what you want.
Assertion composition
| Property | Description |
|---|---|
| source | The location of the data to extract for comparison. See available source type to have authorized values. |
| comparison | The type of operation to perform when comparing the extracted data with the target value. (see Available comparison type). |
| property | The property of the source data to retrieve.
|
| value | The expected value used to compare against the actual value. |
Example:
YAML
```yaml - comparison: equals property: schemas value: User source: response_json ```JSON
```json { "comparison": "equals", "property": "schemas", "value": "User", "source": "response_json" } ```Available source type
| Source | Config name | Description |
|---|---|---|
| HTTP code | response_status |
HTTP response status codes (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status). |
| Response time | response_time |
Duration of the request in seconds. |
| Response Headers | response_header |
Target headers of the response. |
| Body response (JSON) | response_json |
Target the response body extract in JSON. |
| Body response (plain text) | response_text |
Target the response body extract in plain text. |
| Body response (XML) | response_xml |
Target the response body extract in XML. |
Available comparison type
| Comparison | Config name | Description |
|---|---|---|
| is empty | empty |
The actual value exists and is an empty string or null. |
| is not empty | not_empty |
The actual value exists and is a value other than an empty string or null. |
| equals | equal |
A string comparison of the actual and expected value. Non-string values are cast to a string before comparing. For comparing non-integer numbers, use equals (number). |
| does not equal | not_equal |
A string comparison of the actual and target value. |
| contains | contains |
The actual value contains the target value as a substring. |
| does not contain | does_not_contain |
The target value is not found within the actual value. |
| has key | has_key |
Checks for the existence of the expected value within a dictionary's keys. The actual value must point to a dictionary (JSON only). |
| has value | has_value |
Checks a list or dictionary for the existence of the expected value in any of the list or dictionary values. The actual value must point to a JSON list or dictionary (JSON only). |
| is null | is_null |
Checks that a value for a given JSON or XML key is null. |
| is a number | is_a_number |
Validates the actual value is (or can be cast to) a valid numeric value. |
| less than | is_less_than |
Validates the actual value is (or can be cast to) a number less than the target value. |
| less than or equal | is_less_than_or_equals |
Validates the actual value is (or can be cast to) a number less than or equal to the target value. |
| greater than | is_greater_than |
Validates the actual value is (or can be cast to) a number greater than the target value. |
| greater than or equal | is_greater_than_or_equal |
Validates the actual value is (or can be cast to) a number greater than or equal to the target value. |
| equals (number) | equal_number |
Validates the actual value is (or can be cast to) a number equal to the target value. This setting performs a numeric comparison: for example, "1.000" would be considered equal to "1". |
Request Chaining
Using Variables to Pass Data Between Steps
Request steps can define variables that extract data from HTTP responses returned when running the test.
To create a variable, add a variables block to your step and specify the location of the data you'd like to extract
from the response, and the name of this variable.
YAML
```yaml variables: - source: response_json property: point name: active ```JSON
```json { "variables": [ { "source": "response_json", "property": "point", "name": "active" } ] } ```A variable is:
| Source | The location of the data to extract. Data can be extracted from
|
| Property | The property of the source data to retrieve. For HTTP headers this is the name of the header. For JSON content, see below. Unused status code. |
| Variable Name | The name of the variable to assign the extracted value to. In subsequent requests you can retrieve the value of the variable by this name. See Using Variables in Requests. |
Extracting Data from JSON Body Content
Data from a JSON response body can be extracted by specifying the path of the target data using standard JavaScript notation. View sample JSON.
Global Variables
Some variables could be set up at launch, for that you can add options to the run command to pass it.
Common values (base URLs, API tokens, etc.) that are shared across requests within a test, or tests within a bucket,
should be stored in an Initial Variable.
Once defined, the variable is available to all requests within the test.
To add a variable just use the option --variable or -V and specify the key:value of this variable.
./api-scenario run -F your_file.json --variable="baseUrl:http://www.google.com/" -V "token:token1234"Note that if you create a variable in a step with the same name of a global variable it will override it.
Add / Override headers
Overriding headers works the same as global variables.
You can add a header for all your requests by using the option --header or -H, it will add or override the header
for all requests.
./api-scenario run -F your_file.json --header="Content-Type:application/json" -H "Authorization: Bearer Token123"Built-in Variables and Functions
| Variable/Function | Description | Example Output |
|---|---|---|
{{timestamp}} |
Integer Unix timestamp (seconds elapsed since January 1, 1970 00:00 UTC) | 1384035195 |
{{utc_datetime}} |
UTC datetime string in ISO 8601 format. | 2013-11-07T19:24:41.418968 |
{{format_timestamp(value, format)}} |
Timestamp of the specified value in the specified format. Any delimiters (e.g. -, /, ., *, etc.) can be used in the format with a combination of any of the following date/time format options. Also accepts variables. E.g. {{format_timestamp({{timestamp}}, YYYY-MM-DD)}}
|
2019-31-03 |
{{timestamp_offset(value)}} |
Integer Unix timestamp offset by the specified value in seconds (going back in time would be a negative offset value). Values should be passed without surrounding quotes. | 1383948795 |
{{random_int}} |
Random integer between 0 and 18446744073709551615 | 407370955 |
{{random_int(a,b)}} |
Random integer value between a and b, inclusive. | 44674407370 |
{{random_string(length)}} |
Random alphanumeric string of the specified length (max 1000 characters). | ddo1qlQR81 |
{{uuid}} |
Random universally unique identifier (UUID). | 99386c08-6da7-4833-bb31-e70ce747c921 |
{{encode_base64(value)}} |
Encodes value in Base64. Values should be passed without surrounding quotes. Also accepts variables e.g. {{encode_base64({{username}}:{{password}})}} |
dTpwDQo= |
{{md5(value)}} |
Generate an MD5 hash based on value. Values should be passed without surrounding quotes. Also accepts variables e.g. {{md5({{timestamp}})}} |
50b7fe4da64720232c25bc7c6d66f6c5 |
{{sha1(value)}} |
Generate an SHA-1 hash based on value. Values should be passed without surrounding quotes. Also accepts variables e.g. {{sha1({{timestamp}})}} |
e0bd9304537cd8cb4e69ef5d73771fe218c484f5 |
{{sha256(value)}} |
Generate an SHA-256 hash based on value. Values should be passed without surrounding quotes. Also accepts variables e.g. {{sha1({{timestamp}})}} |
e3376ffb4b1e2c04b0fe68b52e8654696814b4883b47a56ff5a7df883725d8c1 |
{{hmac_sha1(value,key)}} |
Generate an HMAC using the SHA-1 hashing algorithm based on value and key. Values should be passed without surrounding quotes. Also accepts variables e.g. {{hmac_sha1({{timestamp}},key)}} |
163a04cd86a82b948a7e85f0ed3cd3b5929a7d0c |
{{hmac_sha256(value,key)}} |
Generate an HMAC using the SHA-256 hashing algorithm based on value and key. Values should be passed without surrounding quotes. Also accepts variables e.g. {{hmac_sha1({{timestamp}},key)}} |
eb0b5c5b2a04ac25ff52c886e115f2e60c0dd8d50bab076dc065e95f5fd37fb9 |
{{url_encode(value)}} |
Create a percent-encoded string suitable for URL querystrings. This is not required for URL or form parameters defined in the request editor which are automatically encoded. Only use this if you need to double encode a value in a URL or include a URL encoded string in a header value. | This%20is%20100%25%20URL%20encoded. |
Using Variables in Requests
Once a variable has been defined, you can use it in any subsequent request.
Variables can be used in any request data field including the method, URL, header values, parameter values and request bodies.
To include the value of a variable in a request, enter the name of the variable surrounded by double braces e.g. {{variable_name}}.
If a variable is undefined when a request using that variable is executed, we will keep the variable un-replaced.