search

apideck-libraries / portman

Port OpenAPI Specs to Postman Collections, inject test suite and run via Newman 👨🏽‍🚀
http://getportman.com/
Apache License 2.0
622 stars 58 forks source link
api api-testing cli cli-app cli-option contract-first contract-testing integration-testing oas openai-tooling openapi postman-collection postman-collections swagger test-framework testing testrunner

readme

portman-hero

Total Downloads Latest Stable Version

Portman 👨🏽‍🚀

Port OpenAPI Spec to Postman Collection, with contract & variation tests included!

Portman leverages OpenAPI documents, with all its defined API request/response properties, to power your Postman collection. Let Portman do all the work and inject contract & variation tests with a minimum of configuration. Customize the Postman requests & variables with a wide range of options to assign & overwrite variables.

[!CAUTION]
Breaking Change: The default behaviour of the Query parameters is changed since version 1.26.0.
Optional query parameters will be disabled in Postman by default. More details can be found in the CHANGELOG.md.

Why use Portman?

Convert your OpenAPI spec to Postman, generate contract & variation tests, upload the Postman collection & run the tests through Newman. Include the Portman CLI as part of an automated process for injecting the power of Portman directly into your CI/CD pipeline.

Read the full blog post

Features

With Portman, you can:

Getting started

  1. Install Portman
  2. Initialize Portman CLI configuration by running: $ portman --init

OR

  1. Install Portman
  2. Copy .env.example to .env and add environment variables you need available to your collection
  3. Copy/rename and customize each of the ____.default.json config files in the root directory to suit your needs
  4. Start converting your OpenAPI document to Postman

OR

If you have an existing OpenAPI specification, try running Portman without any special setup to see how it can generate a Postman collection with contract tests with it's default configuration.

  1. Install Portman
  2. Run portman on your OpenAPI spec, ie:
    • npx portman -l my-openapi-spec.yaml
    • (if your spec is hosted use the -u parameter, ie:
    • npx portman -u https://petstore3.swagger.io/api/v3/openapi.json

This will generate a postman collection that contains a request for every method:endpoint combination defined in your spec, and include a set of "Contract Tests" for each one. You can learn more about contract tests, and how to examine the generated collection here.

(Running portman with no explicit configuration is the same as running it with this configuration file)

All configuration options to convert from OpenAPI to Postman can be found in the openapi-to-postman package documentation. All configuration options to filter flags/tags/methods/operations/... from OpenAPI can be found in the openapi-format package documentation or using the online openapi-format playground.

Installation

Local Installation (recommended)

You can add the Portman CLI to the node_modules by using:

$ npm install --save @apideck/portman

or using yarn:

$ yarn add @apideck/portman

Note that this will require you to run the Portman CLI with npx @apideck/portman -l your-openapi-file.yaml or, if you are using an older version of npm, ./node_modules/.bin/portman -l your-openapi-file.yaml.

Global Installation

$ npm install -g @apideck/portman

NPX usage

To execute the CLI without installing it via npm, use the npx method.

$ npx @apideck/portman -l your-openapi-file.yaml

CLI Usage

Usage: -u <url> -l <local> -b <baseUrl> -t <includeTests>

Options:
 --help                     Show help                                                                        [boolean]
 --version                  Show version number                                                              [boolean]
 --url,-u                   URL of OAS to port to Postman collection                                         [string]
 --local, -l                Use local OAS to port to Postman collection                                      [string]
 --baseUrl, -b              Override spec baseUrl to use in Postman                                          [string]
 --output, -o               Write the Postman collection to an output file                                   [string]
 --oaOutput                 Write the (filtered) OpenAPI file to an output file                              [string]
 --runNewman, -n            Run Newman on newly created collection                                           [boolean]
 --newmanRunOptions         JSON stringified object to pass options for configuring Newman                   [string]
 --newmanOptionsFile        Path/URL to Newman options file to pass options for configuring Newman           [string]
 --newmanIterationData, -d  Iteration data to run Newman with newly created collection                       [string]
 --localPostman             Use local Postman collection, skips OpenAPI conversion                           [string]
 --syncPostman              Upload generated collection to Postman (default: false)                          [boolean]
 --syncPostmanCollectionIds Synchronises the IDs of newly created postman collections with those already
                            on Postman, useful when you want to use Postman pull request (default: false)    [boolean]
 --postmanFastSync          Postman sync creates new collection (new UID),instead of update (default: false) [boolean]
 --postmanRefreshCache      Postman sync will refresh all local cached Postman API data (default: false)     [boolean]
 --postmanUid, -p           Postman collection UID to upload with the generated Postman collection           [string]
 --postmanWorkspaceName     Postman Workspace name to target the upload of the generated Postman collection  [string]
 --includeTests, -t         Inject Portman test suite (default: true)                                        [boolean]
 --bundleContractTests      Bundle Portman contract tests in a separate folder in Postman (default: false)   [boolean]
 --portmanConfigFile, -c    Path/URL to Portman settings config file (portman-config.json)                   [string]
 --postmanConfigFile,-s     Path to openapi-to-postman config file (postman-config.json)                     [string]
 --filterFile               Path/URL to openapi-format config file (oas-format-filter.json)                  [string]
 --envFile                  Path to the .env file to inject environment variables                            [string]
 --collectionName           Overwrite OpenAPI title to set the Postman collection name                       [string]
 --cliOptionsFile           Path/URL to Portman CLI options file                                            [string]
 --ignoreCircularRefs       Ignore circular references in OpenAPI spec (default: false)                      [boolean]
 --logAssignVariables       Toggle logging of assigned variables (default: true)                             [boolean]
 --init                     Configure Portman CLI options in an interactive manner                           [string]
 --extraUnknownFormats      Add extra unknown formats to json schema tests                                   [array]

Environment variables as Postman variables

Portman uses dotenv to not only access variables for functionality, but you can also add environment variables that you'd like declared within your Postman environment. Simply prefix any variable name with PORTMAN_, and it will be available for use in your Postman collection as the camel-cased equivalent. For example:

PORTMAN_CONSUMER_ID=test_user_id

will be available in your collection or tests by referencing:

{{consumerId}}

It is possible to set a spec-specific .env file, that lives next to your config files. The path can be passed in via envFile cli option. This is useful if you have Portman managing multiple specs that have unique environment requirements.

By default, Portman will leverage any ENVIRONMENT variable that is defined that starts with PORTMAN_.

Another option to set variables is by configuring them as collectionVariables in the globals section of your Portman configuration.

CLI Options

Initialize Portman CLI configuration
portman --init

The init option will help you to configure the cliConfig options and put the default config, env file in place to kick-start the usage of Portman.

Pass in the remotely hosted spec
portman -u https://specs.apideck.com/crm.yml
Overwrite the baseUrl in spec and run Newman
portman -u https://specs.apideck.com/crm.yml -b http://localhost:3050 -n true
Path pass to a local data file for Newman to use for iterations
portman -u https://specs.apideck.com/crm.yml -b http://localhost:3050 -n true -d ./tmp/newman/data/crm.json
Pass the path to a local spec (useful when updating your specs) and output Postman collection locally
portman -l ./tmp/specs/crm.yml -o ./tmp/specs/crm.postman.json
Skip tests and just generate collection
portman -l ./tmp/specs/crm.yml -t false
Filter OpenAPI and generate collection
portman -u https://specs.apideck.com/crm.yml --filterFile examples/cli-filtering/oas-format-filter.json

For more details, review the cli-filtering example.

Add extra forms to Json schema validation
portman -l ./tmp/specs/crm.yml -o ./tmp/specs/crm.postman.json --extraUnknownFormats ulid one two

This makes the schema validation more lenient, and solves problems with unknown formats

Upload newly generated collection to Postman, which will upsert the collection, based on the collection name
portman -l ./tmp/specs/crm.yml --syncPostman

Upload newly generated collection to Postman using the collection UID to overwrite the existing.

portman -l ./tmp/specs/crm.yml --syncPostman -p 9601963a-53ff-4aaa-92a0-2e70a8a2a748

When a collection gets large, the Postman API will compare all the requests when updating the collection. This can take some time even result in 5xx errors. To overcome this, you can use the --postmanFastSync option. This option will sync your collection to Postman by using "delete" and "create" operations instead of the "update".

REMARK: Using --postmanFastSync will result in a new Postman collection and Postman UID for each sync.

portman -l ./tmp/specs/crm.yml --syncPostman --postmanFastSync

Portman caches a set of Postman API data to facilitate faster lookups and uploads, preventing unnecessary connecting to the Postman API. In case you need to reset the cache you simply remove the .portman.cache.json file or set the --postmanRefreshCache option when running the Postman sync.

portman -l ./tmp/specs/crm.yml --syncPostman --postmanRefreshCache
Pass custom paths for config files

All configuration options to convert from OpenAPI to Postman can be on the openapi-to-postman package documentation. Portman provides a default openapi-to-postman configuration postman-config.default.json, which will be used if no custom config --postmanConfigFile is passed.

Portman configuration file in JSON format:

portman -u https://specs.apideck.com/crm.yml -c ./tmp/crm/portman-config.json -s ./common/postman-config.json

Portman configuration file in YAML format:

portman -u https://specs.apideck.com/crm.yml -c ./tmp/crm/portman-config.yaml -s ./common/postman-config.json
Pass all CLI options as JSON/YAML file

All the CLI options can be managed in a separate configuration file and passed along to the portman command. This will make configuration easier, especially in CI/CD implementations.

Portman CLI options settings in JSON format

portman --cliOptionsFile ./examples/cli-options/portman-cli-options.json

Portman CLI options settings in YAML format

portman --cliOptionsFile ./examples/cli-options/portman-cli-options.yaml

All the available Portman CLI options can be used in the config file. By passing the CLI options as parameter, you can overwrite the defined CLI options defined in the file.

For more details, review the cli-options example.

Run Newman with Newman options

All Newman configuration options to run Newman can be passed along through Portman.

portman -u https://specs.apideck.com/crm.yml -c ./tmp/crm/portman-config.json --runNewman --newmanOptionsFile ./tmp/crm/newman-options.json

For more details, review the cli-options example.

NOTE: Newman is set to ignore redirects to allow for testing redirect response codes. If you are running collections within Postman UI, you'll need to ensure Postman is set to the same, or your redirect tests will fail.

Postman > Preferences > Automatically follow redirects > OFF

Output

Without specifying the output location, your generated Postman Collection is written to ./tmp/converted/${specName}.json if you are manually importing to Postman or need to inspect for debugging.

By using -o or --output parameter, you can define the location where the Postman collection will be written.

portman -l ./tmp/specs/crm.yml -o ./tmp/specs/crm.Postman.json

Portman settings

The Portman settings consist out of multiple parts:

Portman targeting

It is possible to inject Postman tests and pre-register scripts, assign variables and overwrite query params, headers, request body data with values.

To be able to do this very specifically, there are options to define the targets:

An openApiOperationId is an optional property. To offer support for OpenAPI documents that don't have operationIds, we have added the openApiOperation definition, which is the unique combination of the OpenAPI method & path, with a :: separator symbol. The targeting option excludeForOperations is really useful when using wildcards, to allow exclusions from the wildcard.

This will allow targeting for very specific OpenAPI items.

To facilitate managing the filtering, we have included wildcard options for the openApiOperation option, supporting the methods & path definitions.

REMARK: Be sure to put quotes around the target definition.

Portman - tests properties

The Portman tests is where you would define the tests that would be applicable and automatically generated by Portman, based on the OpenAPI document. The contract tests are grouped in an array of contractTests.

contractTests options

For more details, review the contract-tests example.

variationTests options

For more details, review the content-variation example.

integrationTests options

Portman - contentTests properties

Content tests will validate if the response property values will match the expected defined values. While the Portman tests verify the "contract" of the API, the contentTests will verify the content of the API.

contentTests options

For more details, review the content-tests example.

Portman - extendTests properties

When you need to add additional tests or overwrite the Portman-generated test, you can use the extendTests to define the raw Postman tests. Anything added in the tests array will be added to the Postman test scripts.

extendTests options

Portman - assignVariables properties

The "assignVariables" allows you to set Postman collection variables for easier automation.

assignVariables options

For more details, review the Assign variables example and Assign & Overwrite example.

Portman - overwrites properties

To facilitate automation, you might want to modify properties with "randomized" or specific values. The overwrites are mapped based on the OpenAPI operationId or OpenAPI Operation reference.

overwrites options

For more details, review the Overwrites example and Assign & Overwrite example.

Portman - fuzzing properties - BETA 🏗

NOTICE: This feature is considered BETA, since we are investigating additional fuzzing capabilities.

Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program (a REST API in the case of Portman).

Fuzzing changes the requests (body, query params, ... ) to unexpected values in an effort to cause unexpected behavior and errors in the API response. For Portman, we want to provide a simple form of Fuzzing, with the goal to trigger validation/error responses, which can be contract tested. The automatic fuzzing is based on the OpenAPI request properties, where for each fuzzing variation a new Postman request will be generated, with optional contract tests.

The Fuzzing options describe the configuration setting for available OpenAPI fuzzing variations.

REMARKS:

fuzzing options

For more details, review the fuzzing example.

Portman - operationPreRequestScripts properties

The operationPreRequestScripts configuration will inject pre-request scripts in the Postman collection, on request level. Postman executes pre-request scripts before a request runs. If you want to set the Postman Collection pre-request scripts on the collection level, you can use the globals > collectionPreRequestScripts configuration. The operationPreRequestScripts is inserted on the request level.

operationPreRequestScripts options

Portman - globals property

The configuration defined in the globals will be executed on the full Postman collection. This is handy if you need to do mass replacements of variables or specific words/keys/values in the full collection that cannot be overwritten per request.

globals options

For more details on the globals configuration options , review the globals example and ordering example

Configure automatic upload to Postman App

REMARK: Portman does not require you to have a Postman account.

In case you want to sync the generated Postman collection with the Postman app (portman --syncPostman), you would need a Postman account since Portman leverages the Postman API to sync the collection.

This can be a "free" Postman account or any of the paid Postman plans.

The generated Postman collection can always be imported manually, without a Postman account.

To enable automatic uploads of the generated Postman collection through Portman, follow these steps:

  1. Get your Postman API key

Documentation Pipeline

Documentation Pipeline

Documentation Pipeline

  1. Goto the root folder of your project

  2. Copy env-postman-app-example as .env in the root folder of your project

  3. Enter your Postman API key in a local .env file, as POSTMAN_API_KEY=[replace with Postman api key]

Next to the Postman API key, you can also pass along the Postman Workspace name & the specific Postman Collection UID.

Supported Postman API .ENV variables:

The POSTMAN_WORKSPACE_NAME & POSTMAN_COLLECTION_UID variables can also be set as CLI Options --postmanWorkspaceName & --postmanUid , which will overrule the variables defined in the .ENV file.

RECOMMENDATION: Do not commit the .env file in any versioning system like GIT if it contains confidential credentials.

Credits

Portman started as a PR on the handy openapi-to-postman package to generate basic Postman tests from the OpenAPI specification.

Apideck immediately saw the PR's value and collaborated with the original author, Tim Haselaars, to adopt the functionality and extend the options & tooling to create "Portman".

The goal of Portman is to drive API automation by 'porting' a static OpenAPI document to a dynamic Postman collection that includes a powerful testing suite with variable requests, bodies and more. All this while being easy to configure & ready to use.

Portman is a valuable tool in any OpenAPI workflow, for local development or as part of a CI/CD automation pipeline.

Credits for this package for the hard work of Nick Lloyd and Tim Haselaars.

Future ideas

Resources

A collection of blog posts and resources about Portman