apideck-libraries / postman-to-k6

Converts Postman collections to k6 script code
https://npmjs.com/package/@apideck/postman-to-k6
Apache License 2.0
234 stars 24 forks source link

Question: Is there a plan to provide mentioned `unsupported-features` ? #29

Closed svmundada closed 2 months ago

svmundada commented 2 years ago

As per https://github.com/apideck-libraries/postman-to-k6#unsupported-features, some features are not supported (although this pm.response.to.have.status(200); worked for us!) which creates some blockers in converting our already written postman scripts to k6 scripts.

Is there a plan to provide for those features? Until these are provided, any other workarounds for these missing features?

Thanks for maintaining this friendly fork!

thim81 commented 2 years ago

@svmundada

The documented "unsupported" features were taken over from the original repo. I'll review the list of features, since pm.response.to.have.status is working.

With regards to brining support for these features, it depends on the needs of the community and the feasibility within K6.

For example: postman.setNextRequest is a highly demanded request BUT it wouldn't fit the K6 functionality. We could covert postman.setNextRequest to a series of K6 requests, that are generated as a sequence of seperate requests (the "navigaiton use-case of postman.setNextRequest) but how would K6 know how to handle logic (IF ELSE) that can be triggered in Postman? Or we could trigger a loop in K6, with the IF ELSE, but this would only cover the looping use-case of postman.setNextRequest.

Feel free to share missing features and use-cases, so we can discuss if they could fit postman-to-k6 conversion.

svmundada commented 2 years ago

The documented "unsupported" features were taken over from the original repo. I'll review the list of features, since pm.response.to.have.status is working.

sure thanks.

At the moment, having pm.sendRequest and pm.response.headers.* would be of great help to us. Are these possible?

Kpizzle commented 2 years ago

I'm currently in the process of slowly creating a proof of concept for using postman-to-k6 for our QA team.

Ideally we would need the pm.sendRequest would be awesome to have. As currently we have to call AWS cognito to get user credentials before I can make any other calls, and having a the pre-script to get the tokens would be awesome. Currently I'm looking into creating a bespoke implementation that could fix this, but it does seem to break the workflow using postman-to-k6.

thim81 commented 2 years ago

@Kpizzle The pre-request scripts are supported, but the "pm.sendRequest" would be more difficult, since this concept does not really exist in the K6 concepts.

As a work-around I do it like so:

A simplified example of a packages.json contains the steps mentioned:

{
  "name": "api-k6",
  "version": "1.0.0",
  "description": "API - K6 testing suite",
  "scripts": {
    "generate:k6:api:oauth:token": "node api.pipeline.env.js",
    "generate:k6:api:pipeline": "postman-to-k6 api.postman.json --output  k6-script.js --environment api.pipeline.env.json ",
    "test:k6:api": "k6 run k6-script.js",
    "generate:k6:report": "node utils/generateReport.js"
  },
  "dependencies": {
    "@apideck/postman-to-k6": "^1.8.3",
    "axios": "^0.24.0",
    "dotenv": "^9.0.2",
    "k6-html-reporter": "^1.0.5",
    "k6-to-junit": "^1.0.3"
  }
}

Example of the api.pipeline.env.js script that generates the Postmen environment (using an Oauth flow), which uses ENVIRONMENT variables to pass along authentication parameters.:

const fs = require('fs');
const path = require('path');
const request = require('axios');
const dotenv = require('dotenv');
dotenv.config({path: __dirname + '/.env'});

if (!process.env.OAUTH_ACCESS_TOKEN_URL) {
    console.log('No ENV variables or .env file found.')
    process.exit(1)
}

const currentTime = new Date().getTime();

const requestOptions = {
    url: process.env.OAUTH_ACCESS_TOKEN_URL,
    method: 'POST',
    headers: {"Content-Type": "application/json"},
    data: JSON.stringify({
        "client_id": process.env.OAUTH_CLIENT_ID,
        "client_secret": process.env.OAUTH_CLIENT_SECRET,
        "audience": process.env.OAUTH_AUDIENCE,
        'account_id': process.env.OAUTH_ACCOUNT_ID,
        "grant_type": "client_credentials"
    })
};

request(requestOptions)
    .then(function (res) {
        const resBody = res.data
        let postmanEnv = {
            "id": "7796f73b-b0d8-4b1a-98c8-9951304740b8",
            "name": "api-PIPELINE",
            "values": [
                {
                    "key": "baseUrl",
                    "value": process.env.BASEURL,
                    "enabled": true
                },
                {
                    "key": "oauth_access_token",
                    "value": resBody.access_token,
                    "enabled": true
                },
                {
                    "key": "oauth_access_token_expires",
                    "value": resBody.expires_in,
                    "enabled": true
                },
                {
                    "key": "oauth_access_token_url",
                    "value": process.env.OAUTH_ACCESS_TOKEN_URL,
                    "enabled": true
                },
                {
                    "key": "oauth_grant_type",
                    "value": "client_credentials",
                    "enabled": true
                },
                {
                    "key": "oauth_client_id",
                    "value": process.env.OAUTH_CLIENT_ID,
                    "enabled": true
                },
                {
                    "key": "oauth_client_secret",
                    "value": process.env.OAUTH_CLIENT_SECRET,
                    "enabled": true
                },
                {
                    "key": "oauth_audience",
                    "value": "https://audience.example",
                    "enabled": true
                },
                {
                    "key": "oauth_account_id",
                    "value": process.env.OAUTH_ACCOUNT_ID,
                    "enabled": true
                },
                {
                    "key": "oauth_base_url",
                    "value": process.env.OAUTH_BASE_URL,
                    "enabled": true
                }
            ]
        };

        let postmanEnvOutput = JSON.stringify(postmanEnv, null, 2);
        let currentFile = path.basename(__filename);
        let postmanEnvFile = currentFile.substring(0, currentFile.length - 3);
        fs.writeFileSync(__dirname + '/' + postmanEnvFile + '.json', postmanEnvOutput, 'utf8');

        console.log('Auth0 access token updated');
    })
    .catch(function (err) {
        // handle error
        console.log(`POST ${process.env.OAUTH_ACCESS_TOKEN_URL} failure`, err.message)
        process.exit(1)
    })

so locally or in my CI/CD I use the following sequence:

npm run generate:k6:api:oauth:token && npm run generate:k6:api:pipeline
npm run test:k6:api
thim81 commented 2 years ago

FYI: I have collected your question in the "discussions" section for future reference.

Kpizzle commented 2 years ago

Awesome @thim81 thanks very much! I'll have a deep-dive into this and continue the conversation in the discussion section.

kgrozdanovski commented 2 years ago

Is there any news or plans to support pm.response.headers.*? I noticed @svmundada asked but there was no answer for this specific feature.

We rely a lot on validating response headers so this feature would be super useful for me and my team.

thim81 commented 2 years ago

hi @kgrozdanovski

I can investigate to see what is needed to bring the support.

Could you share an example of what you would do with the pm.response.headers.*?

thim81 commented 7 months ago

hi @kgrozdanovski,

Thanks to a contribution from @aegrey, postman-to-k6 supports 2 header methods: pm.response.headers.all() & pm.response.headers.get(). These are part of the v1.9.0 release.