search

apiaryio / dredd

Language-agnostic HTTP API Testing Tool
https://dredd.org
MIT License
4.2k stars 280 forks source link

Head type requests not working #2280

Open tkrisztian95 opened 1 year ago

tkrisztian95 commented 1 year ago

Describe the bug Cannot test HEAD type request examples because it causes an error in the underlying requests lib. 

error: Error: HTTP HEAD requests MUST NOT include a request body.
    at request (/poc-api-contract-testing-dredd/node_modules/request/index.js:50:11)
    at Object.performRequest [as default] (/poc-api-contract-testing-dredd/node_modules/dredd/build/performRequest.js:41:9)
    at TransactionRunner.performRequestAndValidate (/poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:508:33)
    at TransactionRunner.executeTransaction (/poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:497:14)
    at /poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:131:30
    at /poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:201:78
    at /poc-api-contract-testing-dredd/node_modules/async/dist/async.js:248:13
    at wrapper (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:268:20)
    at replenish (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:435:29)
    at /poc-api-contract-testing-dredd/node_modules/async/dist/async.js:445:13
error: HEAD (204) /api/v1/foo duration: 1ms
error: TypeError: Cannot read properties of undefined (reading 'status')
    at /poc-api-contract-testing-dredd/hooks.js:152:24
    at TransactionRunner.runHook (/poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:228:13)
    at runHookWithData (/poc-api-contract-testing-dredd/node_modules/dredd/build/TransactionRunner.js:175:26)
    at /poc-api-contract-testing-dredd/node_modules/async/dist/async.js:243:13
    at replenish (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:440:21)
    at /poc-api-contract-testing-dredd/node_modules/async/dist/async.js:445:13
    at _asyncMap (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:241:16)
    at mapLimit (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:1763:16)
    at awaitable (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:208:32)
    at timesLimit (/poc-api-contract-testing-dredd/node_modules/async/dist/async.js:4215:16)
error: HEAD (204) /api/v1/foo: NaNms

But, there is no request body in the API docs that I'm using.

I think Dredd provides an empty Buffer for the call here https://github.com/apiaryio/dredd/blob/master/packages/dredd/lib/performRequest.js#L72 that's causing the problem. Instead it should be considered that the request is HEAD or not.

To Reproduce Steps to reproduce the behavior. How do you call the dredd command? Which command line options?

Command: dredd ./head-test.apib https://1234.mock.pstmn.io

API docs:

FORMAT: 1A
HOST: https://1234.mock.pstmn.io

# Mock API
##  Group Users [/api/v1/user/custom-fields-missing]
### Users Total [HEAD /public/v2/users]

Returns the total number of users in header 'X-Users-Total'.

+ Response 204

    + Headers

            X-User-Total: 10

Output:

error: Error: HTTP HEAD requests MUST NOT include a request body.
    at request (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/request/index.js:50:11)
    at Object.performRequest [as default] (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/performRequest.js:41:9)
    at TransactionRunner.performRequestAndValidate (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:508:33)
    at TransactionRunner.executeTransaction (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:497:14)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:131:30
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:120:26
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:115:22
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/async/dist/async.js:243:13
error: HEAD (204) /public/v2/users duration: 1ms
info: Displaying failed tests...
fail: HEAD (204) /public/v2/users duration: 1ms
fail: Group Users > Users Total
request: 
method: HEAD
uri: /public/v2/users
headers: 
    User-Agent: Dredd/14.1.0 (Linux 5.15.90.1-microsoft-standard-WSL2; x64)

body: 

expected: 
headers: 
    X-User-Total: 10

statusCode: 204

complete: 0 passing, 0 failing, 1 errors, 0 skipped, 1 total
complete: Tests took 5ms

Expected behavior Dredd should send out a HEAD type request from the example given in docs. No error is thrown.

What is in your dredd.yml?

empty

What's your dredd --version output?

dredd v14.1.0

Does dredd --loglevel=debug uncover something? If you run Dredd with debugging output, do you see any interesting information relevant to the bug?

...
2023-10-10T08:55:18.250Z - debug: Requesting tested server errored: Error: HTTP HEAD requests MUST NOT include a request body.
    at request (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/request/index.js:50:11)
    at Object.performRequest [as default] (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/performRequest.js:41:9)
    at TransactionRunner.performRequestAndValidate (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:508:33)
    at TransactionRunner.executeTransaction (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:497:14)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:131:30
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:120:26
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:115:22
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/async/dist/async.js:243:13
2023-10-10T08:55:18.251Z - debug: Emitting to reporters: test error
error: Error: HTTP HEAD requests MUST NOT include a request body.
    at request (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/request/index.js:50:11)
    at Object.performRequest [as default] (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/performRequest.js:41:9)
    at TransactionRunner.performRequestAndValidate (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:508:33)
    at TransactionRunner.executeTransaction (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:497:14)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:131:30
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:120:26
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:115:22
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/async/dist/async.js:243:13
error: HEAD (204) /public/v2/users duration: 1ms
2023-10-10T08:55:18.251Z - debug: Running 'afterEach' hooks
2023-10-10T08:55:18.251Z - debug: Running 'after' hooks
2023-10-10T08:55:18.251Z - debug: Evaluating results of transaction execution #1: Group Users > Users Total
2023-10-10T08:55:18.252Z - debug: No emission of test data to reporters Error: HTTP HEAD requests MUST NOT include a request body.
    at request (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/request/index.js:50:11)
    at Object.performRequest [as default] (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/performRequest.js:41:9)
    at TransactionRunner.performRequestAndValidate (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:508:33)
    at TransactionRunner.executeTransaction (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:497:14)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:131:30
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:120:26
    at TransactionRunner.runHooksForData (/home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:204:13)
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/build/TransactionRunner.js:115:22
    at /home/ktoth/.nvm/versions/node/v18.8.0/lib/node_modules/dredd/node_modules/async/dist/async.js:243:13
2023-10-10T08:55:18.253Z - debug: Running 'afterAll' hooks
2023-10-10T08:55:18.253Z - debug: Wrapping up testing and waiting until all reporters are done
info: Displaying failed tests...
fail: HEAD (204) /public/v2/users duration: 1ms
fail: Group Users > Users Total
2023-10-10T08:55:18.254Z - debug: Could not stringify: 
request: 
method: HEAD
uri: /public/v2/users
headers: 
    User-Agent: Dredd/14.1.0 (Linux 5.15.90.1-microsoft-standard-WSL2; x64)

body: 

expected: 
headers: 
    X-User-Total: 10

statusCode: 204

complete: 0 passing, 0 failing, 1 errors, 0 skipped, 1 total
complete: Tests took 6ms
2023-10-10T08:55:18.254Z - debug: Dredd instance run finished.
2023-10-10T08:55:18.254Z - debug: Using native process.exit() method to terminate the Dredd process with status '1'.
2023-10-10T08:55:18.254Z - debug: No backend server process to terminate.

Can you send us failing test in a Pull Request? We'll gladly help you to contribute; answering yes raises the chances the issue gets fixed.

dblock commented 6 months ago

This is pretty old, but still an issue. Will queue up fixing after #2295 is merged.

tkrisztian95 commented 5 days ago

@dblock Any updates on this? I can see that https://github.com/apiaryio/dredd/pull/2295 is still open and pending for review.

dblock commented 5 days ago

We (OpenSearch) decided not to go with Dredd and wrote some tooling. Check out https://github.com/opensearch-project/opensearch-api-specification, I think it's really robust - we would love to extract it into its own toolset, maybe someone wants to take it on?