apiaryio / dredd

Language-agnostic HTTP API Testing Tool
https://dredd.org
MIT License
4.18k stars 279 forks source link

Open API 3.x Dredd sends empty request body for POST api call with requestBody #1339

Open nandhinik opened 5 years ago

nandhinik commented 5 years ago

Dredd sends empty request body for POST api call with requestBody for Open API 3.0.0 and 3.0.2 versions where as it works for Open API 2.0

To Reproduce Run dredd with the following OpenAPI v3 specification (fragment):

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  description: A sample API that uses a petstore as an example to demonstrate features in the OpenAPI 3.0 specification
  termsOfService: http://swagger.io/terms/
  contact:
    name: Swagger API Team
    email: apiteam@swagger.io
    url: http://swagger.io
  license:
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0.html
servers:
  - url: http://petstore.swagger.io/api
paths:
  /pets:
    post:
      description: Creates a new pet in the store.  Duplicates are allowed
      operationId: addPet
      requestBody:
        description: Pet to add to the store
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/NewPet'
      responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
components:
  schemas:
    Pet:
      allOf:
        - $ref: '#/components/schemas/NewPet'
        - required:
          - id
          properties:
            id:
              type: integer
              format: int64

    NewPet:
      required:
        - name  
      properties:
        name:
          type: string
        tag:
          type: string    

Expected behavior Request body should be present(not empty)

What is in your dredd.yml? Not using it

dredd Command dredd petstore.yml http://petstore.swagger.io/api --language=python --hookfiles=p_hooks.py --loglevel=debug

What's your dredd --version output? dredd v9.0.4 (Darwin 17.7.0; x64)

Does dredd --loglevel=debug uncover something? I added print statement to print request body in hooks file @hooks.before_each

2019-04-24T19:53:31.523Z - debug: Loading configuration file: ./dredd.yml 2019-04-24T19:53:31.527Z - debug: Dredd version: 9.0.4 2019-04-24T19:53:31.527Z - debug: Node.js version: v11.13.0 2019-04-24T19:53:31.527Z - debug: Node.js environment: node=11.13.0, v8=7.0.276.38-node.18, uv=1.27.0, zlib=1.2.11, brotli=1.0.7, ares=1.15.0, modules=67, nghttp2=1.34.0, napi=4, llhttp=1.1.1, httpparser=2.8.0, openssl=1.1.1b, cldr=34.0, icu=63.1, tz=2018e, unicode=11.0 2019-04-24T19:53:31.528Z - debug: System version: Darwin 17.7.0 x64 2019-04-24T19:53:31.840Z - debug: npm version: 6.7.0 2019-04-24T19:53:31.840Z - debug: Configuration: {"server":"http://petstore.swagger.io/api","options":{"":["petstore.yml"],"color":true,"language":"python","a":"python","hookfiles":"p_hooks.py","f":"p_hooks.py","loglevel":"debug","l":"debug","dry-run":null,"y":null,"require":null,"server":null,"g":null,"server-wait":3,"init":false,"i":false,"custom":{},"j":[],"names":false,"n":false,"only":[],"x":[],"reporter":[],"r":[],"output":[],"o":[],"header":[],"h":[],"sorted":false,"s":false,"user":null,"u":null,"inline-errors":false,"e":false,"details":false,"d":false,"method":[],"m":[],"path":["petstore.yml","petstore.yml"],"p":["petstore.yml","petstore.yml"],"hooks-worker-timeout":5000,"hooks-worker-connect-timeout":1500,"hooks-worker-connect-retry":500,"hooks-worker-after-connect-wait":100,"hooks-worker-term-timeout":5000,"hooks-worker-term-retry":500,"hooks-worker-handler-host":"127.0.0.1","hooks-worker-handler-port":61321,"config":"./dredd.yml","$0":"/usr/local/bin/dredd"},"custom":{"cwd":"/Users/nandhini/dredd_workspace","argv":["petstore.yml","http://petstore.swagger.io/api","--language=python","--hookfiles=p_hooks.py","--loglevel=debug"]}} 2019-04-24T19:53:31.844Z - debug: No backend server process specified, starting testing at once 2019-04-24T19:53:31.844Z - debug: Running Dredd instance. 2019-04-24T19:53:31.845Z - debug: Configuring reporters 2019-04-24T19:53:31.845Z - debug: Using 'base' reporter. 2019-04-24T19:53:31.846Z - debug: Configuring reporters: [] 2019-04-24T19:53:31.847Z - debug: Using 'cli' reporter. 2019-04-24T19:53:31.847Z - debug: Resolving --require 2019-04-24T19:53:31.847Z - debug: Resolving API descriptions locations 2019-04-24T19:53:31.850Z - debug: Reading API description files. 2019-04-24T19:53:31.857Z - debug: Parsing API description files and compiling a list of HTTP transactions to test. 2019-04-24T19:53:31.857Z - debug: Parsing API description: /Users/nandhini/dredd_workspace/petstore.yml 2019-04-24T19:53:31.904Z - debug: Compiling HTTP transactions from API description: /Users/nandhini/dredd_workspace/petstore.yml 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'Info Object' contains unsupported key 'termsOfService' on line 6 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'Info Object' contains unsupported key 'contact' on line 7 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'Info Object' contains unsupported key 'license' on line 11 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'OpenAPI Object' contains unsupported key 'servers' on line 14 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'Request Body Object' contains unsupported key 'required' on line 23 2019-04-24T19:53:31.915Z - warn: Parser warning in '/Users/nandhini/dredd_workspace/petstore.yml': 'Schema Object' contains unsupported key 'allOf' on line 38 2019-04-24T19:53:31.915Z - debug: Starting transaction runner 2019-04-24T19:53:31.915Z - debug: Starting reporters and waiting until all of them are ready 2019-04-24T19:53:31.916Z - debug: Beginning Dredd testing... 2019-04-24T19:53:31.916Z - debug: Sorting HTTP transactions 2019-04-24T19:53:31.917Z - debug: Configuring HTTP transactions 2019-04-24T19:53:31.918Z - debug: Reading hook files and registering hooks 2019-04-24T19:53:31.919Z - debug: Found Hookfiles: 0=/Users/nandhini/dredd_workspace/p_hooks.py 2019-04-24T19:53:31.920Z - debug: Looking up hooks handler implementation: python 2019-04-24T19:53:31.921Z - debug: Starting hooks handler. 2019-04-24T19:53:31.921Z - debug: Spawning 'python' hooks handler process. 2019-04-24T19:53:31.927Z - debug: Connecting to hooks handler. 2019-04-24T19:53:32.018Z - debug: Hooks handler stderr: /Users/nandhini/dredd_workspace/p_hooks.py:2: RuntimeWarning: Parent module 'p_hooks' not found while handling absolute import import dredd_hooks as hooks

2019-04-24T19:53:32.018Z - debug: Hooks handler stdout: Starting Dredd Python hooks handler

2019-04-24T19:53:32.430Z - debug: Starting TCP connection with hooks handler process. 2019-04-24T19:53:32.433Z - debug: Successfully connected to hooks handler. Waiting 0.1s to start testing. 2019-04-24T19:53:32.534Z - debug: Registering hooks. 2019-04-24T19:53:32.534Z - debug: Executing HTTP transactions 2019-04-24T19:53:32.535Z - debug: Running 'beforeAll' hooks 2019-04-24T19:53:32.535Z - debug: Running hooks... 2019-04-24T19:53:32.537Z - debug: Sending HTTP transaction data to hooks handler: fe9c7776-a039-4e18-9903-d7ca89f43934 2019-04-24T19:53:32.538Z - debug: Hooks handler stdout: ########before all

2019-04-24T19:53:32.538Z - debug: Dredd received some data from hooks handler. 2019-04-24T19:53:32.538Z - debug: Dredd received a valid message from hooks handler: fe9c7776-a039-4e18-9903-d7ca89f43934 2019-04-24T19:53:32.538Z - debug: Handling hook: fe9c7776-a039-4e18-9903-d7ca89f43934 2019-04-24T19:53:32.539Z - debug: Processing transaction #1: /pets > POST > 200 > application/json 2019-04-24T19:53:32.539Z - debug: Running 'beforeEach' hooks 2019-04-24T19:53:32.539Z - debug: Running hooks... 2019-04-24T19:53:32.539Z - debug: Sending HTTP transaction data to hooks handler: 80ac3fe6-bff9-419a-b177-de9834e8a721 > 2019-04-24T19:53:32.539Z - debug: Hooks handler stdout: ########before each ('$$$$$request', {u'body': u'', u'headers': {u'Content-Type': u'application/json', u'User-Agent': u'Dredd/9.0.4 (Darwin 17.7.0; x64)'}, u'method': u'POST', u'uri': u'/pets'}) ('$$$$$uri', u'/pets') ('$$$$$headers', {u'Content-Type': u'application/json'}) ('$$$$$requestbody', u'')

2019-04-24T19:53:32.540Z - debug: Dredd received some data from hooks handler. 2019-04-24T19:53:32.540Z - debug: Dredd received a valid message from hooks handler: 80ac3fe6-bff9-419a-b177-de9834e8a721 2019-04-24T19:53:32.540Z - debug: Handling hook: 80ac3fe6-bff9-419a-b177-de9834e8a721 2019-04-24T19:53:32.540Z - debug: Running 'before' hooks 2019-04-24T19:53:32.540Z - debug: Emitting to reporters: test start 2019-04-24T19:53:32.541Z - debug: Performing HTTP request to the server under test: POST http://petstore.swagger.io/api/pets 2019-04-24T19:53:33.476Z - debug: Handling HTTP response from the server under test 2019-04-24T19:53:33.477Z - debug: Running 'beforeEachValidation' hooks 2019-04-24T19:53:33.477Z - debug: Running hooks... 2019-04-24T19:53:33.477Z - debug: Sending HTTP transaction data to hooks handler: 6ce86188-bb4f-4fd4-98e9-e878295698d4 2019-04-24T19:53:33.478Z - debug: Hooks handler stdout: #########before each validation

2019-04-24T19:53:33.478Z - debug: Dredd received some data from hooks handler. 2019-04-24T19:53:33.479Z - debug: Dredd received a valid message from hooks handler: 6ce86188-bb4f-4fd4-98e9-e878295698d4 2019-04-24T19:53:33.479Z - debug: Handling hook: 6ce86188-bb4f-4fd4-98e9-e878295698d4 2019-04-24T19:53:33.479Z - debug: Running 'beforeValidation' hooks 2019-04-24T19:53:33.479Z - debug: Validating HTTP transaction by Gavel.js 2019-04-24T19:53:33.479Z - debug: Determining whether HTTP transaction is valid (getting boolean verdict) 2019-04-24T19:53:33.490Z - debug: Validating HTTP transaction (getting verbose validation result) 2019-04-24T19:53:33.492Z - debug: Running 'afterEach' hooks 2019-04-24T19:53:33.492Z - debug: Running hooks... 2019-04-24T19:53:33.492Z - debug: Sending HTTP transaction data to hooks handler: aaa75764-100c-4f24-bf8c-17f5d445c5fa 2019-04-24T19:53:33.493Z - debug: Hooks handler stdout: #########after_each

2019-04-24T19:53:33.493Z - debug: Dredd received some data from hooks handler. 2019-04-24T19:53:33.494Z - debug: Dredd received a valid message from hooks handler: aaa75764-100c-4f24-bf8c-17f5d445c5fa 2019-04-24T19:53:33.494Z - debug: Handling hook: aaa75764-100c-4f24-bf8c-17f5d445c5fa 2019-04-24T19:53:33.494Z - debug: Running 'after' hooks 2019-04-24T19:53:33.494Z - debug: Evaluating results of transaction execution #1: /pets > POST > 200 > application/json 2019-04-24T19:53:33.494Z - debug: Emitting to reporters: test fail fail: POST (200) /pets duration: 953ms 2019-04-24T19:53:33.495Z - debug: Running 'afterAll' hooks 2019-04-24T19:53:33.495Z - debug: Running hooks... 2019-04-24T19:53:33.495Z - debug: Sending HTTP transaction data to hooks handler: b67c3932-be4f-4955-86a3-ba070097e889 2019-04-24T19:53:33.495Z - debug: Hooks handler stdout: ##########after_all

2019-04-24T19:53:33.496Z - debug: Dredd received some data from hooks handler. 2019-04-24T19:53:33.496Z - debug: Dredd received a valid message from hooks handler: b67c3932-be4f-4955-86a3-ba070097e889 2019-04-24T19:53:33.496Z - debug: Handling hook: b67c3932-be4f-4955-86a3-ba070097e889 2019-04-24T19:53:33.496Z - debug: Terminating hooks handler process, PID 15832 2019-04-24T19:53:33.497Z - debug: Gracefully terminating the hooks handler process 2019-04-24T19:53:33.497Z - debug: TCP communication with hooks handler closed. 2019-04-24T19:53:33.497Z - debug: Hooks handler stderr: Connection closed, could not parse JSON

2019-04-24T19:53:33.498Z - debug: Gracefully terminating the hooks handler process 2019-04-24T19:53:33.499Z - debug: Wrapping up testing and waiting until all reporters are done info: Displaying failed tests... fail: POST (200) /pets duration: 953ms fail: headers: Header 'content-type' has value 'text/html; charset=ISO-8859-1' instead of 'application/json' statusCode: Status code is '404' instead of '200'

2019-04-24T19:53:33.500Z - debug: Could not stringify: request: body:

headers: Content-Type: application/json User-Agent: Dredd/9.0.4 (Darwin 17.7.0; x64)

method: POST uri: /pets

expected: headers: Content-Type: application/json

statusCode: 200

2019-04-24T19:53:33.500Z - debug: Could not stringify:

Error 404 Not Found

HTTP ERROR 404

Problem accessing /api/pets. Reason:

    Not Found


Powered by Jetty://

actual: body:

Error 404 Not Found

HTTP ERROR 404

Problem accessing /api/pets. Reason:

    Not Found


Powered by Jetty://

headers: content-length: 292 via: 1.1 ord1fwsg02-wcg.security.rackspace.net age: 0 server: Jetty(9.2.9.v20150224) connection: close access-control-allow-methods: GET, POST, DELETE, PUT cache-control: must-revalidate,no-cache,no-store date: Wed, 24 Apr 2019 19:53:42 GMT access-control-allow-origin: * access-control-allow-headers: Content-Type, api_key, Authorization content-type: text/html; charset=ISO-8859-1

statusCode: 404 bodyEncoding: utf-8

complete: 0 passing, 1 failing, 0 errors, 0 skipped, 1 total complete: Tests took 1583ms

Can you send us failing test in a Pull Request? No

honzajavorek commented 5 years ago

Thanks for the issue! We fixed a bunch of issues in https://github.com/apiaryio/dredd/pull/1351. Could you please verify that this is still relevant with the latest Dredd version? If so, I’ll re-open the issue. Thank you 🙏

yourithielen commented 5 years ago

Had the same issue but I think i fixed it by using type: object in your model definition when defining an object.

philiplinell commented 5 years ago

@honzajavorek seems to me like this is still an issue. I hope it is ok if I continue on this issue.

I get the same result with the following:

To Reproduce

const Dredd = require('dredd');

let configuration = {
    options:  {
        endpoint: 'http://127.0.0.1:3000'
    },
    loglevel: 'debug',
    path: ['animals.yaml'],
};

let dredd = new Dredd(configuration);

dredd.run((err) => {
    if (err) {
        console.error('Encountered problems!');
        console.error(err);
        process.exit(1);
    }
})

animals.yaml:

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Animals

paths:
  /animals:
    post:
      summary: Create animal
      requestBody:
        content:
          application/vnd.api+json:
            schema:
              #type: object
              properties:
                data:
                  $ref: '#/components/schemas/AnimalRequest'
      responses:
        '201':
          description: Animal created
          content:
            application/vnd.api+json:
              schema:
                properties:
                  data:
                    $ref: "#/components/schemas/Animal"

components:
  schemas:

    AnimalRequest:
      #type: object
      properties:
        type:
          description: Resource type
          enum:
            - animals
          type: string
        attributes:
          #type: object
          properties:
            name:
              description: 'The name of the animal'
              type: string
              example: 'Pluto'

    Animal:
      properties:
        type:
          description: Resource type
          enum:
            - animals
          type: string
        id:
          description: Unique ID of the animal
          type: string
          example: '123e4567-e89b-12d3-a456-426655440000'
        attributes:
          properties:
            name:
              description: 'The name of the animal'
              type: string
              example: 'Pluto'

will return:

...
2019-05-21T13:40:20.746Z - debug: Could not stringify:
request:
method: POST
uri: /animals
headers:
    Content-Type: application/vnd.api+json
    User-Agent: Dredd/11.1.0 (Linux 4.15.0-50-generic; x64)

body:

expected:
headers:
...

Expected behavior

If I uncomment line 14, 32 and 40 (#type: object) in animals.yaml a body is correctly returned.

...

request:
method: POST
uri: /animals
headers:
    Content-Type: application/vnd.api+json
    User-Agent: Dredd/11.1.0 (Linux 4.15.0-50-generic; x64)

body:
{
  "data": {
    "type": "animals",
    "attributes": {
      "name": "Pluto"
    }
  }
}

expected:
headers:
    Content-Type: application/vnd.api+json
...

Seems that Dredd really wants us to specify type: object. I could be wrong, but I thought you don't have to specify type: object according to the specification. Perhaps Dredd should assume an object if nothing else is specified?

What is in your dredd.yml?

No dredd.yml.

What's your dredd --version output?

dredd v11.1.0 (Linux 4.15.0-50-generic; x64)
honzajavorek commented 5 years ago

Thanks folks, we'll investigate this.

nandhinik commented 5 years ago

Hi All!

It seems there was some issue in parsing request body in OAPI v3 and installing dependent libraries solved this issue. Please let me know if this change makes sense and update the source code accordingly. Thanks!

Please refer below for code changes:

Screen Shot 2019-05-24 at 5 18 20 PM
honzajavorek commented 5 years ago

@nandhinik Dredd should already have the latest Dredd Transactions, which already has the latest OAS3 adapter: https://github.com/apiaryio/dredd/blob/master/package.json#L43 Install the latest Dredd to see whether it solves your problem. According to reports here though it seems this particular problem wasn't fixed yet by our changes.