Dredd crashes on Petstore example

[bagrat]

Describe your problem

In order to evaluate dredd I have tried it on the Petstore example and I got a failure with the following error message:

error: Cannot read property 'toValue' of undefined

What command line options do you use?

$ dredd http://petstore.swagger.io/v2/swagger.json http://petstore.swagger.io/v2 --level debug

What is in your dredd.yml?

I did not use any configuration file.

What's your dredd --version output?

dredd v4.5.0 (Darwin 16.7.0; x64)

Does dredd --level=debug uncover something?

Nope, the output is as follows:

verbose: Loading configuration file: ./dredd.yml
debug: Dredd version: 4.5.0
debug: Node.js version: v8.6.0
debug: Node.js environment: http_parser=2.7.0, node=8.6.0, v8=6.0.287.53, uv=1.14.1, zlib=1.2.11, ares=1.10.1-DEV, modules=57, nghttp2=1.25.0, openssl=1.0.2l, icu=59.1, unicode=9.0, cldr=31.0.1, tz=2017b
debug: System version: Darwin 16.7.0 x64
debug: npm version: 5.3.0
debug: Configuration: {"server":"http://petstore.swagger.io/v2","options":{"_":["http://petstore.swagger.io/v2/swagger.json"],"level":"debug","l":"debug","color":false,"c":"false","dry-run":null,"y":null,"hookfiles":null,"f":null,"language":"nodejs","a":"nodejs","sandbox":false,"b":false,"server":null,"g":null,"server-wait":3,"init":false,"i":false,"custom":{},"j":[],"names":false,"n":false,"only":[],"x":[],"reporter":[],"r":[],"output":[],"o":[],"header":[],"h":[],"sorted":false,"s":false,"user":null,"u":null,"inline-errors":false,"e":false,"details":false,"d":false,"method":[],"m":[],"timestamp":false,"t":false,"silent":false,"q":false,"path":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2/swagger.json"],"p":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2/swagger.json"],"hooks-worker-timeout":5000,"hooks-worker-connect-timeout":1500,"hooks-worker-connect-retry":500,"hooks-worker-after-connect-wait":100,"hooks-worker-term-timeout":5000,"hooks-worker-term-retry":500,"hooks-worker-handler-host":"127.0.0.1","hooks-worker-handler-port":61321,"config":"./dredd.yml","$0":"/usr/local/bin/dredd"},"custom":{"cwd":"/Users/bagrat/sauce/sauce/hostess/resto","argv":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2","--level","debug","--color","false"]}}
verbose: Using 'base' reporter.
verbose: Configuring reporters: []
verbose: Using 'cli' reporter.
verbose: No backend server process specified, starting testing at once
verbose: Running Dredd instance.
verbose: Expanding glob patterns.
verbose: Reading API description files.
verbose: Downloading remote file: http://petstore.swagger.io/v2/swagger.json
verbose: Parsing API description files and compiling a list of HTTP transactions to test.
verbose: Compiling HTTP transactions from API description file: http://petstore.swagger.io/v2/swagger.json
verbose: Dredd instance run finished.
error: Cannot read property 'toValue' of undefined
verbose: Exiting Dredd process with status '1'.
debug: Using native process.exit() method to terminate the Dredd process.
verbose: No backend server process to terminate.

Can you send us failing test in a Pull Request?

Not yet 😢

[kylef]

I'm able to confirm this behaviour, looks like the specific Swagger document doesn't contain any status code in the response for the post method of /user. dredd-transactions is assuming there is one. The specific part of the Swagger document causing the problem:

      },
      "/user":{  
         "post":{  
            "tags":[  
               "user"
            ],
            "summary":"Create user",
            "description":"This can only be done by the logged in user.",
            "operationId":"createUser",
            "produces":[  
               "application/xml",
               "application/json"
            ],
            "parameters":[  
               {  
                  "in":"body",
                  "name":"body",
                  "description":"Created user object",
                  "required":true,
                  "schema":{  
                     "$ref":"#/definitions/User"
                  }
               }
            ],
            "responses":{  
               "default":{  
                  "description":"successful operation"
               }
            }
         }
      },

This results in the underlying parser returning a httpResponse which doesn't have a status code. I am not sure what the expected behaviour should be here @honzajavorek, perhaps Dredd can either show a warning/error and not validate the status code in this case. This should help support default responses when the user does not define a status code.

[kylef]

Also note, it appears that Apiary rendered documentation would assume a 200 status code in this case.

[honzajavorek]

Default responses are ignored by Dredd (we didn't really figure out how to properly test them so the result would make sense). Dredd is looking for regular responses with status code.

I see several issues:

• Docs probably do not mention at all default responses are not supported and tested. We need to make sure it's there. ➡️ docs bug
• Dredd crashes as reported by @bagrat (thanks for the issue! ✨) and that's definitely not nice. It should give a comprehensible error or warning and it should end gracefully ➡️ bug
• Dredd ignores the default response. ➡️ feature request, solution design needed

For the last one, feel free to file a separate issue if there isn't an existing one. Let's track the first two in this issue.

[honzajavorek]

We agreed assuming 200 on a single default response would be a good way to go. Also, I'm not sure, but Gavel.js might be able to validate the rest without status code provided - that needs exploration.

[honzajavorek]

@bagrat This should be now fixed in v4.7.1. Would you mind to confirm it's okay now?

[jnj16180340]

@honzajavorek It's still broken in v4.7.1 and v4.7.2. The second example (below petstore) was working up until dredd 4.6.2

input + output

nate@momo:~/BioBright/nodeBioBright/node_modules/dredd$ ./bin/dredd http://petstore.swagger.io/v2/swagger.json http://petstore.swagger.io/v2 --level debug
verbose: Loading configuration file: ./dredd.yml
debug: Dredd version: 4.7.2
debug: Node.js version: v8.6.0
debug: Node.js environment: http_parser=2.7.0, node=8.6.0, v8=6.0.287.53, uv=1.14.1, zlib=1.2.11, ares=1.10.1-DEV, modules=57, nghttp2=1.25.0, openssl=1.0.2l, icu=59.1, unicode=9.0, cldr=31.0.1, tz=2017b
debug: System version: Linux 4.10.0-40-generic x64
debug: npm version: 5.5.1
debug: Configuration: {"server":"http://petstore.swagger.io/v2","options":{"_":["http://petstore.swagger.io/v2/swagger.json"],"level":"debug","l":"debug","dry-run":null,"y":null,"hookfiles":null,"f":null,"language":"nodejs","a":"nodejs","sandbox":false,"b":false,"server":null,"g":null,"server-wait":3,"init":false,"i":false,"custom":{},"j":[],"names":false,"n":false,"only":[],"x":[],"reporter":[],"r":[],"output":[],"o":[],"header":[],"h":[],"sorted":false,"s":false,"user":null,"u":null,"inline-errors":false,"e":false,"details":false,"d":false,"method":[],"m":[],"color":true,"c":true,"timestamp":false,"t":false,"silent":false,"q":false,"path":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2/swagger.json"],"p":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2/swagger.json"],"hooks-worker-timeout":5000,"hooks-worker-connect-timeout":1500,"hooks-worker-connect-retry":500,"hooks-worker-after-connect-wait":100,"hooks-worker-term-timeout":5000,"hooks-worker-term-retry":500,"hooks-worker-handler-host":"127.0.0.1","hooks-worker-handler-port":61321,"config":"./dredd.yml","$0":"/home/nate/.nvm/versions/node/v8.6.0/bin/node ./bin/dredd"},"custom":{"cwd":"/home/nate/BioBright/nodeBioBright/node_modules/dredd","argv":["http://petstore.swagger.io/v2/swagger.json","http://petstore.swagger.io/v2","--level","debug"]}}
verbose: Using 'base' reporter.
verbose: Configuring reporters: []
verbose: Using 'cli' reporter.
verbose: No backend server process specified, starting testing at once
verbose: Running Dredd instance.
verbose: Expanding glob patterns.
verbose: Reading API description files.
verbose: Downloading remote file: http://petstore.swagger.io/v2/swagger.json
verbose: Parsing API description files and compiling a list of HTTP transactions to test.
verbose: Compiling HTTP transactions from API description file: http://petstore.swagger.io/v2/swagger.json
verbose: Dredd instance run finished.
error: Cannot assign to read only property 'parent' of object '#<AuthScheme>'
verbose: Exiting Dredd process with status '1'.
debug: Using native process.exit() method to terminate the Dredd process.
verbose: No backend server process to terminate.

---

It also crashes (identical error) on swagger.yaml definitions (v2) with paths like this:

  /devices/{device_id}:
    x-swagger-router-controller: devices
    get:
      operationId: get_devices_id
      security:
        - JWT: ["read:devices"]
      description: Returns details of a specific device
      tags:
        - devices
      parameters:
        - $ref: "#/parameters/device_id_path"
      responses:
        "200":
          $ref: "#/responses/DeviceIdResponse"
        default:
          $ref: "#/responses/ErrorResponse"

On the custom swagger.yaml file, this runs:

  /devices/{device_id}:
    x-swagger-router-controller: devices
    get:
      operationId: get_devices_id
      security:
        - JWT: ["read:devices"]
      description: Returns details of a specific device
      tags:
        - devices
      parameters:
        - $ref: "#/parameters/device_id_path"
      responses:
        #"200":
        #  $ref: "#/responses/DeviceIdResponse"
        default:
         $ref: "#/responses/ErrorResponse"

This crashes:

  /devices/{device_id}:
    x-swagger-router-controller: devices
    get:
      operationId: get_devices_id
      security:
        - JWT: ["read:devices"]
      description: Returns details of a specific device
      tags:
        - devices
      parameters:
        - $ref: "#/parameters/device_id_path"
      responses:
        "200":
          $ref: "#/responses/DeviceIdResponse"
        #default:
        # $ref: "#/responses/ErrorResponse"

This does not crash:

  /devices/{device_id}:
    x-swagger-router-controller: devices
    get:
      operationId: get_devices_id
      #security:
      #  - JWT: ["read:devices"]
      description: Returns details of a specific device
      tags:
        - devices
      parameters:
        - $ref: "#/parameters/device_id_path"
      responses:
        "200":
          $ref: "#/responses/DeviceIdResponse"
        #default:
        # $ref: "#/responses/ErrorResponse"

Omitting some other examples, it looks like a security block and explicitly specified response code (anything other than default is necessary and sufficient to crash.

In this case, the hooks file is like this:

hooks.before('auth > /api/v0/oauth2/access_token > POST > 200 > application/json; charset=utf-8', function (transaction) {
  transaction.request.body = `{ "username": "stuff", "password": "stuff", "nodeBioBright_client_id": "stuff", "scopes":
 ""}`
})
hooks.after('auth > /api/v0/oauth2/access_token > POST > 200 > application/json; charset=utf-8', function (transaction) {
  jwt.token = JSON.parse(transaction.real.body).access_token
})

hooks.beforeEach(function (transaction) {
  transaction.request.headers['Authorization'] = `Bearer ${jwt.token}`
})
``

[jnj16180340]

@honzajavorek Actually, it looks like the behavior I described is a separate issue. Sorry...

On dredd 4.7.0,

• the other example I posted works fine
• petstore fails with error: Cannot read property 'toValue' of undefined.

On dredd >4.7.0,

• both the other example and petstore fail with error: Cannot assign to read only property 'parent' of object '#<AuthScheme>'

[honzajavorek]

It looks like we fixed the problem with default response, but there is more of them in the Petstore example, so the title of this issue still holds 🙂

I think the easiest way to fix this would be to just put the Petstore to Dredd's integration tests and to make sure Dredd is able to deal with it. The main problem here is now auth, which Dredd doesn't support, but it should gracefully inform about that fact and it should skip it.

The preferred way to implement auth with Dredd at this moment is to use hooks, which is exactly what you do 👍