apiaryio / dredd

Language-agnostic HTTP API Testing Tool
https://dredd.org
MIT License
4.18k stars 279 forks source link

question: dredd testing request/response structures only #153

Closed ducin closed 8 years ago

ducin commented 9 years ago

Is there a possibility to define a request JSON structure and provide a response JSON structure in markdown so that dredd will check if apropriate fields exist in JSON maps, but it won't check their values?

For example, take a look at this: https://github.com/apiaryio/api-blueprint/blob/master/examples/7.%20Parameters.md#retrieve-all-messages-get, there is a response JSON defined:

[
  {
    "id": 1,
    "message": "Hello World!"
  },
  {
    "id": 2,
    "message": "Time is an illusion. Lunchtime doubly so."
  },
  {
    "id": 3,
    "message": "So long, and thanks for all the fish."
  }
]

How will dredd behave when such blueprint found? Will it expect this exact URL/GET to return this exact value? This would not be very helpful. Can I tell Dredd to check whether id and message fields exist?

netmilk commented 9 years ago

Dredd validates only presence of keys in JSON bodies, so values are not tested for exact match. At this moment problem is when you wont to validate only some presence of some keys and some mark as optional. But in the API Blueprint, there is ongoing work on support of MSON for describing data structures and their validations.

If you want to know more about how validation mechanisms works pleas refer to Gavel project and it's js npm.

ducin commented 9 years ago

Adam, thank you for your response. This was not clear for me after reading Dredd docs.

I've got a further questions related to MSON that you mentioned - don't know if I understand it correctly. Please correct me in case I'm wrong... here is how I see this:

JSON Schema and MSON work together. JSON schema tool stack can be used with all supported platforms (e.g. python jsonschema module, or java's json-schema-validator or just other implementations) and MSON is just another automate-friendly alternative syntax that works together with api blueprint stack. In some way, MSON to JSON Schema is like Jade to HTML. I can use MSON to define structures of my API on the design and I can still use java's json-schema-validator to do automatic API validation. And everything works together: MSON is used to generate JSON schema structures which are eventually used for automatic API tests.

Is that right?


After a quick-read, I love MSON. But I haven't found the most important thing yet - its generator. Is there at least a MSON->JSON schema converter written already? Where can I find current status of MSON work? What is available for MSON in the entire stack already?


And concerning Gavel - I can see that it's a tool that validates HTTP communication (comparing requests and responses by checking both headers and body). The question in: if gavel is here, then... what does dredd do? Do they work together somehow? Or are these alternatives to each other?

netmilk commented 8 years ago

@ducin Just letting know here's the way how to render MSON's from JS: https://tonicdev.com/zdne/mson-renderer