search

taverntesting / tavern

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax
https://taverntesting.github.io/
MIT License
1.03k stars 194 forks source link

[Feature request]: slurp openapi spec and produce coverage report #362

Open tommyjcarpenter opened 5 years ago

tommyjcarpenter commented 5 years ago

I think a really neat idea would be to take the OpenAPI (aka swagger) spec, and produce a "coverage report", i.e., show how many of the API endpoints in the API were successfully tested/hit by tavern. What do you think of this idea?

michaelboulton commented 5 years ago

There is an example script of generating tavern tests from an openapi spec here https://github.com/taverntesting/tavern/tree/master/example/generate_from_openapi but it's not really something that Tavern was designed to do. There isn't really an easy way to map the number of endpoints tested to the number of endpoints in the API spec other than maybe writing another Pytest plugin that checked which requests were made to which endpoints and comparing it to the swagger file.

On a related note - if you do want to run example tests from inside a swagger file then I think Dredd does something like that which might be better suited to your needs: https://dredd.org/en/latest/