search

stoplightio / prism

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.
https://stoplight.io/open-source/prism
Apache License 2.0
4.3k stars 347 forks source link

Produce API coverage report #1843

Open yjaaidi opened 3 years ago

yjaaidi commented 3 years ago

User story.

As an API developer, I can generate coverage reports when running prism proxy, so that I know which routes, requests, properties, and responses have been seen by prism proxy when testing the API automatically through the proxy.

Describe the solution you'd like

A new --coverage option (e.g. prism proxy --coverage) could produce lcov coverage output (or other format) based on the routes inputs and outputs that have been seen by the proxy. If we use the lcov format, this could point to the spec file as source code.

For example, if a route is never used, the coverage could show the line 3 as not covered:

paths:
  /farms:
    get:
      ...

Concerning properties, this could be more tricky as models are shared.

Does this deserve to be an extension or a dedicated tool?

Additional context

Similar initiatives:

philsturgeon commented 3 years ago

This would definitely be really interesting, and I'd love to see it happen, but it's a lot of work for something not currently on the Stoplight roadmap. Maybe you could submit the idea over there too and we can see if the product folks want to take it on, or we can try and get this done in the community. We're offering bigger tree-bribes than usual to get things done this October. :D

Would anyone be interested in taking a swing at it? There's Stoplight engineers on here and in the Stoplight Discord who can give pointers for things like this.