search

OAI / OpenAPI-Specification

The OpenAPI Specification Repository
https://openapis.org
Apache License 2.0
29.02k stars 9.07k forks source link

Describe expected response times for responses #3063

Open agardnerIT opened 2 years ago

agardnerIT commented 2 years ago

I can describe expected return codes, response types and even expected responses via the spec: https://swagger.io/docs/specification/describing-responses/

What I feel is missing right now is the ability to document expected response times. As a developer, I know my endpoint should return in <100ms with a 201.

Is there a way to achieve this today (perhaps via some custom annotation?) or would this require a spec update. If so, how do I formally submit that as a request?

rafalkrupinski commented 2 years ago

That's an interesting idea.

I guess it would make sense to put it under either operation or server, you could use description or add a custom attribute e.g. x-timeout-ms.

The only problem is, there can always be a delay on the way, so if the operation, including the network transmission, takes 101ms, and client developer imposes that 100ms as a timeout, they lose the response.

The typical default timeout is 30sec for the reason that the internet doesn't guarantee response times.

agardnerIT commented 2 years ago

My intention isn't to fail the response but more use it as a hint. Ie. when the request hits the API to the backend and back - that's the time I'm interested in hinting at.

Other actors can then infer expected behaviour. It would eliminate false callouts as I can see that actually, the dev intended it to take 5 seconds so no point in moaning that it is taking 3.

image

agardnerIT commented 2 years ago

@rafalkrupinski how do I progress this? Can this be achieved now and if so, how? If not possible today, I'm more than happy to jump on calls / present this proposal. Just let me know where and when and I'll turn up.

Looking forward to PoCing htis if it is possible (I'm an OpenAPI newbie so a noob guide would be great for me :) )

rafalkrupinski commented 2 years ago

@agardnerIT OAS allows arbitrary extension fields so I don't think you need to start a long process to change it for your particular needs.

How you support timeouts depends on your tooling. If you use a code generator to create the client code, you can add an extension field and handle it in the generator (You say you don't want to fail the request, but I guess you'd add a warning).

If the client is coded by hand, you can either use an extension or simply the description field and let the client developer handle it in their code.

It's also a matter of how you create the schema. If it's generated from the server code, it also needs to support adding extension fields.

darrelmiller commented 1 year ago

I would start by suggesting some x- extensions and let's see whether there is interest from a broader group of people. If it becomes popular we can considering incorporating the values into a future spec update.

agardnerIT commented 1 year ago

Thanks. x-expected-response-time-ms

AMorgaut commented 1 year ago

I join this request

We're working on a framework that reads many information from a mandatory Open API specification file including:

We also have mandatory response time requirements that we need to enforce in the specification as all routes don't have the same response time criticity based on business use cases.

I take this occasion to remind that this request also exists in fact in a still opened issue from 2016 ;-)

As no-standard field exists yes in OAS, we'll for sure start defining a vendor specific x-timeout, x-expected-response-time-ms, or x-max-response-time related field, but that will not help getting a better native support by existing open api tools

We expect such field, once standardized, to be interpreted & used by:

Moreover we could define rules in tools like Spectral to expect a 408 response from routes that define such timeout to detect such kind of potential issues before running service contact tests

rafalkrupinski commented 1 year ago

Does any of the open-source tools you've mentioned support timeout via a x- property so far?

AMorgaut commented 1 year ago

I still have to dig

The first I can mention is the timeoutInMillis property of the x-amazon-apigateway-integration vendor specific object property from the AWS API Gateway

On its side Apigee natively handles different types of timeouts in its products, but while you can publish Open API based services, their timeouts cannot be defined through that Open API specification. It only supports standard Open API fields

agardnerIT commented 1 year ago

Costs Codes

Suggestions: x-cost and related x-cost-code

x-cost: "0.03"
x-cost-code: "USD"

Examples

Endpoint Evaluation

I have a system which calls a third party API. I am evaluating between two different third parties for this APIs. All the usual SLOs apply (response time etc.) but cost is also a major factor. If endpointA costs 3 "whatevers" per call and endpointB costs 30 "whatevers" per call - that's a major plus point for endpointA.

image

Direct cost tracking

If I have the above, my tooling can aggregate the header data and show:

You have spent 123.45USD over the last 24hrs with endpointA. This is x% of your budget.

Different Definitions of Cost

Cost of course can be monetary, but a "cost" could also indicate other quantities (like a carbon emissions costs).

GabenGar commented 1 year ago

Doesn't this fall apart once proxy/reverse-proxy servers enter into the picture? They can add to a response time which can't be tracked by the application returning the header, and the client will not be able to tell apart a proxy/app response of 2/5 from 5/2.