“Websocket” communication specification. Alternative (non-REST) way to communicate

[artem-korolev]

Hi everybody,

Dunno if OpenAPI guys are interested in that thing. Maybe HTTP2.0 going to replace websocket and we can easily forgot about this topic. anyways...

Client can receive messages even without sending any requests. It can be some status messages, chat messages, or whatever you want. This is full duplex connection. So it’s two way communication. Client can send messages and receive response for it. Can send messages and not wait for response (unbelievable situation, but why not). Client can wait for messages from server and not send any request to server. When it receive message from server, then it can send back to server response message. Websocket connection is alive for the whole runtime lifecycle of the application. This is the main goal in most scenarios We have new keyword - "message". It is not request or response. It is message. And we need also mechanism to answer with messages to some messages, if we need it, in any direction.

With best regards, Artem

[artem-korolev]

https://github.com/swagger-api/swagger-socket/issues/47#issuecomment-163627431

[artem-korolev]

this is good article to understand the difference https://blogs.oracle.com/PavelBucek/entry/websocket_vs_rest1

and understand that it is not really replacement of REST, but simply another technology to communicate

this is very good presentation also - https://www.youtube.com/watch?v=B-ElrhYxPQU1 - on 25 min they talk about SOAP on websocket. very interesting to investigate

[casualjim]

You can kind of represent it like this: https://github.com/swagger-api/swagger-spec/issues/396

[artem-korolev]

Okay. But what it gives to represent it in swagger, but without tooling? more interesting is to extend swagger specs and then extend tools. I'm 100% ready to do it. Just want to ask for help from community to show me some starting point or whatever.

[casualjim]

Ok but it starts with somebody making a proposal, this is my shot at that proposal. If people like it, we can agree on it being a worthwhile extension and add it to tooling.

[fehguy]

Parent issue #586

[Somebi]

Agree, should be added as the specification part.

[gioppoluca]

+1 anyone is working on this?

[Somebi]

Too bad that there is no news about it. Will have to write it manually like a simple html doc or something like that...

[darrelmiller]

@gioppoluca @Somebi To my knowledge, no. There are lots of other HTTP specific issues that we are working on first.

To be honest, I don't really understand why someone would want to take a specification designed for describing a HTTP API and then try and use it to describe a non-HTTP API. The existing HTTP constructs are just going to get in the way of describing the websocket payload and if extensions were made to support websocket description it would just add complexity for those people trying to describe HTTP APIs.

It would seem to make more sense, to create a new spec for websockets. Perhaps someone could fork the existing spec and create a variant designed specifically for websockets.

[casualjim]

why is http different from websockets? Isn't swagger just a way to describe a contract for an API that just happens to default to the http transport? the message is

method, path, version headers body

sent over tcp

I can do that with many different transports. This message format is almost a 1:1 match amqp for example.

[darrelmiller]

@casualjim HTTP isn't a transport, it is an application protocol. The HTTP methods have semantic meaning, as do the status codes and the HTTP headers. Sure, if you want to re-implement HTTP message semantics over web sockets, then you can use Open API to describe it. But then what is the point of using Web Sockets. If you try and reuse HTTP messaging then every time you try and take advantage of the benefits of Web Sockets you are going to run into unspecified areas of HTTP. Web Sockets exists in part to enable bi-directional communication which isn't supported in HTTP. Web sockets is effectively a transport and requires you to build an application protocol that will take advantage of the capabilities it enables.

WSDL was a way to describe a contract for an API that was independent of transport. OpenAPI isn't.

[casualjim]

Interesting, because I thought that webhooks etc all serve as the bi-di side. So squinting a bit you can get the bi-di communication flow using various http supported additions, as we can see in a number of other issues.

https://github.com/OAI/OpenAPI-Specification/issues/770 https://github.com/OAI/OpenAPI-Specification/issues/716 https://github.com/OAI/OpenAPI-Specification/issues/396

[darrelmiller]

Webhooks require an explicit step where a call back URL is registered and the return call is standard request/response. It is publish/subscribe convention that has been layered on top of the HTTP semantics.

HTTP/2 push is not bi-directional, it is simply a server initiated prediction of a future request and piggybacking one response on another. Again in that issue I argued against any native Open API description of it.

SSE is very much a new application protocol that can be squeezed into HTTP using long polling.

Personally, I don't believe SSE payloads should be natively described by OpenAPI. This is consistent with the fact that in the callbacks PR there is no specification of payloads for event identification and subscription formats. I think standardization of WebHooks would be a great thing, I just think that standardization effort should be independent of the OpenAPI spec.

These are just my opinions, I'm not making any claims about OpenAPI will do in the future. I'm just saying what I think is best for the long term success of OpenAPI.

I realize that there are lots of valuable features that could be added to OpenAPI, but scope creep can be a real problem in spec design.

[pouyanh]

+1

[rberger]

We have a service that combines REST and Websockets. They work together and are part of a single application.

For us, if there was just a way to add a new operation, like message we could use OpenAPI to describe the main aspects of our web socket interface. It would get us at least 90% of the way there as we could describe the body of the message, its use, etc.

[darrelmiller]

@rberger What kind of information would you like to describe in an operation that you currently cannot do?

[Somebi]

@darrelmiller Isn't websocket built on top of http? Handshake is implemented on http level if i'm not mistaken.

[fehguy]

Only the negotiation

[Somebi]

We cannot create one json document where http and websocket will be described. Have to generate http spec separately and then modify and fill it already at UI/docs stage... Would be nice if docs generate one spec for http and websocket.

[MaksimOrlov]

Agree with @Somebi, it would be useful feature. My projects on Spring framework, usually, have two endpoints for the same action: one with ReqeustMapping (http) and one with MessageMapping (websockets). It would be very nice if I had both together in generated docs.

[EronWright]

May I suggest a limited goal of supporting the WebSocket handshake interaction, which is HTTP. I'd like to specify that a certain operation initiates a websocket connection. The initial handshake can be understood as a normal REST request with additional request headers, resulting in 101 Switching Protocols. It would be nice to articulate that at a high-level way rather than in a literal way.

I see a few open issues that mention websockets, but all seem to relate to post-handshake communication.

[dansiviter]

Until someone comes up with a way to describe a bi-directional socket (STOMP, MQTT, etc., over WebSockets might not be too bad, but gets messy with custom transports and protocols) a way of permitting and handing off well formed Upgrade requests would IMO satisfy most use-cases. Message validation on the socket would just have to fall to the app level. Gateways such as Tyk and Barracuda just hand-off WebSocket connections post checking of the Upgrade so could OpenAPI do the same?

[MaksimOrlov]

There are callbacks that were introduced in swagger spec. 3.0.

"A typical example of a callback is a subscription functionality – users subscribe to certain events of your service and receive notification when this or that event occurs."

May be that will help to describe STOMP or other endpoints.....

[dansiviter]

'callbacks' appear to be to get events on the occurrence of an API call, not a Messaging Protocol working over Socket transport.

If 'callbacks' we're a mechanism to invoke another service to validate the shape of the traffic, we'd be halfway there, but they do not appear to be. Plus they'd also ruin the performance of a socket connection by adding in request/responses.

[MaksimOrlov]

Yes, I agree with you, callbacks were not designed to describe web socket endpoints, but with callback can be described any async call from api, (that is what STOMP does, your app subscribe on endpoint and then receive message or app sends directly messages to different endpoints). That it seems to be little bit better then nothing, doesn't it?

[dansiviter]

I need low latency, so I need WebSockets; ReST is not suitable.

[darrelmiller]

@dansiviter Have you looked at https://www.asyncapi.com/ ?

[dansiviter]

@darrelmiller That's a great shout. Some link/merge between the two would be ideal.

[slavafomin]

I believe that usage of both HTTP REST APIs and WebSockets are quite common in modern web applications, especially with the popularity of highly interactive and dynamic SPA applications (Angular/React).

From the developers perspective it would be extremely useful to have a single instrument (standard/specification) to define both styles of our web APIs because we are working with a single domain model and it would be great to share common properties, schemas and definitions (e.g. OAS components) and to not duplicate them in two different specifications. Open API itself is a very good effort to standardize the documentation and modelling processes of modern web APIs and it would be a good pragmatic step to take WebSocket (or it's abstraction) under the hood as well. Maybe with the separate specification, but with the ability to share the common domain model.

[boeboe]

@darrelmiller this is indeed exactly what the industry needs! The reality (one that the maintainers of the OpenAPI spec seems finding hard to acknowledge) is that most API's out there are a mix between synchronous REST + synchronous Pub/Sub. The addition of web-hooks is actually only a small commitment to acknowledge that reality, not nearly enough to cut the deal.

In the meanwhile, many companies are maintaining in-house developed solutions to cover this obvious gap.

Any plans to integrate https://www.asyncapi.com in a v4 of the OpenAPI specification?

[MikeRalphson]

@boeboe

The reality [...] is that most API's out there are a mix between synchronous REST + synchronous Pub/Sub

are there numbers available which support that? Possibly the range of API's currently being described by OAS is self-selecting out this majority, but I slightly doubt it.

Any plans to integrate https://www.asyncapi.com in a v4 of the OpenAPI specification?

Lines of communication are open between OpenAPI and AsyncAPI folk, but the exact nature of future collaboration is still being discussed.

@slavafomin

it would be great to share common properties, schemas and definitions (e.g. OAS components)

OpenAPI 3.x and AsyncAPI 1.x are very closely aligned today, and there should be no problem sharing common components such as schema objects between the two using external $refs.

[boeboe]

@MikeRalphson

In a microservice context with a high degree of spokes, REST is often abandoned in favor of losely coupled pub/sub. Currently PACT is adressing this issues on the testing part (PACT files are not really contracts like OAS, although they won't agree... it's really just a BDD tool).

Towards public facing API's I agree with your statement of majority. Given the fact that MS architectures are being deployed so often... there is truly a gap in the contractual space. One option is Avro schemas from kafka... but this would result in maintaining 2 contracts per microservice... one for sync, one for async.

Really curious on your view on this? Just can't really see a downside for OAS to also embrace async...

PS: I don't really have hard ball numbers to support this claim... but being a freelance consultant with a large network... the financial and telecom sector are all into MS archs... all having this (a)sync dilemma. PACT is one part of the puzzle... async support in OpenAPI the second... just my 2 cents of course.

[darrelmiller]

@boeboe The maintainers of the OAS do recognize there is an industry need to describe async/messaging APIs. We have spent a significant amount of time discussing how the OAI can help. AsyncAPI is a great effort that shares many concepts with OpenAPI and is sufficient for many people's needs.

The challenge that OAS faces is that currently it describes an HTTP API using HTTP semantics. Many messaging/async APIs don't use those same semantics. Blending the two sets of distinct semantics into a single spec might be the best solution, or creating two distinct specs that can reuse elements from each other may be the better solution. We have not yet been able to reach consensus on this.

Rather than rathole on the topic, we have been working on other hopefully easier challenges like alternativeSchemas, encryption/signing, overlays, client certificates and conformance testing. There is no lack of desire to address the needs of the async community, but at the moment, AsyncApi is doing a good job of that on its own and we have only so many hours in the day and so many hands.

Additionally, describing APIs that are not "REST" styled APIs is currently not in the charter of the OAI. Therefore we need to update our charter in order to bring this work into scope. Hence, the problem is more than just a technical one, it becomes an issue for all of our corporate members. We have done work towards the goal of broadening the scope of our charter, but as soon as you open this door, the follow-up question is how broad should it be?

[exapsy]

@darrelmiller Thanks for suggesting Async API, for us who want to support a specification with Websocket and asynchronous APIs.

Although, I second @slavafomin 's point that we need one tool that will describe our whole API instead of seperating our API into multiple products of multiple different tools made with different specifications. At one point it really gets troublesome and confusing to deal with more than one specifications for just one objective (API specification).

Let alone companies who may have the people to support that, but for a small team of people or individuals, it gets exponentially harder to support multiple specifications when we are in need for them, if we have to deal with many tools at once for doing one job.

Of course, you may have other priorities for now, and I suppose other developers would support much the decision of OpenAPI supporting Websocket specification implementation in the future. After all, its name is OpenAPI, not httpAPI. :P Thanks for your work nonetheless.

[kael-shipman]

Totally support this, and additionally support features to add RPC specs (refs #801). These two additions would make OAS a functionally complete documentation language for all modern API schemes except GraphQL, which is self-documenting and I don't see much of a need to include in the OAS. Would love to see some movement here.

[artem-korolev]

Hi. I think swagger may be just enough to describe websocket/subscription/notifications. In short: client behave itself as a server (so it can receive regular rest calls from server). How I see this:

1. create server schema for server operations
2. create client schema for client operations
3. Generate 4 stubs

Now server provide rest api endpoint, so client can call it using generated client code. Client can call "subscribe" function for example.

And client provide rest api endpoint, so server can use generated client code to access client operations (basically push notifications).

So all what is needed is support for a protocol (HTTP/2, or websocket). And generating server/client stub, which gonna support this flow.

[cromefire]

And client provide rest api endpoint

Doesn't work for browsers which can only act as a http client or websocket client. The same is true for phones and other clients that are behind a firewall/NAT (so most consumer devices) because they can only act as a TCP (which includes http & ws)/UDP client(/initiating party). Reverse tunneling for example would be possible for the latter but that's not really supported by most technology stacks (and far from ideal).

[philsturgeon]

Duplicate of https://github.com/OAI/OpenAPI-Specification/issues/55

[Jane-Zeng]

is there a limitation of url length of websocket protocol? or is there a limitation of url length of websocket client for code implementation ( for lwip)？