Support request/reply pattern

adrianhopebailie commented 5 years ago

Somewhat related to:

It would be useful if it was possible to describe messages that are explicitly requests and responses and for the auto-generated code to deal with creating the appropriate ephemeral queues and performing matching on the correlationid.

The pattern that seems most common when using a pub-sub message broker is for the requestor to create a single use topic and provide this address as the 'reply-to' header in the request message. The requestor also provides a correlationId which is echoed back to help match requests and replies.

However, when using a transport like WebSockets it would be necessary to do additional work in the generated code to match requests and responses and also deal with message state and expiry.

Ideally this should be abstracted away from API designers who may prefer to define their API in a manner similar to Open API as follows (see the Responses Object - https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#responsesObject) :

topics:
  UPDATEARTICLE:
    request:
      headers:
        type: object
        properties:
          ...
      payload:
        type: object
        properties:
          ...
     reply:
       match-on: {$response.headers.type}
       responses:
         'success':
           headers:
             type: object
             properties:
               ...
           payload:
             type: object
             properties:
               ...
         'fail':
           headers:
             type: object
             properties:
               ...
           payload:
             type: object
             properties:
               ...

I've used the topics object but maybe a new operations object would be more appropriate?

One of the challenges here is the flexibility of having multiple possible responses but also defining the logic for identifying what response has actually been received. (Open API matches on HTTP response code so that's pretty simple).

In my example I just provide a matching rule but this could probably be a lot more flexible.

fmvilas commented 5 years ago

Thanks for the suggestion @adrianhopebailie. I think we'll need to somehow support the request/response pattern in the future. We need to figure out how to properly design it so it ages well in the spec. Sorry for the short answer, this feature can't be added quickly without adding so much complexity to the current spec status. I'll have a look in detail as soon as I have time.

SensibleWood commented 5 years ago

+1 to this from me. Common pattern on messaging platforms. Done many such implementations on WebSphere MQ in a former life. As mentioned above, #78 would be fundamental to this. Something in the style of a Callback Object would be a good approach.

mpe85 commented 5 years ago

Also +1 We are interested in support for the new request/response pattern introduced in MQTT 5.

fmvilas commented 5 years ago

Interesting, I haven't heard about MQTT 5. Going to take a look. Thanks for commenting!

SensibleWood commented 5 years ago

Doing the investigation on this for a post v2.0.0 solution

prdatur commented 5 years ago

Hi there, we are starting to use async communication more and more and we have both variants in use, the basic request/response pattern where we send an responseEvent parameter like the mqtt corrId, on incoming data it will do stuff and send the wanted information back to the responseEvent.

Also we use the publish/subscribe pattern were we need it.

So we are very intereseted to get this into the definition.

For me the most flexible way would be just a combination of

subscribe:
publish:

I think a flexible and good way would be to name subscribe as mentioned "request" Within the request there will be a response: which is also defined as a operation obect

https://www.asyncapi.com/docs/specifications/2.0.0-rc1/#operationObject

For example:

channels:
  get-last-messages:
    request:
      summary: Get the latest messages from the user
      message:
        payload:
          responseEvent:
            type: string
      response:
        message:
          payload:
            messages:
              type: array
              description: A list of messages
              items:
                type: string
                description: The message.

If we assume errors / success status, we could just create a template by our own defintions which we can $ref in a, for example, called "status" property

status:
 $ref: '#/components/schemas/responseStatus'
content:
  type: object
  properties:
    resultData:
      type: string

With the use of the operation object we would not get into trouble in my opinion, as the definition should cover all feature variants of communication, so the operation object will be mutated and extended in future versions of asyncapi and that will extend also the response type.

What do you think?

jdall commented 5 years ago

The suggestion of @prdatur matches pretty well with our thought (except we would expect the response/reply object on the same structural level as the request object). One thing that we think is also required, is to state whether the application should act as a request/reply producer or consumer - i.e. whether the application would send requests, or would receive requests and send replies (such detail could maybe be specified in the binding object for the channel)

ig0rsky commented 4 years ago

hey, any update on this?

fmvilas commented 4 years ago

It's on the list for the next minor versions. Any research on how this could be done is appreciated.

basickarl commented 4 years ago

Interested party here also for request/response pattern!

basickarl commented 4 years ago

@fmvilas Any draft available for a sneak peek?

fmvilas commented 4 years ago

Hey @basickarl! There's no literature on this yet. Anything you can provide as a starting point would be greatly appreciated.

jruizaranguren commented 4 years ago

Request-Response in the context of MQTT 5.

There are request and response topics.
Both sides are subscribed to those topics.
Requests carry information about the response topics.
You can add correlation data so response can be bound to request.

basickarl commented 4 years ago

@jruizaranguren

Slightly different scenarios. The one in the MQTT has a broker. Client <-> Broker <-> Client. I'm specifically interested in the Client <-> Client scenario, e.g. for WebSocket connections from web clients to servers. Also for public facing WebSocket API's.

prdatur commented 4 years ago

@fmvilas based on your first examples which is a way we can go. https://github.com/asyncapi/asyncapi/blob/b21cf1d854994a8488c6575ae0dda461b51eb1d4/examples/2.0.0/rpc-server.yml

I have one question or one change request. Was is meant with '{queue}' I know that this is the response queue which is defined in amqp within the replyTo header but in the example we would only have one response queue because the string '{queue}' can only occure once.

We should just name the reply queue within the request definition like adding a new field before "correlationId" which is named "replyTo" of type string, this string can hold an existing channel which defines the response data.

Another way would be a variable as the channel, in this case a generator or user will not be confused to check wether the channel is just a response or actually a channel which the client can subscribe. For example defining the same way as within your example

channels:
   '{$rpc_queue.publish.message.header#/replyTo}':
      subscribe:
         ...

So this channel would define the response channel which is located within the rpc_queue message header field "replyTo". The definition will be a subscribe operation.

Third option is to define that if publish and subscribe exists within the same channel declares that this channel is a request / response operation.

A self subscribing channel where the software will publish data would be a bit confusing :) So we can use this for the request response definition. Additional we could create the "replyTo" field to let generates know where to find the reply queue name.

What do you mean?

fmvilas commented 4 years ago

I think that opening the door for expressions in the channel name would open a door for potential complexity but we will explore it.

...because the string '{queue}' can only occure once.

{queue} is a channel item variable, meaning that it can be anything. It doesn't mean that the queue should be exactly {queue}.

Does it make sense?

prdatur commented 4 years ago

{queue} is a channel item variable, meaning that it can be anything. It doesn't mean that the queue should be exactly {queue}.

I understood that this is a variable. While I was writing an example what I didn't understand, I got your point. for example we can just write multiple '{queue}' or better for each a name like {sendSumResponseQueue} which then will be mostly the same definition like the one in your example.

However with this, we currently have two problems

First, the response is not linked to any operation.

After I used and wrote the word "operation" I might got a solution, why not link the operationId's... they must be unique per definition, so we could use something like responseOf: {operationId}

Second, linking such a response to a publish operation we are not able to know for which message the response is

Let me explain. Within the issue #303 I made an example. This is exactly what I mean here. Many systems just connect to one queue/exchange and publish an own structure, mostly something like:

{
   "action": "do-something",
   "data": "...."
}

This means, a schema will look like:

publish:
  message:
    oneOf:
    - $ref: '#/components/messages/message1'
    - $ref: '#/components/messages/message2'
    - $ref: '#/components/messages/message3'

If we have only one response channel, we will not know if the defined response is for message 1, 2 or 3. Also If we define 3 response messages, we will not know if the response message 1 will be published for message 1, 2 or 3.

So with a simple change to not link the response channel to publish operation, we should link a response to a message. In that way we never get in trouble where something can be published and we did not know what we get returned. If we link it to a message we really know this published message will us return message XY.

So the solution with channel expression would be one way. Other solutions:

Add messageId same ways as the operationId, then it would be possible to define within the message object something like responseTo: {messageId}
Just add "response" keyword to a message object which is another messageObject, infinity loop must be adressed, Would be not the ideal solution because we have no definition for the response channel.

I would prefer the expression, because it will not be a big change within the specification, but yes it will be more complex also for generators. Next would be the messageId and responseTo or responseOf solution. Mostly there will be just one channel we need to describe, because a system will mostly response in the same way as before. So there will be just a bunch of ref's within message.oneOf...

Example of messageId solution:


components:
  messages:
    responseSum:
      payload:
        type: object
        properties:
          result:
            type: number
            examples:
              - 7
    responseDevide:
      payload:
        type: object
        properties:
          result:
            type: float
            examples:
              - 7.5
    operationSum:
      bindings:
        amqp:
          replyTo:
            type: string
      responseTo: '#/components/messages/responseSum'
      payload:
        type: object
        properties:
          operation:
            type: string
            enum:
            - sum
          numbers:
            type: array
            items:
              type: number
            examples:
              - [4,3]
    operationDevide:
      bindings:
        amqp:
          replyTo:
            type: string
      responseTo: '#/components/messages/responseDevide'
      payload:
        type: object
        properties:
          operation:
            type: string
            enum:
            - devide
          numbers:
            type: array
            items:
              type: number
            examples:
              - [15,2]

channels:
  '{responseQueue}':
    parameters:
      responseQueue:
        schema:
          type: string
          pattern: '^amq\\.gen\\-.+$'
    bindings:
      amqp:
        is: queue
        queue:
          exclusive: true
    subscribe:
      bindings:
        amqp:
          ack: true
      message:
        oneOf:
        - $ref: '#/components/messages/responseSum'
        - $ref: '#/components/messages/responseDevide'

  rpc_queue:
    bindings:
      amqp:
        is: queue
        queue:
          durable: false
    publish:
      message:
        oneOf:
        - $ref: '#/components/messages/operationSum'
        - $ref: '#/components/messages/operationDevide'

Example for using just response within the message


components:
  messages:
    responseSum:
      payload:
        type: object
        properties:
          result:
            type: number
            examples:
              - 7
    responseDevide:
      payload:
        type: object
        properties:
          result:
            type: float
            examples:
              - 7.5

    operationSum:
      bindings:
        amqp:
          replyTo:
            type: string
      response:
         $ref: '#/components/messages/responseSum'
      payload:
        type: object
        properties:
          operation:
            type: string
            enum:
            - sum
          numbers:
            type: array
            items:
              type: number
            examples:
              - [4,3]
    operationDevide:
      bindings:
        amqp:
          replyTo:
            type: string
      response:
         $ref: '#/components/messages/responseDevide'
      payload:
        type: object
        properties:
          operation:
            type: string
            enum:
            - devide
          numbers:
            type: array
            items:
              type: number
            examples:
              - [15,2]

channels:
  rpc_queue:
    bindings:
      amqp:
        is: queue
        queue:
          durable: false
    publish:
      message:
        oneOf:
        - $ref: '#/components/messages/operationSum'
        - $ref: '#/components/messages/operationDevide'

Problem we have not defined the queue for the response.

The problem is, that the request/response is really needed by us now. So it would be nice if we can get this finished. :)

Anyway, as a real quick answer. Yes your explanation makes sense but we have no information which message will get response if someone publish to rpc_queue :)

prdatur commented 4 years ago

After a discussion with a co-worker, we will just use an x-responses property within the publich.message section which will include one or oneOf references to the response message (not the channel) example:

components:
  messages:
    responseMessage1:
      ...
    responseMessage2:
      ...
channels:
  '{rQueue}':
    ...
    subscribe:
      message:
        oneOf:
        - $ref: '#/components/messages/responseMessage1'
        - $ref: '#/components/messages/responseMessage2'
  pubQueue:
    ...
    publish:
      ...
      message:
        ...
        x-responses:
          $ref: '#/components/messages/responseMessage1'

Or if we have multiple responses like the example with ack, status, response

  pubQueue:
    ...
    publish:
      ...
      message:
        ...
        x-responses:
          oneOf:
          - $ref: '#/components/messages/responseMessage1'

This allows us to define all required things (channel, messages) and just link the message to the response message so the documentator for example place a link below one of the accepeted messages to the response message.

Maybe we can use this idea as the solution and just remove the x- part.

basickarl commented 4 years ago

Me and a co-worker had a chat and we would like to support several different actions on a "channel". for channel mychannel/{myChannelId} in our case we would like a publish/subscribe (with unsubscribe and acks when subscribing and unsubscribing, we had a look at mqtt 5.0 and followed their example), request/response and command (command doesn't expect a response).

If you guys head over to https://crossbar.io/ (responsible for the WAMP protocol) and look at their fancy gif on the home page a little further down, you'll see a chronological scenario of what we want to accomplish via asyncapi.

An example of how thew asyncapi could be updated to support it:

channels:

  mychannel/{myChannelId}/:

    command: ...
    request: ...
    response: ...
    unsubscribe: ...
    subscribe: ...
    publish: ...

And we can take it a step further and include ACKs (inspired by mqtt 5.0):

channels:

  mychannel/{myChannelId}/:

    command: ...
    command ack: ...
    request: ...
    request ack: ...
    response: ...
    response ack: ...
    unsubscribe: ...
    unsubscribe ack: ...
    subscribe: ...
    subscribe ack: ...
    publish: ...
    publish ack: ...

Again, take a look at the gif at https://crossbar.io/.

Regarding the mqtt 5.0 inspiration, comes from the following:

https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901121 https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901171 https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901187

(It feels like asyncapi would have to include these properties if it wants to help support mqtt 5.0?)

I also understand @prdatur approach but would as @fmvilas says let people do their own thing and increase complexity. With the suggestion I proposed it would add on to how asyncapi is already written, add structure and also help conform to mqtt 5.0.

jonaslagoni commented 4 years ago

{queue} problem

@prdatur cant figure out if this is a current problem you have or a suggestion for solution to request reply 😄

If it is a current problem

If it is a current problem is this not something that can be solved at the implementation level i.e. is there a use-case for this property to be present in the specification? If we take template clients they would look at your AsyncAPI file and (dependent on the implementation of course) create a method for your channel `'{responseQueue}'` where you can subscribe a callback (or whatever) and get notified when a response ticks in. The generated client then tries parsing the received data to one of the `oneOf` messages and call your callback with the given message. Then with the use of `correlationId` (or something else) you can cross check and figure out which response has returned?

If it is a suggestion for the request reply, then I gotta say I am more a fan of doing something like @basickarl suggests, at the channel level, which would provide complete clarity what is defined as request and reply in a given channel without having to go through the request operation first 😄

Request reply suggestion

channels:

  mychannel/{myChannelId}/:

    command: ...
    request: ...
    response: ...
    unsubscribe: ...
    subscribe: ...
    publish: ...
@basickarl isn't this a bit too specific for mqtt for it to be in a specification? Many other protocols just have request/reply and pubsub, which is what I see as the commonality between the protocols I have encountered.

However I also like the idea of adding request and reply properties to a channel i.e.

channels:

  mychannel/{myChannelId}/:

    request: ...
    response: ...
    subscribe: ...
    publish: ...

One question i'd like to raise: IF request/reply is not a feature we foresee as a standard feature for all protocols should it really be included in the specification and not just stay in the bindings as a boolean or a request/reply property for those protocols which do support it?

basickarl commented 4 years ago

@jonaslagoni As stated every protocol implements their own terminology for these things so I guess AsyncAPI will either have to add ALL of these terms to make everyone happy or put their foot down and define terms that everyone should adopt. These terms should however be able to satisfy what you wish to do with each protocol.

MQTT was just an example, and one of the specs in MQTT 5.0 was that when you send a subscribe message than it sends a subscribe acknowledgement back to who sent the subscribe message. If we wished to implement this today and specify it in a document AsyncAPI would fall short.

And answering your last question @jonaslagoni I don't see the harm in adding these things, you don't need to add it to your own AsyncAPI documentation, if you don't use request/response pattern in your API, just don't include it!

GreenRover commented 4 years ago

+1

basickarl commented 4 years ago

Would like to say we have forked 4 of the repos from https://github.com/asyncapi and have implemented the following:

command: ...
request: ...
response: ...
unsubscribe: ...
unsubscribe ack: ...
subscribe: ...
subscribe ack: ...
publish: ...
publish ack: ...

It's fufilling our needs pretty well.

fmvilas commented 4 years ago

Interesting. What's the meaning of each of them? We can probably incorporate them into future versions.

nei-l commented 4 years ago

Hi,

Has any progress been made with this? We too have a need to document more than just pub/sub. Whilst terminology differs slightly by protocol, personally I would favour using the convention of command vs event to distinguish, and then adding reply as an optional property on the message that it would be a reply to.

Although I favour the command and event convention, I don't necessarily think it should be enforced. I see this more as an optional specialization of a message that can be provided. My object oriented mind sees this as: -

I also find mixing publish and subscribe confusing. I personally don't see a necessity in documenting the subscription as it's the publisher that actually owns the message contract (and in a lot of cases the channel). Whilst there are likely benefits of documenting it in some scenarios I would like to see a clear distinction between an interface a service provides and those that it consumes, therefore I would suggest breaking out subscriptions as a new root.

As a suggestion, I could see this coming together as something like: -

channels:
  channelName:
    messages:
      - name: messageName
        specialization: event # optional, can either be event, command or not provided for use cases where distinction isn't helpful
        payload: ....
      - name: messageName
        specialization: command
        payload: ....
        reply:
          #a message object
consumes: # subscriptions moved into a new root property
  channels:
    channelName:
      messages:
        - $ref: '#/aSchemaProvidedByTheProducer'

You'll notice that messages is an array, as another issue that I have are certain scenarios where different messages are sent over the same channel. Although this isn't necessarily best practice, these situations do exist so I would favour support for them.

Whilst the proposal above changes the structure somewhat, I don't see why it couldn't be implemented in a non-breaking way with messages being introduced optionally alongside publish/subscribe. Having some sort of specialization concept also allows flexibility for new types should they be needed.

There are many different ways that this could be represented and whilst I would favour something similar to the above, any non-breaking change to the spec that would allow documentation of messages that are not pub/sub (so they don't have pub/sub next to them in the output) would be extremely helpful.

basickarl commented 4 years ago

@fmvilas

Interesting. What's the meaning of each of them? We can probably incorporate them into future versions.

All acks work the same: this is sent to the sender to notify them that the receiver has received their x message. No acks are sent for acks. (we work on airport systems and we require acks to make sure things have been sent).

We have since my last post included acks in all types now.

command: ... <-- sender sends this and there is no obligation for the receiver to send a response to the sender request: ... <-- sender sends this and there is an obligation that the receiever will send a response to the sender response: ... <-- this is what the receiver of the request sends back to the sender of the request unsubscribe: ... <-- as it is now subscribe: ... <-- as it is now publish: ... <-- as it is now

the request response is mimicking traditional http requests, example would be a GET request. command is telling something to do something and does not need a response.

I am currently writing a library which is an async version of express which follows the asyncapi specification, it will take a while to release though. I'll update you guys when it's done.

kvaleev commented 4 years ago

@basickarl, may I suggest one more type:

event: ... <-- the same as command in terms of response but implies no command meaning and rather works as a notification.

basickarl commented 4 years ago

@kvaleev When I think of events I think that an event is broadcast meaning anyone who listens will receive it, and it feels like subscribe/publish pattern solves this? You are referring to 1-to-1 communication here (client <-> server) where both know about each other?

Events in general do not know about who will listen to them (think OS events, mouse click, the mouse has no idea who will consume the click, it just sends it out into the unknown and whoever cares about it will use it, hence subscribe/publish).

If you want to look at a full fledged example of different types of communication working together look at the gif animation on this page: https://crossbar.io/

The gif animation is what my proposal would fix. (in the gif animation someone sends a publish message type, which triggers an event message type, it's just naming differences)

jonaslagoni commented 4 years ago

Would like to say we have forked 4 of the repos from https://github.com/asyncapi and have implemented the following:
command: ...
request: ...
response: ...
unsubscribe: ...
unsubscribe ack: ...
subscribe: ...
subscribe ack: ...
publish: ...
publish ack: ...
It's fufilling our needs pretty well.

IMO this is too much since most protocols won't ever use these operations. subscribe ack and publish ack I feel like is more a protocol binding for the subscribe/publish operations then an actual operation? The command this is pretty much the same as a single publish operation or what would the difference be? :thinking: Agree with the request/reply operations :smile:

What would unsubscribe be used for @basickarl?

basickarl commented 4 years ago

@kvaleev Now when I think of it event could be added, which as the gif animation explains, would be the result of a publish message type coming into the server.

basickarl commented 4 years ago

@jonaslagoni I agree most probably won't use them, but not having them will block them that wish to use them. So I don't see the harm in adding more to cover more protocols. You don't need to use them if you don't want to basically.

Difference between command and publish is that the command is a 1-to-1 operation. A publish is a 1-to-* operation which also requires whoever to receive it do a subscribe before hand.

It took me a while to get my head around it all. Basically, what I am proposing is that the "server" acts not only as a broker (subscribe/publish/unsubscribe) but also as a normal backend server api (request/response/command). You can mix these however you wish! Look at the gif in the crossbar website I linked in my previous post, it's pretty dynamic.

Unsubscribe is for when someone does not want messages from a publish anymore (no more updates)!

prdatur commented 4 years ago

The whole ack, no-ack. subscribe, unsubscribe is protocol specific in my opinion and has nothing to do with asyncAPI, for example RabittMQ implements this by it self, you define this within the channel if an acknowledge event needs to be sent or not.

the "command" is basically just a server who subscribes. That's what asyncAPI already has included with the normal subscribe/publish.

In my opinion the only thing what we require is "request/reply" in a non complicated way. Which is just a link between subscribe/publish. So in event-based systems a request/reply pattern is just a pre-defined way to first define a reply channel, posting the message to the backend which subscribes the event and publish's back to the reply-channel the results.

jonaslagoni commented 4 years ago

@jonaslagoni I agree most probably won't use them, but not having them will block them that wish to use them. So I don't see the harm in adding more to cover more protocols. You don't need to use them if you don't want to basically.

If most protocols dont use them I dont agree they should be added to the core AsyncAPI specification, but rather reside in the operation protocol bindings. Adding anything to the core of the AsyncAPI specification just raises the complexity of the specification for everyone which I think is unnecessary if only 1 or two protocols use it, since this can easily be achieved with protocol bindings 😄

Difference between command and publish is that the command is a 1-to-1 operation. A publish is a 1-to-* operation which also requires whoever to receive it do a subscribe before hand.

I mean, adding request/reply operations does (IMO) not entail that they both have to be defined, i.e. you can have a request operation on a channel without the reply. Which is basically what your command operation are supposed to do right?

Unsubscribe is for when someone does not want messages from a publish anymore (no more updates)!

For me this sounds more like an actual channel then a special operator? 😄

basickarl commented 4 years ago

@prdatur Regarding protocol specifics some protocols do not have acks in-built (websocket). The asynchronous communication in the internet today is a mess, one, giant mess! There are no standards regarding API/server interfaces anywhere and I would like to think AsyncAPI could play a role in standardizing this just like OpenAPI has standardized synchronous API's.

basickarl commented 4 years ago

If most protocols dont use them I dont agree they should be added to the core AsyncAPI specification, but rather reside in the operation protocol bindings. Adding anything to the core of the AsyncAPI specification just raises the complexity of the specification for everyone which I think is unnecessary if only 1 or two protocols use it, since this can easily be achieved with protocol bindings

Then subscribe/publish should be removed since WebSockets doesn't have to use them (1-to-1 communication). Then what is the point of AsyncAPI? I think this is more of a question of which scope AsyncAPI should address.

I mean, adding request/reply operations does (IMO) not entail that they both have to be defined, i.e. you can have a request operation on a channel without the reply. Which is basically what your command operation are supposed to do right?

So how would you know as a sender that you will receive a response? How do you know if something has gone wrong (timeout)? In HTTP you always get a response 200 for success, 404 for not found etc. This is the response. Having a request without a response will be extremely confusing for people implementing this and those coming from a HTTP background. I would like to think that AsyncAPI will make things easy to understand for people coming from a sync/OpenAPI/HTTP background. It should also cover all protocols it has stated and cover most likely scenarios that the protocols will be used. This is the only way AsyncAPI will gain adoption.

For me this sounds more like an actual channel then a special operator?

I'm sorry I don't quite follow? When you subscribe to a broker (say MQTT) the broker will forward the published message to those on that channel (subscribed to that channel). If a client no longer wishes to receive published message on that channel they leave the channel and they do so by unsubscribing. See more here: https://www.hivemq.com/blog/mqtt-essentials-part-4-mqtt-publish-subscribe-unsubscribe/ In WebSockets you need to implement this yourself, ontop of the WebSocket protocol, see Socket.io as an example implementation.

basickarl commented 4 years ago

I feel like the ack discussion is it's own one and the request response is another.

jonaslagoni commented 4 years ago

Then subscribe/publish should be removed since WebSockets doesn't have to use them (1-to-1 communication). Then what is the point of AsyncAPI?

Don't see subscribe and publish as a 1-to-1 or 1-to-* operation. They can be both depending on the context. If you use AsyncAPI for WebSockets then it will be 1-to-1 (unless you implement it to handle 1-to-*). When used for brokers they do indeed become 1-to-*. All this depends on the protocol you use it for. Merly see the publish and subscribe operations as what messages are being published and what can be received, it needs context to be any more specific IMO.

It should also cover all protocols it has stated and cover most likely scenarios that the protocols will be used. This is the only way AsyncAPI will gain adoption.

Completely agree, there is just a very fine line in what is protocol specific and what should be included in the core specification. With all the protocols around it gets more and more complex, so I am just trying to poke holes in ideas and provide alternatives 😄

So how would you know as a sender that you will receive a response? How do you know if something has gone wrong (timeout)? In HTTP you always get a response 200 for success, 404 for not found etc. This is the response. Having a request without a response will be extremely confusing for people implementing this and those coming from a HTTP background. I would like to think that AsyncAPI will make things easy to understand for people coming from a sync/OpenAPI/HTTP background.

Yeah you right, suggesting to use request only makes little sense 🤔

I don't see the issue with implementing acks, if your protocol has acks inbuilt, then just don't use that part of the specification. It's like Request/Response, if you don't need it, don't use it. The reason I would like acks to be put in because it is an obvious thing people want.

But if it's only used in one protocol, why force all the users of AsyncAPI learn about the acks. This is exactly what the ws bindings are for to keep what is protocol specific aside from the core spec and if users use websockets, they need to read up on the bindings as well.

I'm sorry I don't quite follow? When you subscribe to a broker (say MQTT) the broker will forward the published message to those on that channel (subscribed to that channel). If a client no longer wishes to receive published message on that channel they leave the channel and they do so by unsubscribing. See more here: https://www.hivemq.com/blog/mqtt-essentials-part-4-mqtt-publish-subscribe-unsubscribe/ In WebSockets you need to implement this yourself, ontop of the WebSocket protocol, see Socket.io as an example implementation.

Yea but this happens at runtime i.e. when you have subscribed to a topic, you might get a Subscription object back where you can call subscription.unsubscribe(). The same goes for Socket.io. Now you want to implement this yourself in WebSocket using AsyncAPI to specify the messages being send when unsubscribing right? Idk, might be wrong, but for me this sounds like one of the existing operations or request/reply are meant for specifying it 🤔 ?

I feel like the ack discussion is it's own one and the request response is another.

Yeah you right, a bit off topic, can you create another issue for it @basickarl? We can keep discuss it there instead 😄

github-actions[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 30 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

bkubiak commented 3 years ago

Hi, what are the conclusions here? Do you plan to add this feature in the near future?

ltressens commented 3 years ago

Hello,

Very interested in documenting request/reply along with classic publish subscribe topics ! From what I understand of your previous message @fmvilas is that there is no will to support request/reply natively in AsyncAPI, but to tweak the usage of classical publish and subscribe operations documentation? Am I right ?

Our full need is :

We are writing message driven micro-services (they implement request/reply) in PHP
we would like to document them using PHP doc comments
we would like to use Scribe (https://scribe.readthedocs.io/) to generate the AsyncAPI yaml output from the PHP docs (we would contribute to Scribe to this purpose, but as they already generate the OpenSpec API yaml, it won"t be too difficult)

Thanks

Lionel

derberg commented 3 years ago

I have a feeling that the discussion here went too far into details looking at all those different proposals for different operations. The spec does not say that subscribe operation is only for messages that will subscribe the user to a channel. Also the spec doesn't enforce/say that publish operation is just for fire and forget use case. We have 2 operations, for messages that you send to server and receive from it. Sometimes they are not related but sometimes they are (especially in Websocket, but not only) so you get a message only in response to a message that you sent. To me it looks like just semantics issue and better would be to explore a lightweight approach from @prdatur where you have an optional property to reference message that you get in response when you send a message (this could be a Message object or Reference object and oneOf would be allowed)

jjimenezroda commented 3 years ago

Not 100% agreed. Semantics are important here. There are multiple communication patterns available in message-driven systems. Pub-sub is just one of them and this is the one covered explicitly by the specificiation. Great. But Request-Reply has just different means and specifications should, in my opinion, not allow for implicit assumptions. Saying that something is publishing a message when what we really mean is that it is replying to a message sounds weird. I would rather explicitly support these definitions. I'm personally missing this a lot in the AsyncAPI. I was considering adopting AsyncAPI for a message-driven system we are building, but this is only covering one part of it. Back to the first point: semantics matter. In fact this is one of the reasons why we write APIs. Is not only about the message format, structure and contents. It is also about the behavioral aspects and semantics arround it. I would propose to reconsider adding this to the specification.

finswimmer commented 3 years ago

Hello everyone,

what's the state of this topic? Any ongoing process?

I would like to use the AsyncAPI specs as a documentation for the RPC. As it stands now, I can define how to trigger the remote procedure but there seems to be no way to document how a "response" (actually a reply) would look like.

fin swimmer

derberg commented 3 years ago

@finswimmer so far, from the perspective of the contribution process this topic is not even at Stage 0 because nobody clearly opt-in to drive to topic forward. So at the moment, it is just an open discussion.

I wrote few articles around AsyncAPI and WebSocket recently and faced the need for request/reply pattern in a WebSocket API that has just one channel with multiple messages. I might find some time in few months to be a champion here because this topic is super interesting for me, but I cannot commit to anything atm.

My current recommendation for folks is to explore solutions using specification extensions. Have a look at how I did it when I was exploring an API of cryptocurrency-related WebSocket API -> click here.

If anyone in this thread is ready to drive the topic forward to add a solution to the spec, please jump in. This is a community-driven project.

derberg commented 3 years ago

@basickarl Hi. could you update us, a year after you introduced different operations, how it works for you, how it evolved? I'm super interested as here and there people tend to say that spec should have more operations, but nobody ever did it in practice

derberg commented 3 years ago

folks! looks like we finally found a champion that wants to drive this topic forward and introduce proper changes in the specification to make this happen. Please join the conversation here https://github.com/asyncapi/spec/pull/594 so people from different fields and different experiences see the proposal and confront with their use cases

basickarl commented 3 years ago

@basickarl Hi. could you update us, a year after you introduced different operations, how it works for you, how it evolved? I'm super interested as here and there people tend to say that spec should have more operations, but nobody ever did it in practice

Unfortunately I left the company as I was there on a project basis! I do still support this though!

Hassen-BENNOUR commented 2 years ago

Hi folks, Unfortunately it seems to need many time to find the right solution.

We have many amqp request reply communication between services as many people today. We do it with spring integration and spring cloud stream. Without support of request reply i think there is no reason or possibility to use asyncapi now 😔

Keep in touch and hope a support for this soon.

So thank you for your amazing stuff.

autodidaddict commented 2 years ago

Just dropping in to point out that this issue started 4 years ago. If async API is never going to support request/reply, I'd rather know that now rather than investing in a format that will ultimately never suit my needs.

derberg commented 2 years ago

@autodidaddict thanks for taking the time to share your feedback, not many do.

AsyncAPI Initiative is community-driven, with all needed mechanisms in place that enable anyone to contribute, work on features openly in a transparent environment. Some changes in the spec require a driver (champion), just nobody yet drove it to the end. There was a great attempt by @smarek (https://github.com/asyncapi/spec/pull/594) but looks like he discontinued the work and we simply need someone else to champion this topic.

The problem is that many people come and say "it is needed" and make some generic statements on "need for request/reply", but nobody really takes time to share in detail their use cases. Without clear and documented use cases, individual maintainers of the spec are unable to pick it up and work on the topic, thus we look forward to contributors.

Some time ago I recorded this simple guide for spec contributors -> https://www.youtube.com/watch?v=QQYyGlMzJCc Maybe you will find it interesting and feel encouraged to contribute. I'm would be super happy to onboard here anyone how would like to contribute to the spec. Now is a great opportunity as the community started working on 3.0 release of the spec. Feel free to join the next call -> https://github.com/asyncapi/community/issues/330

asyncapi / spec

Support request/reply pattern #94

{queue} problem

Request reply suggestion