Feature: Add Support for AsyncAPI v2/v3 as Source Definition

[ivangsa]

Adding support for AsyncAPI definitions would extend the types of workflows that can be described with Arazzo, as many organizations are now combining Request/Response APIs (OpenAPI) with Event-Driven architectures (AsyncAPI).

The latest version of AsyncAPI is v3, but since v2 is still the most widely adopted version, the intention is to provide support for both versions as feasible, and prioritize v3 only when supporting both becomes impractical.

Required Changes to Arazzo Specification

• Add asyncapi as a source definition type.
• Extend the meaning of workflow.step.operationId to include pointing to an AsyncAPI operationId (either v2 or v3):
  • AsyncAPI v2 operationId
  • AsyncAPI v3 operationId
• Extend the meaning of workflow.step.parameters.in:
  • header should also include AsyncAPI message headers.
  • path should include AsyncAPI channel.parameters, as they are similar to OpenAPI path parameters (or consider adding a channel value for disambiguation).
• Runtime expressions already include support for $message.body and $message.header this would be used for context in successCriterias

Additional Considerations for Broker-Intermediated APIs

While AsyncAPI and OpenAPI are similar, and it’s tempting to use the operationId in AsyncAPI, there is a key difference: broker-mediated APIs are reciprocal (publishing/sending on one side is subscribing/receiving on the other).

• Sometimes, an API might only document one side’s perspective. For example:
  • A service that publishes events might document its "publish" operations.
  • However, it might not document the "subscribe" operation that consumers would use to receive these events (consumers just use the "publish" operation documentation and tooling).
• Teams and organizations may model this symmetry differently, and sometimes the "inverse" operation may not be defined at all.

To support this, we would need an alternative way to reference the reciprocal operation.

Additional Changes to Arazzo Specification

When the desired operationId is not documented in the existing AsyncAPI, we can support referencing the channel instead of the operation and specify the reciprocal action (send|receive).

• Add workflow.step.channel as a fixed field pointing to an AsyncAPI channel:
  • AsyncAPI v2 channels
  • AsyncAPI v3 channels
• Add workflow.step.action with supported values send|receive.

The combination of channel+action would be mutually exclusive with operationId, with operationId being the preferred option when available.

Meaning of AsyncAPI Steps

Send/Publish

A workflow step pointing to a Send/Publish operation will send a message with the workflow.step.requestBody.

workflow.step.parameters will also be used to populate AsyncAPI message.headers and channel.parameters.

The only meaningful successCriteria is "OK, message accepted," so it may be omitted.

This action is non-blocking and has no outputs; it will immediately continue to the next step, either onSuccess or onFailure.

This is an example of a workflow step sending a message:

  - stepId: "send_order_received"
    operationId: "$sourceDescriptions.OrdersAsyncAPI.onOrderEvent"
    requestBody:
      contentType: "application/json"
      payload: |-
        {
            "id": "{$steps.place_order.outputs.orderId}",
            "orderTime": "{$steps.place_order.outputs.order.orderTime}",
            "status": "RECEIVED",
            "customerDetails": {$steps.fetch_customer_details.outputs.customer},
            "restaurantDetails": {$steps.fetch_restaurant_details.outputs.restaurant},
            "orderItems": [
            {$inputs.menuItem}
            ]
        }
    onSuccess:
    - name: "goto_wait_for_downstream"
      type: "goto"
      stepId: "wait_RestaurantsStatusChannel_accepted"

Receive/Subscribe

This is a blocking action: the workflow will wait until successCriteria is met or it times out.

In the following example, in the wait_KitchenOrdersStatusChannel_accepted step, the workflow would "wait until a message is received where the message’s body contains a customerOrderId that matches the orderId of the previous step’s output."

  - stepId: "wait_KitchenOrdersStatusChannel_accepted"
    channel: "$sourceDescriptions.RestaurantsOpenAPI.KitchenOrdersStatusChannel"
    action: "receive"
    successCriteria:
    - condition: "$message.body.customerOrderId == $steps.place_order.outputs.orderId"

This step can include outputs using the contexts $message.body and $message.header

Support for Parallel Invocation: Forking and Joining

When using non-blocking steps, there may be a need for parallel invocation, involving forking and joining.

Parallel Invocation and Looping can also be applied to OpenAPI Request/Response steps.

Because of the complexity of this, we could address this in a separate issue for further discussion.

[frankkilcommins]

@ivangsa thank you for the very thoughtful proposal. Below I'll aim to process the proposal, add comments/remarks, seek clarification, or acknowledge the suggestion.

General Assumptions:

The latest version of AsyncAPI is v3, but since v2 is still the most widely adopted version, the intention is to provide support for both versions as feasible, and prioritize v3 only when supporting both becomes impractical.

Acknowledge - we should strive to support both unless we prove that impractical.

High-level changes to the Arazzo Specification:

Add asyncapi as a source definition type.

Acknowledge - this is as expected.

Extend the meaning of workflow.step.operationId to include pointing to an AsyncAPI operationId (either v2 or v3)

Acknowledge - this is as expected.

Extend the meaning of workflow.step.parameters.in:

• header should also include AsyncAPI message headers.

Acknowledge - this is as expected.

Extend the meaning of workflow.step.parameters.in:

• path should include AsyncAPI channel.parameters, as they are similar to OpenAPI path parameters (or consider adding a channel value for disambiguation).

I have a preference for a new channel value to be explicit but others are welcome to share their preferences too and we'll work towards one of these options.

Runtime expressions already include support for $message.body and $message.header this would be used for context in successCriterias

Acknowledge. These are being removed in 1.0.1 for hygiene purposes, so this would involve adding them back in and reestablishing the text and construct of the Runtime Expressions to support messages.

Dealing with reciprocal pub/sub scenarios when only one side is documented

When the desired operationId is not documented in the existing AsyncAPI, we can support referencing the channel instead of the operation and specify the reciprocal action (send|receive). Add workflow.step.channel as a fixed field pointing to an AsyncAPI channel

To follow the pattern used for OpenAPI, this might be more applicable as mutually exclusive construct of a channelId (v3) and channelPath (v2)

Add workflow.step.action with supported values send|receive

Do we have any opinions on the choice of send|receive v publish|subscribe in this context? Are there pros/cons to the nomenclature choice?

The combination of channel+action would be mutually exclusive with operationId, with operationId being the preferred option when available.

Acknowledge - if we introduce this (or channelId / channelPath) they would be mutually exclusive with operationId and we'd clarify recommendations.

Remark: Generally in Arazzo we have a goal to be as deterministic as possible and to not make assumptions, we rely on the underlying API descriptions to express what is possible. If we are to go down a path allowing workflows to be described across omitted parts of an AsyncAPI description are we asking ourselves of trouble in the long run. Will this cause challenges for tooling w.r.t. validations/parsing OR is it an undisputed given that the channels always express both the ability to send and receive? I would like some further input from the AsyncAPI community/core team on this. Albeit laborious to enforce authors/providers to document both sides of the coin, perhaps that's the only way we can be sure what's possible. Will we also have to make assumptions on message schemas / parameters etc?

Step and Messages - Send/Publish

A workflow step pointing to a Send/Publish operation will send a message with the workflow.step.requestBody.

Acknowledge - this is as expected.

workflow.step.parameters will also be used to populate AsyncAPI message.headers and channel.parameters.

Acknowledge - this is as expected.

The only meaningful successCriteria is "OK, message accepted," so it may be omitted.

It would be good practice to specify the criteria and assuming a HTTP status code will be return, it can be set (especially if you want to handle onSuccess or onFailure flows)

This action is non-blocking and has no outputs; it will immediately continue to the next step, either onSuccess or onFailure.

I assume if no onSuccess is described, then the next sequential step is what we continue with.

Step and Messages - Receive/Subscribe

What is described makes sense but I am wondering would we benefit from an ability to set a wait/timeout?