asyncapi / spec

The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.
https://www.asyncapi.com
Apache License 2.0
4.2k stars 269 forks source link

Recomendation of not deriving a consumer from a producer is problematic for most users #931

Closed ivangsa closed 8 months ago

ivangsa commented 1 year ago

https://github.com/asyncapi/spec/blob/next-major-spec/spec/asyncapi.md?plain=1#L41

I think this official recommendation introduces more problems to users than it really solves.

This is a very corner case that doesn't apply to the great majority of cases and it will send most users to a rabbit hole of problems that tooling (IDEs, generators…) are not prepared to assist them with.

If you can not derive a consumer from a producer, who are you writing the producer API for? A producer writing an API just for himself doesn't sound very useful...

Then users will need to:

That is already some extra work, but linking to external resources is very problematic for two reasons:

(I expect most users are going to be linking to a git http interface)


Regarding naming, I think the example is backwards: in this example you can describe the producer and naming should be fine for consumers as well. Using the API to describe what the exchanged messages mean, not what actions consumers will perform with them.

I think consuming an event/message is analogous to reading data from a REST API, you don't describe what every consumer will do with that data, just what the data is (and its meaning).


How is asyncapi tooling prepared to assist users with this?

github-actions[bot] commented 1 year ago

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

derberg commented 1 year ago

this change you are referring to was added because of this issue -> https://github.com/asyncapi/spec/issues/538

so it is all about the purpose of the AsyncAPI document. If you will create this document for your application with application generation in mind, you will make operationId and summary and other stuff sound and fit the app generation use case. Then if you try to reuse the same document to generate client, the same operationId may sound dummy.

options are to:

what do you recommend?

ivangsa commented 1 year ago

if this is just about the naming of the operationId for generation pourposes I find that the convention of using onXXX for events and doXXX for commands fits perfectly fine for both producer and consumers..

I find the following code perfectly fine to read and understand (which is what the other issue is worried about):

You are not reacting to an onCustomerEvent but asking the customerEventsProducer to 'do its thing' onCustomerEvent

image

but even IF someone find that code confusing or difficult to understand, I would much rather to live with that than creating N different APIs for the same message

because when you publish a DomainEvent/Message on Event-Driven Architectures there will be potentially many consumers.

For instance if you have 5 consumers you will end up with 6 different API definitions for that message, 5 of them identicals... and linking to potentially protected resources..

And editing, dereferencing and generating code with protected resources is a pain...

What worries me about this recomendation is its character of an official recomendation... because it will make it much much harder when advocating AsyncAPI and proposing "deriving client from provider solution" which is way simpler.

That is my main worry.

Maybe this calls for an article exploring both solutions pros and cons, I would be happy to contribute one of the sides...

ivangsa commented 1 year ago

hi @fmvilas, tagging you to join the conversation..

fmvilas commented 1 year ago

What about "summary" and "description" fields? From the recommendation you linked:

operations:
  onUserSignedUp:
    summary: On user signed up.
    description: Event received when a user signed up on the product.
    action: receive
    channel:
      $ref: "#/channels/userSignedUp"

We can't automatically assume that an opposite application exists by simply replacing receive with send:

operations:
  onUserSignedUp: # <-- This doesn't make sense now. Should be something like sendUserSignedUp.
    summary: On user signed up. # <-- This doesn't make sense now. Should say something like "Sends a user signed up event".
    description: Event received when a user signed up on the product. # <-- This doesn't make sense now. Should speak about sending an event, not receiving it.
    action: send
    channel:
      $ref: "#/channels/userSignedUp"

And also the last paragraph:

Aside from the issues mentioned above, there may also be infrastructure configuration that is not represented here. For instance, a system may use a read-only channel for receiving messages, a different one for sending them, and an intermediary process that will forward messages from one channel to the other.

Remember this is a recommendation. You don't have to follow strictly if that's not suiting your needs. We will anyway offer tools to derive one from another to make the process a bit easier to handle when needed.

ivangsa commented 1 year ago

I think that naming (summary and description) is a minor issue comparing with ending up with N+1 (cuasi)duplicated APIs referencing each other.

(For instance, following description fits both sides: summary: Event produced when a user signed up on the system.)

And the infraestructure problem when a channel is read-only is a very corner case that can be dealt individually ("With this API you can not derive a consumer because...").

What troubles me is that RECOMENDATIONS in the official documentation will affect conversations on this topic about the convenience or not of "deriving consumer from producer" without mentioning the list of (IMO big) CONS:

What about creating a blog post or page explaining PROS/CONS of each option instead?

ivangsa commented 1 year ago

by the way: by "deriving" I understand use the same API definition on the other side just inverting the meaning, not replicating it replacing "send" with "receive"

fmvilas commented 1 year ago

I'm definitely up for clarifying stuff on blog posts or documentation. It is a recommendation because we don't want to enforce how people should do it. If you want to derive it and it's fine for you, go ahead and do it. We just noted a really bad experience when doing it in the past. It usually leads to poor documentation because you have to be careful not to say send/produce/receive/subscribe because that introduces confusion. In a distributed architecture, every node is different, and usually, a consumer of your API is not just a consumer of your API but also a consumer of other APIs and a producer of events itself as well. So actually having an AsyncAPI document per node makes total sense. That said, we don't want people to repeat themselves. We encourage you to have another AsyncAPI file but to use $ref to the producer one for certain things like channels, servers, messages, etc. Just not for operations as they would be very specific about that node.

Maybe we should clarify that?

jonaslagoni commented 1 year ago

Also just wanted to jump in, because we discussed this subject in some of the past v3 meetings.

I agree that if you ONLY have two parties, where these two parties are the ONLY ones who interact with each other, then yes, it becomes slightly cumbersome for that use-case, i.e. the recommended way is to define two AsyncAPI documents.

But it does not outweigh the benefits in my eyes. What's hard is if you are coming from using OpenAPI, where you have a single file, explaining both the server and the client behavior or rather what the client can perform on the server, it will be a bit of a learning curve.

What makes this setup amazing is you can have a server exposing HTTP (or any other protocol that exposes an endpoint) endpoints, while it also interacts with Kafka as a producer (or consumer), all defined within a single AsyncAPI document, i.e. it defines the behavior of that server.

Then you can have an AsyncAPI document explaining the external application and how it may interact with the HTTP endpoint in the above server. It of course does NOT need to know about the Kafka interaction, because it only interacts with the HTTP endpoint.

Taking that a step further, if you have two users of your endpoint (admin and user for example), they can each have their own AsyncAPI document, explaining how they interact with the server.

Starting to mix in deriving sets of actions from AsyncAPI documents that describe the behavior of someone else just creates the same set of confusion that publish/subscribe was, it's the whole reason v3 was put into the world (among other things of course).

It's true that tooling MUST be there to support this setup, but I think it will be without a doubt. I am definitely not saying it's a silver bullet, but I personally would say this setup is better than what we had in v2. But people might also say I am slightly biased πŸ˜†

This is after I started integrating the v3 structure into my projects.

  • Referencing remote resources or local duplication
  • Remote resources may be protected with 2FA

This has nothing to do with the structure changes in the spec? It's a general problem also for v2. v3 might just emphasize the problem even further. Tooling will get there, faster the more people help with providing tooling πŸ™‚

Every consumer will need to write their own API definition even for the topics that are already published and 'owned' by other services.

They do not, channels can be reused (i.e. you can figure out the ownership of topics how you wish), only actions are application specific πŸ˜„

Dupplication effort: N+1 APIs

Just emphasizing the first section I wrote, I agree that if you ONLY have two parties, where these two parties are the ONLY ones who interact with each other with no exception, then yes, it becomes slightly cumbersome for that use-case to be described with AsyncAPI.

But as @fmvilas said, no one is stopping you from deriving meaning as you see fit πŸ™‚

ivangsa commented 1 year ago

What if you can describe all the event-driven interactions of an whole company with AsyncAPI v2 or v3 without any duplication or external http/reference between API files?

Well, you can!

External http references are trouble: for versioning and for authentication.

And what is troubling me about the official recomendation (which is written as NOT RECOMENDED in capital letter) is the following:

-> It will make it harder to explain to unaware clients that you don't actually need external references at all...

jonaslagoni commented 1 year ago

What if you can describe all the event-driven interactions of an whole company with AsyncAPI v2 or v3 without any duplication or external http/reference between API files?

Well, you can!

Can you clarify what you mean here?

Cause to me this does not sound like something you can do with a single AsyncAPI document (regardless of v2 or 3), regardless of whether you derive the opposite behavior of an application. Some have an AsyncAPI document as a lookup of all common definitions, but that's the closest you come to this?

Cause what exactly are you describing then? πŸ˜„

Or do you see it as you have a mono-AsyncAPI document, that contains the behavior of all underlying applications within your company/system? i.e. when you then generate code, it's basically a library every application should use, even though it contains far more operations then what a single application should be able to perform?

External http references are trouble: for versioning and for authentication.

Definitely agree yea πŸ‘ And tooling are not at all where it should be to support these use-cases (hopefully this year, lets see πŸ˜‰)

ivangsa commented 1 year ago

Yes, I would like to write a small blog post for this, but this is how it goes in a nutshell:

Now, for code generation, when an application wants to interact with 3rd party topics/messages it will ask the code generator to generate a client for that 3er party API. (With this you have all the applications connected)

Naming conventions:

(Async data request can be modeled as a Command+Event like doRequest/onResponse)

And that's it. I implemented this on a quite large company since early 2020 and I believe it's still in production.. And this is my recommended aproach.

fmvilas commented 1 year ago

I'm a bit lost now. Are you talking about a new format you invented on top of AsyncAPI? Can you come up with a simple example so that we're all aligned?

which is written as NOT RECOMMENDED in capital letter

Well that's because we're following RFC2119 syntax. It's in the beginning of the spec:

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

That's why is in all caps, to make it visible that it's part of these special words. In particular, NOT RECOMMENDED is defined as follows:

SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

Which I think it's perfect for that case. We're not saying you can't (for that we use MUST NOT), we're saying that we don't recommend it but you're free to still do it if it's the preferred way in your case.

External http references are trouble: for versioning and for authentication.

I also agree but it's mainly a problem with tooling. It's tooling that should catch up, not the other way around. Can you imagine that someone suggests that the <table> tag is dropped from the HTML specification because it's usually troublesome? It used to be like this back when all websites were prototyped using tables. Frontpage, Dreamweaver, and many others were exporting your designs as tables, even though we knew that wasn't the right way. Fortunately, new tools appeared and the mentioned ones updated the way they worked, so the problem disappeared. IMHO, that's the way, we should improve our tools to deal with external references in a better way but even with the current state of art, it's still advisable to do it like this in the majority of cases. And of course not forget about those who are limited by this and keep improving the tools.

ivangsa commented 1 year ago

I'm talking about proper AsyncAPI definition files, nothing special on top..

One AsyncAPI file per application describing only the topics/messages this application is the provider (<- important role) of the funcionality: publish events or consume commands (not consuming events or producing commands). Each application documents just "their owned events/commands".

When applications need to consume 3rd party events or produce 3rd party command they are acting as client of that API, they don't need to create any API definition for that. Just use the existing one but from the role/perspective of a client (not the provider). With this approach you don't need duplication or $ref anything, just use what is already defined.

We got a code generator for SpringCloud Streams that can generate code following this perspective of roles, but nothing special about AsyncAPI definitions files, just propper and 100% compatible v2 or v3 asyncapi.yml.

I would stay away from external $refs as much as posible because:

But more over because they are an unnecesary trouble.

ZenWave SDK is the opensource succesor of that battle tested code generator.

Will try to explay in the next comment a simple example about how this works...

PS: I still see oportunity in v3 for external $refs for aggregating application channels from existing APIS, but that is another matter...

ivangsa commented 1 year ago

(This is an example that uses Java/SpringBoot but the asyncapi.yml or the technique is not special about Java)

Let's think we have a "Customers Service" that accepts commands for creating/updating/deleting customers and informs via domain events when a customer is created/updated/deleted.

Two channels:

Customers service provides the following asyncapi:

customers-asyncapi.yml (expand to see) ```yaml asyncapi: 2.6.0 info: title: Customers Service API version: 0.0.1 defaultContentType: application/json tags: - name: "Customer" channels: customer.requests: subscribe: summary: Accepts Customer Commands to create/update/delete a Customer operationId: doCustomerRequest tags: - name: Customer message: $ref: "#/components/messages/CustomerRequestMessage" customer.events: publish: summary: Informs about Customer created/updated/deleted Domain Events operationId: onCustomerEvent tags: - name: Customer message: $ref: "#/components/messages/CustomerEventMessage" components: messages: CustomerRequestMessage: messageId: CustomerRequestMessage name: CustomerRequestMessage title: Async Command/Request for a Customer summary: Async Command/Request for a Customer schemaFormat: application/vnd.aai.asyncapi;version=2.6.0 traits: - $ref: '#/components/messageTraits/CommonHeaders' payload: $ref: "#/components/schemas/CustomerRequestPayload" CustomerEventMessage: name: CustomerEventMessage messageId: CustomerEventMessage title: Message for a Customer Event summary: Message for a Customer Event schemaFormat: application/vnd.aai.asyncapi;version=2.6.0 traits: - $ref: '#/components/messageTraits/CommonHeaders' payload: $ref: "#/components/schemas/CustomerEventPayload" messageTraits: CommonHeaders: headers: type: object properties: kafka_messageKey: type: string description: This header value will be populated automatically at runtime x-runtime-expression: $message.payload#/id tracingId: type: string description: This header value will be populated automatically at runtime x-runtime-expression: $tracingIdSupplier schemas: Customer: type: "object" x-business-entity: "Customer" required: - "username" - "password" - "email" - "firstName" - "lastName" properties: id: type: "string" username: type: "string" minLength: 3 maxLength: 250 password: type: "string" minLength: 3 maxLength: 250 email: type: "string" minLength: 3 maxLength: 250 firstName: type: "string" minLength: 3 maxLength: 250 lastName: type: "string" minLength: 3 maxLength: 250 CustomerRequestPayload: type: object properties: requestType: type: string enum: - "create" - "update" - "delete" customer: $ref: "#/components/schemas/Customer" CustomerEventPayload: type: object properties: id: type: string eventType: type: string enum: - "created" - "updated" - "deleted" customer: $ref: "#/components/schemas/Customer" ```

Customers Service

Code generation for Customer Service (see <role>provider</role>:

<!-- Generate PROVIDER -->
<inputSpec>${pom.basedir}/src/main/resources/model/customers-asyncapi.yml</inputSpec>
[...]
<execution>
  <id>generate-asyncapi</id>
  <phase>generate-sources</phase>
  <goals>
      <goal>generate</goal>
  </goals>
  <configuration>
      <generatorName>spring-cloud-streams3</generatorName>
      <configOptions>
          <role>provider</role><!-- IMPORTANT ROLE -->
          <style>imperative</style>
          <transactionalOutbox>mongodb</transactionalOutbox>
          <modelPackage>io.zenwave360.example.core.domain.events</modelPackage>
          <producerApiPackage>io.zenwave360.example.core.outbound.events</producerApiPackage>
          <consumerApiPackage>io.zenwave360.example.adapters.commands</consumerApiPackage>
      </configOptions>
  </configuration>
</execution>

This will generate:

This is how it looks some of the generated code:

// Autogenerated: you can @Autowire it in your code
public interface ICustomerEventsProducer {
    // headers object omitted for brevity
    /**
     * Customer Domain Events
     */
    boolean onCustomerEvent(CustomerEventPayload payload, CustomerEventPayloadHeaders headers);

}

See ZenWave AsyncAPI Generated Code in a Nutshell for all the details about how generated code looks like and can be used.

Other Applications

Any other application that wants to send "Customer Command" for creating/updating/deleting customers or want to be informed about changes in customer will do the following:

Code generation for any other service that wants be a client of Customers Service:

<!-- Generate CLIENT -->
<inputSpec>${pom.basedir}/src/main/resources/model/customers-asyncapi.yml</inputSpec>
[...]
<execution>
  <id>generate-asyncapi</id>
  <phase>generate-sources</phase>
  <goals>
      <goal>generate</goal>
  </goals>
  <configuration>
      <generatorName>spring-cloud-streams3</generatorName>
      <configOptions>
          <role>client</role><!-- IMPORTANT ROLE -->
          <style>imperative</style>
          <transactionalOutbox>mongodb</transactionalOutbox>
          <modelPackage>io.zenwave360.example.core.domain.events</modelPackage>
[...]
      </configOptions>
  </configuration>
</execution>

This will generate:

Company Wide

You can extend this model about how applications use each others APIs without duplication or external $refs


This project has executable examples for most combinations, that I use for testing releases (so it may not be as clear as examples) https://github.com/ZenWave360/AsyncAPI-ApiFirst-Generator-KitchenSink

jonaslagoni commented 1 year ago

When applications need to consume 3rd party events or produce 3rd party command they are acting as client of that API, they don't need to create any API definition for that. Just use the existing one but from the role/perspective of a client (not the provider). With this approach you don't need duplication or $ref anything, just use what is already defined.

As far as I can understand this is the root of this discussion, this is a design pattern that you implemented and prefer, and with reason to back it up.

Where on the opposite side each application has an AsyncAPI document, that defines its behavior (this includes how the public can interact with your system). This seems to be the focus and direction AsyncAPI moves, and why I guess it grinds a little bit when it comes to the recommendation section for you πŸ™‚

With your approach, you are forcing the need of <role>provider</role> which is highly tooling specific, to your use-case. In the code generator from Solace they use view instead and only provider as an option (otherwise default).

With the recommended approach, you would not need the option in the code generator, and its straightforward what the operation keywords mean, instead of having this switch between what they mean and which context. I.e. creating confusion.

This is the entire reason I love this approach, cause there is no confusion about the meanings of the operation keywords and other perspectives through implementation details. What you see is exactly what the application does, nothing more nothing less.

You can of course still use your design pattern and the parameter in the code generator πŸ‘ Maybe it makes more sense as a mirror boolean parameter, but yea.

I personally don't think that design pattern unlocks AsyncAPI to its full potential, i.e. why I am also in favor of keeping that recommendation πŸ™‚

PS: I still see oportunity in v3 for external $refs for aggregating application channels from existing APIS, but that is another matter...

In v3 a channel definition can be a reference to an external channel definition. Unless you mean something like inheriting channels from other applications? Nwm, feel free to start another discussion about that, let's not mix more stuff into the discussion πŸ˜†

ivangsa commented 1 year ago

This is not about the tooling, because zw can generate code for both aproaches in v2 and v3 already, it is about what we recommend AsyncAPI users (mostly new/unaware/unexperienced users).

The root recomendation tries to prevent the following two problems for users:

But when linking to external resources, it introduces these two (imo major) problems:

May only be me but I think that the second two problems are way bigger for end users than the first ones...


Let's do an exercise, let's say that there are 2 application "Customers Service" and another one that will use (produce and consume) "customer.request" and "customer.events"

https://gist.github.com/ivangsa/a1c7fa242ee9f38319deeb0e1dfd0991

Sounds fair?


My take on this is to replace the current recomendation with a detailed explanation of both aproaches, probably linking to a different page or blog post with pros, cons and examples... With that I'm also happy stating that the offical recomendation is A, as long as B gets also explained...

(My worry is trying to explain that "external $refs are trouble" to some users that will not look behind an official recomendation, not the recomendation on itself)

ivangsa commented 1 year ago

Well at least now we got a shared understanding that it's possible to document all event-driven interactions without duplication or external $refs, right? I think this is something that we didn't have before..

jonaslagoni commented 1 year ago

This gist represents all you need to document with AsyncAPI v3 with my aproach, and it can be used for both editing and code generation with: AsyncAPI Studio, VSCode, Modelina, ZenWave and Solace template...

To me it doesn't no, cause you have nothing that explains who interacts with your Customers Service, it's one big black box. It solves your immediate problem with not having a second AsyncAPI document but you limit yourself and start introducing other problems (such as discoverability, documentation, fine-graned control).

I think there is definitely a reason to keep something like role parameter in code generators, cause there can be scenarios, i.e, if people use your design pattern, or you wish to quickly generate a mirror library to an application.

  • Let's create a gist with the current official recomendation and see how it looks (also in terms of extra files), and let's place it behind an authenticated repo and see how all the above tools work..

Sounds fair?

Not exactly, and let me show you why πŸ˜„

Here you go: https://gist.github.com/jonaslagoni/66958036cb28af76dc1efe33dbf21d37. Please ignore spelling mistakes, and I bet there is some wording that's incorrect, so take it as an example πŸ˜„

Take notice namings of operations and descriptions and how straightforward it is to know what it means for the application itself.

All of these AsyncAPI documents are located within the same repository, where they have full access to relative files ./ because that's what I personally use. Each generated library, documentation, bundles, linting, etc are done from here and into other repositories.

With this approach and design pattern, you achieve the full power of AsyncAPI:

As I bet you are thinking I skipped completely over your problem statement with references and authentication, but that's the underlying setup I have that allows me to do that.

Problems with references and authentication are that it's mainly tool-specific, something that should be setup at runtime i.e. with environment variables, etc. You could include tokens etc as part of the URL, and even do a pre-action that replaces placeholders with environment variables, etc.

For this, we definitely need better tooling. We 100% agree here πŸ˜‰ Because if each repository had an AsyncAPI document, and needed to reference across GitHub repositories you would most probably run into a lot of issues πŸ˜„

My take on this is to replace the current recomendation with a detailed explanation of both aproaches, probably linking to a different page or blog post with pros, cons and examples... With that I'm also happy stating that the offical recomendation is A, as long as B gets also explained...

(My worry is trying to explain that "external $refs are trouble" to some users that will not look behind an official recomendation, not the recomendation on itself)

The problem here is your implementation details start to creep into the spec itself, cause without ALL tooling implementations having role or view it becomes impossible to use. That I would strongly discourage to say it mildly πŸ˜„

ivangsa commented 1 year ago

Yes, you skiped all together the main concerns which is external references are a trouble waaay bigger that "read-only topics" and "naming style"...

I'm the only one seeing this?

Let's document both aproaches and let users decide for themselves..

fmvilas commented 1 year ago

I want to clarify that I also see these downsides. What I don't agree with is to remove this recommendation. I think the spec should recommend you to use it this way. IMHO, we should document the two approaches as part of the docs on the website but not as part of the spec.

ivangsa commented 1 year ago

ok, but maybe we should warn users about the downsides with external refs preferably next to this recomentation on the spec..

ivangsa commented 1 year ago

anyway, did we get a shared understanding that you can document all interactions with AsyncAPI without duplication or references (v2 or v3) that we didn't have before, right?

I implemented this approach succesfully in production for a quite large company for a number of years without problems about "naming" or "readonly topics" (I understand the official approach but I read the contents of the recomendation and it doesn't sound like the reason for the official recomendation)

ivangsa commented 1 year ago

thanks to this conversation I learn a lot about my thought process and AsyncAPI

publish/subscribe was never a confusion to me, that's why I manage to propose and succesfully adopt AsyncAPI without duplicaton or external $refs

Scenario: there are 100 application that each one publish a different event to a different topic, no one subscribes to anything. There will be 100 asyncapi.yml without crossed references (nothing confusing here).

Application 101 kicks in and publishes one event but it also would like to subscribe to all other 100 messages.

(I preffer this thoughts not to be buried here, but until I get to write them somwhere else. This is just food for thought)

fmvilas commented 1 year ago

ok, but maybe we should warn users about the downsides with external refs preferably next to this recomentation on the spec..

I don't think so. The spec is full of things that can be done in multiple ways, each with pros and cons. The pros and cons in this case have more to do with the state of tooling than the spec itself. That warning that you want to put there doesn't belong in the spec but in the docs.

anyway, did we get a shared understanding that you can document all interactions with AsyncAPI without duplication or references (v2 or v3) that we didn't have before, right?

For sure. It's a cool approach that can serve multiple people and should be documented. Definitely not in the spec but in the docs.

I implemented this approach succesfully in production for a quite large company for a number of years without problems about "naming" or "readonly topics" (I understand the official approach but I read the contents of the recomendation and it doesn't sound like the reason for the official recomendation)

Yeah, the thing is that using $ref has also been implemented in many large companies successfully too. Among others, are big banks and telcos. My point with this is that your approach is great but it's not the only great approach and $ref can be painful but it doesn't have to be if you have the proper infrastructure. E.g., APIs to serve shared resources, schema registries to serve versioned schemas, etc.

publish/subscribe was never a confusion to me, that's why I manage to propose and succesfully adopt AsyncAPI without duplicaton or external $refs

I'm glad it was never a confusion for you but what we found since the very beginning of the spec is quite the opposite. We tried hard to educate people on that point of view but damn, the confusion was coming again and again for everyone. So yeah, it's a valid point of view but it's definitely not preferred by the majority. In fact, the world is now full of AsyncAPI files using publish and subscribe the wrong way πŸ˜… As @jonaslagoni said above, this was the reason that triggered us to release v3 in the first place.

Scenario: there are 100 application that each one publish a different event to a different topic, no one subscribes to anything. There will be 100 asyncapi.yml without crossed references (nothing confusing here).

Application 101 kicks in and publishes one event but it also would like to subscribe to all other 100 messages.

Should it create one AsyncAPI with one topic/message (A) or with 101 operations (B)?

How option B works as a Public API for app 101?

What information would App 102 get from Asyncapi-101-B.yml? There are 100 'receive' operations.. can it also send messages to those? Why not? Aren't they on its public API? What is the difference?

With option A: 100 apps don't know anything about who their clients are.

With option B: 100 apps still don't know anything about who their clients are... (Unless you have an API Portal that processes and centralizes that information Asyncapi-101-B.yml is just a file in some repo, 100 application don't necesarily know about it).

I'm glad you're going through all the thought process that we did during the last year. It's refreshing to see that people care about it. Here are some issues that are related to these thoughts:

(I preffer this thoughts not to be buried here, but until I get to write them somwhere else. This is just food for thought)

Please πŸ™ keep them coming. And make sure you're heard and it's properly tracked and discussed in its own issue. The spec is not set in stone and things can always evolve.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart: