[FEATURE REQUEST] Provide formal type for message examples

lbroudoux commented 4 years ago

Is your feature request related to a problem? Please describe. I'm working on an API mocking tool that aims to use a specification examples for providing mock endpoints and realize contract testing (see https://microcks.io). For REST APIs specified with OpenAPIs, Microcks tooks advantage of clearly defined exampleObject allowing to specify example values and references for every request or response fragments.

I found it very valuable to have the same mechanism with AsyncAPI to allow specification of real-life examples for providing mock channels and messages helping speed-up development on new consumers / providers. However, examples are just for now a [Map[string, any]] that makes them difficult to agree on a common way to structure samples.

Can't it be tackled using specification extensions? I do not see any extension mechanism to enable this...

Describe the solution you'd like I'd like a formal type description on how a message example should be structured including some metadata, an optional set of headers and the associated payload. Something like:

message:
  description: An event describing that a user just signed up.
  traits:
    - $ref: '#/components/messageTraits/commonHeaders'
  headers:
    type: object
    required: true
    properties:
      sentAt:
        type: string
        format: date-time
        description: Date and time when the event was sent
  payload:
    type: object
    additionalProperties: false
    properties:
      fullName:
        type: string
      email:
        type: string
        format: email
      age:
        type: integer
        minimum: 18
  examples:
    - laurent:
        summary: Example for Laurent user
        headers: |-
          {"my-app-header": 23, "sentAt": "2020-03-11T08:03:28Z"}
        payload: |-
          {"fullName": "Laurent Broudoux", "email": "laurent@microcks.io", "age": 41}
    - john:
        summary: Example for John Doe user
        headers:
          my-app-header: 24
          sentAt: "2020-03-11T08:03:38Z"
        payload:
          fullName: John Doe
          email: john@microcks.io
          age: 36

Describe alternatives you've considered I considered providing example fragments at each level of the specification (header, traits and payload) but I found it confusing because trait does not seem to allow examples and also because examples in the case of headers are the schema level and not the instance level. This proposition looks the more concise and understandable (at least to me ;-) )

Additional context This feature will allow :

to complete specification with real-life examples (and not only generated ones from type schemas) which is very valuable and gives, free and human-readable documentation,
to build a set of tools (like Microcks ;-)) that can turn your specification in something executable whether for doing mocking or contract testing.

github-actions[bot] commented 4 years ago

Welcome to AsyncAPI. Thanks a lot for reporting your first issue.

Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

yada commented 4 years ago

+1 this is expected by many async devs, archi... and will be great to be able to rely on specs for this purpose.

github-actions[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 30 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

lbroudoux commented 4 years ago

Hi @fmvilas and AsyncAPI team,

Just to let you know how we plan to use AsyncAPI examples in Microcks.io, we released a preview video of this feature here: https://www.youtube.com/watch?v=uZaWAekvUz4

You'll see the previous suggested formalism in action. Please tell us what you think about it ;-)

Cheers,

derberg commented 4 years ago

@lbroudoux Awesome demo, thanks a lot for sharing. It super nicely shows why would it make sense to have more structured and defined examples

github-actions[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 30 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

lbroudoux commented 4 years ago

Hi AsyncAPI team,

Now that we have a working implementation in Microcks and are close to release 1.0.0 version with AsyncAPI support, I'd like to see how to push this ideas further. Would you guide me contributing a draft enhancement proposal?

What could be the best way:

directly forking the repo and submitting a PR on spec?
establishing some kind of design documentation on GDoc or other?

I've got some time to spend on this in coming weeks so really like to contribute back ;-)

derberg commented 4 years ago

@lbroudoux there is no formal process like KEP for Kubernetes for example. We simply work and discuss in issues to have all the thinking process in one place. So we first discuss under issue and then once there is a consensus you go on with a PR.

lbroudoux commented 4 years ago

OK nice!

So I actually implemented some samples using the solution that was here first described:

Defining examples as a Map of sample messages with key being the name of the sample,
A sample message got a summary that may help refining sample / use-case matching, a headers and payload attributes that are simply Objects,
payload should follow message payload schema and provide actual payload that may be expressed directly in YAML or by embedding JSON,
headers should follow aggregation of message level header schema and traits schema - expressed in YAML or embedded JSON as well.

For now - after having tested it on a dozen of schemas and use-case - I cannot foresee any caveats, difficulties or challenges following this specification. What do you think?

derberg commented 4 years ago

I like the suggestion a lot. I only have a question in my head to the example object from OAS that you provided in the description. There is value and externalValue and I'm now curious if externalValue is adopted by the community. My guess is that there are 2 separate properties because example might be super different than json, like xml for example, thus cannot be used with $ref. Anyway thinking about these 2 somehow made my thing about a property that I could use to link to an external resource that contains an example, it could even be a link to documentation with an anchor to a specific section. This would also be an interesting property next to the summary 🤔 any thoughts, have you see a need for something like this?

lbroudoux commented 4 years ago

It's a good point ! From what I see until now, externalValue does not seem to be adopted by the community.

As a very personal note and opinion, I also think that referencing stuffs outside of the specification file can lead to imprecise, too generic or totally mismatching examples. For me - and some other folks I discuss with - specifying examples is all about being very specific to the context (think of bounded context) of the operation or the API at a whole. Scaffolding valid examples embedded within the spec is usually an issue of tooling and some tools like Apicurio do this very well and help keeping examples in sync when evolving schema.

Also, from my understanding using value in OpenAPI does not prevent from using some other types that JSON as long as they can be represented as String. I see users specifying XML or Text representations using value. However, right now I'm scratching my head 🤔 thinking on how this could be done using formats like Avro for example ... maybe with JSON to Avro conversion ?? - or is there some subtleties we cannot easily convert ??

So, even if they do not sound essential to me, I tend to think that we could add something like externalPayload and externalHeaders in order to be fully analogous to OpenAPI and to allow future usages we are not aware of right now ;-)

derberg commented 4 years ago

afaik Avro can be presented as JSON or YAML so you can easily put it inside AsyncAPI file. Have a look at this example for our Avro schema parser https://github.com/asyncapi/avro-schema-parser#usage. Avro is part of the spec, and we specify it is Avro with schemaFormat, so if in some tooling I would have to somehow, in some special way treat the example, I would have to first read schemaFormat to know what kind of format is followed also in the example. Of course schemaFormat is described in the way that it is for the payload, so we would just have to make sure spec is clear that example format must match the payload format?

I see users specifying XML or Text representations using value

how do they do it in JSON, xml is represented as string or?

lbroudoux commented 4 years ago

see users specifying XML or Text representations using value how do they do it in JSON, xml is represented as string or?

Yes, XML represented as string between double-quotes. JSON represented as string or JSON embedded into YAML or in YAML depending on the authoring tool. In the case of Microcks importing YAML, we take care of YAML to JSON conversion based on specified content-type like you suggest for schemaFormat.

derberg commented 4 years ago

oh 🤦 we have contentType as well of course and this is why we should use to interpret the example, not schemaFormat. Now it also makes sense to me why the example in the issue description has JSON as String and not directly JSON 👍

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 30 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

lbroudoux commented 3 years ago

Hi @fmvilas and @derberg !

Regarding your last comment on this issue: yes you're right, contentType is the way to go ;-)

Now that we have stabilized things and demonstrate pragmatic usages of examples through Microcks mocking and testing features, I'd like to tackle making this issue and suggestions a real proposal for 2.1.0 release of the specification ! What could be the next steps for it ? Initializing a 2.1.0 folder with https://github.com/asyncapi/asyncapi/tree/master/versions and creating a PR ? Starting a markdown patch in some other way ?

Let me know, I'll be happy to start writing it in a more formal way.

derberg commented 3 years ago

@lbroudoux I think we need to figure out this first https://github.com/asyncapi/asyncapi/issues/463. Feel free to jump into discussion

lbroudoux commented 3 years ago

Thanks @derberg for the heads-up. I'll jump 😉

WaleedAshraf commented 3 years ago

@lbroudoux I like the idea, but I don't fully understand it. Why don't you define examples like this:

Doesn't this cover your point,

to complete specification with real-life examples (and not only generated ones from type schemas) which is very valuable and gives, free and human-readable documentation,

lbroudoux commented 3 years ago

Hi @WaleedAshraf

Examples at the property level are useful to understand business meaning or illustrate values range of individual property. However they quickly present limitations when it comes to describe complex structures having oneOf or anyOf constructs for the like. And even if you're dealing with more simple messages with optional nodes or branches that may or may not be included depending on some other node value.

I have one example coming to my mind about an Insurance company emitting events for natural or legal persons ... It's the same type of event representing a sole status change but with very different properties depending on the type of person.

So we do think that - for comprehensive understanding - it is better to have also the ability to describe full examples and to describe many of them using different keys/names and associated summary of description.

Let me know if you think it makes sense.

WaleedAshraf commented 3 years ago

I understand it now and yes, it can be useful. Now my concern is what if the provided examples are not according to the defined schema. i.e someone updated the schema, but not the example OR updated the example, but it's not according to the schema.

This should be handled and validated by the tooling. AFAIK, OpenAPI also doesn't validate examples. See this:

You can see that the example is not according to the schema. But it's still allowed.

Invalid examples can be really problematic for the end-user. And if there is an option to define custom examples, there is more chance that some of those will be invalid. (if not validated)

Thoughts?

derberg commented 3 years ago

According to AsyncAPI schema, example must match payload and therefore we will support this kind of validation, we just didn't have time to add it yet -> https://github.com/asyncapi/parser-js/issues/99

lbroudoux commented 3 years ago

Agree with the role of tooling for this. This is where tooling can bring additional value. FYI, I am a contributor to Apicurio Studio and it has validation rules for OpenAPI including examples.

WaleedAshraf commented 3 years ago

Ok. If we plan to provide example validation in the future, then this (formal type for message examples) is a good feature to support. 👍

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

lbroudoux commented 3 years ago

Still there ;-) with hope to tackle this one soon !

derberg commented 3 years ago

@lbroudoux let's do this. Please have a look at the latest contribution guide and update here information about the stage your proposal reached. I bet you want to be the champion for it and drive it forward until the end

derberg commented 3 years ago

@lbroudoux I had a thought recently about this proposal and wanted to suggest an alternative that would not introduce a breaking change to the spec.

Initial proposal is

  examples:
    - laurent:
        summary: Example for Laurent user
        headers: |-
          {"my-app-header": 23, "sentAt": "2020-03-11T08:03:28Z"}
        payload: |-
          {"fullName": "Laurent Broudoux", "email": "laurent@microcks.io", "age": 41}
    - john:
        summary: Example for John Doe user
        headers:
          my-app-header: 24
          sentAt: "2020-03-11T08:03:38Z"
        payload:
          fullName: John Doe
          email: john@microcks.io
          age: 36

Current structure:

  examples:
    - headers: |-
          {"my-app-header": 23, "sentAt": "2020-03-11T08:03:28Z"}
      payload: |-
          {"fullName": "Laurent Broudoux", "email": "laurent@microcks.io", "age": 41}
    - headers:
          my-app-header: 24
          sentAt: "2020-03-11T08:03:38Z"
      payload:
          fullName: John Doe
          email: john@microcks.io
          age: 36

Why not doing:

  examples:
    - name: laurent #machine readable as it is now with message.name (for consistency)
      summary: Example for Laurent user
      headers: |-
          {"my-app-header": 23, "sentAt": "2020-03-11T08:03:28Z"}
      payload: |-
          {"fullName": "Laurent Broudoux", "email": "laurent@microcks.io", "age": 41}
    - name: john #machine readable as it is now with message.name (for consistency)
      summary: Example for John Doe user
      headers:
          my-app-header: 24
          sentAt: "2020-03-11T08:03:38Z"
      payload:
          fullName: John Doe
          email: john@microcks.io
          age: 36

Not everyone uses spec for testing, so not everyone should be "forced" to provide a key for the example object, I think. Also as result all example related properties will be in one place.

lbroudoux commented 3 years ago

Hi @derberg,

thanks for the suggestion, this is a great one and make the trick possible! The initial proposal of having a key was intended to align the structure on the one from OpenAPI Spec but it's better to introduce this without a breaking change.

Luckily enough, I had some time in the coming days and will review the new contribution guide to see how ti push this onward.

lbroudoux commented 3 years ago

Hi @derberg!

Pushed this 2 PRs few days ago. However, I did not have access to labels in order to propose a Strawman, Proposal or Draft status on them ...

derberg commented 3 years ago

@fmvilas I think we reached Stage 1 here, right?

@lbroudoux we also need a PR against the parser as this is release requirement, to add support for new fields, not only bump node-asyncapi but also extend https://github.com/asyncapi/parser-js/blob/master/lib/models/message-traitable.js#L89 as for now we just have a simple helper that returns all examples.

Thoughts?

lbroudoux commented 3 years ago

Great! As I saw that examples are still of Any type (despite payload and headers already in the spec) I thought there was a design decision to keep them generic ;-) Anyway, I will push the PR!

derberg commented 3 years ago

keep in mind parser is still a bit behind the spec, some features are missing like examples validation against the schemas, so don't pay that much attention to the type that you see there

fmvilas commented 3 years ago

@derberg Yup, label added.

lbroudoux commented 3 years ago

Suggestions have been resolved into existing PRs. Parser is now implemented and pushed here: https://github.com/asyncapi/parser-js/pull/307

I think the check list is now complete for entering Stage 2: DRAFT 😄

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity :sleeping: It will be closed in 60 days if no further activity occurs. To unstale this issue, add a comment with detailed explanation. Thank you for your contributions :heart:

magicmatatjahu commented 3 years ago

I think that this issue is resolved @lbroudoux If you think that something should be added, please reopen it. Thanks again for the contribution! :)

asyncapi / spec

[FEATURE REQUEST] Provide formal type for message examples #329