Possible changes to Message Resource

[duglin]

Today, Endpoints, with Message inlined, looks like this:

 "endpoints": {
    "ID": {
      "id": "STRING",                           # xRegistry core attributes
      ... common xReg metadata: name, epoch, self, ...

      # Endpoint specific metadata
      "usage": "STRING",                        # subscriber, consumer, producer
      "format": "STRING", ?                     # Delete?
      "binding": "STRING", ?                    # AMQP, HTTP, ...
      "channel": "STRING", ?                    # For Endpoint correlation
      "deprecated": { ... }, ?

      "config": {                               # Transport specific metadata - share name with "binding" above? e.g. bindingConfig?
        "protocol": "STRING",                   # Should this be the same as "binding" above? kill one of them?
        "endpoints": [ { "uri": URI } * ], ?    # plus endpoint(uri) extensions

        "authorization": { ... }, ?
        "deployed": BOOLEAN, ?                  # am I alive and ready to be used?

        "options": { "STRING": JSON-VALUE *}, ? # Protocol specific flags - e.g. HTTP "method", "headers" / MQTT "topic" / ...
      }, ?                      # Do we need the "config" bucket? Does it make it look more complex?
                                # E.g. extensions at top vs "config" vs "config.options" 

      "messagegroups": [ URI * ], ?

      "messagesurl": "URL", ?
      "messagescount": UINTEGER, ?
      "messages": {
        "ID": {
          "id": "STRING",                       # xRegistry core attributes
          ... common xReg metadata: name, epoch, self, ...

          "basemessageurl": "URL", ?            # Message attributes

          "format": "STRING", ?                 # Envelope - e.g. CE/1.0 vs SOAP
          "metadata": { ... }, ?                # Env metadata - e.g. ce.type

          "binding": "STRING", ?                # or "format"
          "message": { ... }, ?                 # See Message Bindings section

          "schemaformat": "STRING", ?
          "schema": "STRING" | {}, ?
          "schemaurl": "URL", ?

          "versionsurl": "URL",
          "versionscount": UINTEGER,
          "versions": { ... } ?
        }, *
      } ?
    } *

One issue with this is that often times messages/events are sent over different protocol, with different "envelopes" and even different formats (json vs xml). In order to represent that the SAME bit of content(data) has these alternatives many different instances of a Message needs to be created. It is then up to the Registry maintainer to find a way to correlate all of these Messages together and to keep them all in-sync if changes are needed.

If we view the content as the central piece of data of interest, we then have an hourglass type of structure around it - with the content being the middle of the hourglass. For example, abstractly we have:

Endpoints -> Messages -> Envelopes -> CONTENT -> schemes

Notice that all of those entities can have multiple instances of them except CONTENT - hence the (sort of) hourglass idea.

The proposal is to allow for a single definition of CONTENT to be associated with multiple entities above and below it in the above abstract model:

 "endpoints": {
    "ID": {
      "id": "STRING",                           # xRegistry core attributes
      ... common xReg attributes: name, epoch, self, ...

      # Endpoint specific metadata
      "usage": "STRING",                        # subscriber, consumer, producer
      "format": "STRING", ?                     # CE, SOAP, ...  - Selector for message
      "protocol": "STRING",                     # AMQP, HTTP, ... - Selector for message
      "channel": "STRING", ?                    # For Endpoint correlation
      "deprecated": { ... }, ?

      "endpoints": [ { "uri": URI } * ], ?
      "authorization": { ... }, ? 
      "deployed": BOOLEAN, ?                    # am I alive and ready to be used? Move outside of config?

      "config": {
        ...                                     # protocol specific options - same as we see in Message below
      }, ?

      "messagegroups": [ URI * ], ?             # To allow for reuse of messages across Endpoints,
                                                # could use xref if intra-Registry in "messages" below

      "messagesurl": "URL", ?                   # xRegistry "messages" collection
      "messagescount": UINTEGER, ?
      "messages": {
        "ID": {
          "id": "STRING",
          ... common xReg attributes: name, epoch, self, ...

          "basemessageurl": "URL", ?

          "content": {                          # MUST all be semantically equivalent
            "SCHEMA-FORMAT": {                  # e.g. avro, jsonschema, xmlschema
              "schemaurl": "URL", ?
              "schema": "STRING" | { } ?
            } *
          }, ?

          "format": {                           # Map of format/envelope types - do we need an "none" type of option?
            "ENV-FORMAT": {                     # e.g. "CloudEvents/1.0", "SOAP"
              ... metadata ...                  # format specific metadata - e.g. "type": ...
              "protocol": {                     # These will override? the "binding.PROTOCOL" metadata above
                "PROTOCOL": {
                  ... metadata ...              # very advanced cases where this env needs protocol specific config
                } *
              } ?
            } *
          }, ?                                  # In the simple inline-single-use case, this data might appear in Endpoint above

          "protocol": {                         # Map of transport protocols. These will override? Endpoint level config
            "PROTOCOL": {
              ... metadata ...                  # e.g. "HTTP": { "headers": { ... } }  Was "message" object
            } *
          }, ?                                  # In the simple inline-single-use case, this data might appear in Endpoint above

          "versionsurl": "URL",
          "versionscount": UINTEGER,
          "versions": { ... } ?
        }, *
      } ?
    } *

The Endpoints format and protocol attributes become selectors for the Message - this is how the Endpoint indicates which one it supports.

Do we need a "content" selector for which schema it'll use? Mainly when it's a producer of messages.

While there are questions in the Endpoint section, this issue is really about changing the Message entity.

[duglin]

An example:

{
  "specversion": 1.0",
  "id": "myregistry",

  "endpoints": {
    "myinput": {
      "id": "myinput",
      "usage": "producer",
      "format": "",                          # same as missing, means none
      "protocol": "HTTP/1.1",
      "channel": "myadapter",
      "endpoints": [{"uri": "https://example.com"}]
      "config": {
        "method": "POST"
      },
      "messagegroups": [ "/messagegroups/mymsgs/messages/mymessage" ]
    },
    "myoutput": {
      "id": "myoutput",
      "usage": "consumer",
      "format": "CloudEvents/1.0",
      "protocol": "Kafka",
      "channel": "myadapter",
      "endpoints": [{"uri": "https://server.com"}]
      "config": {
        "topic": "serviceA/{topic}",
        "consumergroup": "mygroup"
      },
      "messagegroups": [ "/messagegroups/mymsgs/messages/mymessage" ]
    }
  },

  "messagegroups": {
    "mymsgs": {
      "id": "mymsgs",
      "messages": {
        "mymessage": {
          "id": "mymessage",

          "content": {
            "jsonSchema": {
              "schemaurl": "...url to popped event schema..."
            }
          },
          "format": {
            "CloudEvents/1.0": {
              "metadata": {
                "type": { "value": "popped" },
                ...                
              }
            }
          },
          "protocol": {
            "HTTP/1.1": {
              "headers": [
                { "name": "action", "value": "popped" }
              ]
            },
            "Kafka": {
              "topic": "serviceA/mytopic",         # Optional
              "key": "{key}"
            }
          }
        }
      }
    }
  }
}