Upgrade the structure of Resources for block media

[markboots]

Rationale:

We've encountered a need to have multiple pieces of media within a resource. These multiple pieces of media might provide different formats (ie: ContentTypes) for a single mode (e.g. rich_messaging).

• e.g. A chatbot block may need to provide an image AND text, or an image and an audio file.

There is also a need to specify different content for different modes,

• e.g. A flow builder may want to specify different text-format content for the SMS channel (ie: shortened) vs. the text-format content for the rich_messaging channel.

The proposed upgrade to the Resource structure to support this is:

Example:

Resource {
   uuid: "c2dbafbd-e9bd-408f-aabc-25cf67040002",
   values: [
      {
         language_id: "47",
         modes: ["sms", "ussd"],
         content_type: "text",
         mime_type: "text/plain",
         value: "Howdy! You've reached the Museum of Interoperability!"
      },
      {
         language_id: "47",
         modes: ["rich_messaging"],
         content_type: "text",
         mime_type: "text/plain",
         value: "Howdy! You've reached the Museum of Interoperability! This is a long message for you because we've gone beyond the limitations for 180 characters. I'm your guide, Florian. I hope you're excited for this two hour tour through the history of interoperable data systems."
      },
      {
         language_id: "47",
         modes: ["rich_messaging"],
         content_type: "image",
         mime_type: "image/png",
         value: "https://your-server-somewhere.flowinteroperability.org/example-image.png"
      }
   ]
}

Structure:

Resource {
  uuid: string,
  values: ResourceVariant[]
}

ResourceVariant {
  language_id: string,
  content_type: SupportedContentType,
  mime_type: string,
  modes: SupportedMode[],
  value: string,
}

SupportedContentType {
  TEXT = 'text',
  AUDIO = 'audio',
  IMAGE = 'image',
  VIDEO = 'video',
}

SupportedMode {
  SMS = 'sms',
  USSD = 'ussd',
  IVR = 'ivr',
  RICH_MESSAGING = 'rich_messaging',
  OFFLINE = 'offline',
}

Note: In the course of developing the open-source Flow Runner and Flow Builder implementations, the initial dev team proposed a list of cleanups and clarifications to the spec. This is one of a set of changes proposed to improve consistency, usability, and functionality of the spec based on the first complete build-outs.

[smn]

In some rich messaging transports, it's possible for some text & media combinations to be sent in a single message, like an image with a caption for example.

I'm assuming it would be up to the transport layer to bundle these into a single API call in the example above?

[markboots]

I think I agree; the way to specify a text+media combination (like an image with caption) in the Flow design would be:

• The block prompt specifies one resource ID
• This resource has both of the following values within it, for the rich_messaging channel and language specified (e.g. below)
• The transport layer around the flow runner would know that it should bundle these in one API call on transports that support it. Where not supported in one API call, they'd be sent immediately sequentially. We don't have anything in the design to specify the order that combos like this should be sent in; if that order is important, one could use two sequential blocks rather than a "combo" resource.

         language_id: "47",
         modes: ["rich_messaging"],
         content_type: "text",
         mime_type: "text/plain",
         value: "Howdy! You've reached the Museum of Interoperability! This is a long message for you because we've gone beyond the limitations for 180 characters. I'm your guide, Florian. I hope you're excited for this two hour tour through the history of interoperable data systems."
      },
      {
         language_id: "47",
         modes: ["rich_messaging"],
         content_type: "image",
         mime_type: "image/png",
         value: "https://your-server-somewhere.flowinteroperability.org/example-image.png"
      }

[markboots]

(Historical context: This change to the Resource structure enables combos like image + caption in one resource; the initial resource concept would not have been able to contain both.)