[JS] Inline YouTube, Vimeo and Dailymotion video playback

dclaux commented 2 years ago

This proposal is implemented here: https://github.com/microsoft/AdaptiveCards/pull/7112

Problem Statement

Currently, it is not possible to embed a YouTube, Dailymotion, Vimeo (and quite a few other for that matter) video in an Adaptive Card. This feature would make that possible.

Proposed solution

We extend the existing Media element with no schema change. The updated logic does the following:

If a URL matching a a YouTube, Vimeo or Dailymotion pattern is found in the sources collection, the Media element is setup to embed an inline player instead of the standard HTML5 media player
- The mimeType property isn't required for YouTube sources
- If a URL pattern is detected in the sources property, all other sources are ignored
The Media element will attempt to pull the poster URL automatically from the various providers IF the poster property sin't specified
As the play button is clicked, the Media element instantiates the embedded player and automatically starts playback

Alternatives or Workarounds

The only alternative currently is to use an image that represents a video thumbnail and to include an OpenURL action somewhere in the card to open the video on the appropriate web site.

paulcam206 commented 2 years ago

Presumably to support this in non-browser renderers we'd either need to host a web control of some sort or allow card authors to specify other sources as fallback. How would poster work in that case? Perhaps it should be optional in the source entry? I'd expect that poster in the Media element would apply to all sources, with each source being able to override it (do we need a "poster": "auto" for HTML5?).

Perhaps something like this:

{
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "type": "AdaptiveCard",
  "version": "1.6",
  "body": [
        {
            "type": "Media",
            "sources": [
                {
                    "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
                },
                {
                    "mimeType": "video/mp4",
                    "url": "https://adaptivecardsblob.blob.core.windows.net/assets/AdaptiveCardsOverviewVideo.mp4",
                    "poster": "https://adaptivecardsblob.blob.core.windows.net/assets/AdaptiveCardsOverviewVideoPoster.png"
                }
            ]
        }
    ]
}

dclaux commented 2 years ago

It's an interesting idea, although IMO it's overkill.

One of the reasons why the Media element as it currently stands hasn't been able to satisfy the needs of developers is because there's not fallback to a YouTube video: a YouTube video only exists in YouTube (the same goes for Dailymotion and Vimeo). So while we could implement what you're suggesting, IMO that fallback model would basically never be exercised.

I think 99.9% of use cases are the developer has one single video URL and thus includes one single source to the Media element.

In terms of the poster, I don't think we need to introduce a new property on a per-source basis. The poster on the Media element itself is good enough and can be applied to whichever source is selected (that's how it currently work). The difference my proposal introduced is that for YouTube, if poster isn't specified, the code attempts to fetch the poster from the target service. Also, we wouldn't be able to honor "poster": "auto" for mp4 videos.

dclaux commented 2 years ago

Another thing to consider. Imagining the source property looks something like this:

"sources": [
    mp4,
    YouTube,
    mp4,
    mp4
]

The HTML5 media player (and I believe the media players on Android and iOS as well) take multiple URLs as their own sources and automatically pick the one they can play.

What would we do with the above sources?

Create an HTML5 media player and give it the first source only
If it can't play, switch to an IFramed player and give it the second URL
If it can't play (which we probably can't even detect), switch back to an HTML5 player and feed it the last two sources

Or

Instantiate an HTML5 player and give it all the sources that are not YouTube
And only fallback to an IFramed player if none of these can be played?

paulcam206 commented 2 years ago

I suppose it's probably easier to go with a minimum viable implementation and add stuff later if we get feedback :)

sueess commented 2 years ago

We would highly appreciate this feature too. Beeing able to display Youtube & co videos directly in the cards would be great.

rliberoff commented 2 years ago

Hello,

We also support this request. Today, many companies have their video assets in Youtube and other platforms. Please provide support for this.

Thank you!

microsoft / AdaptiveCards