Server Guided Ad Insertion & TimeLine Manipulation for VOD Usecases

Hi,

We are investigating Server Guided Ad Insertion (SGAI) in the latest DASH-IF specifications for our dynamic + personalized ad use cases and we had several questions/concerns that we were hoping to get addressed by the standards board:

Our ads will have to be personalized for each user. I may not see the same ads you see. If the player is resolving xlinks to get to ads, we have to have a way to inject customer identifying information into the requests. Please keep in mind this can involve cookies or special headers (Authorization or otherwise). How is DASH-IF influencing the community (Exoplayer, ShakaPlayer, etc.) to accommodate such a thing?
We did a small prototype to see how Ads using Xlink would work, and we see that it influences the player's timeline (duration gets longer as each ad is resolved). We are concerned this only makes it harder for us to do things like resume playback positions for the user and would be an industry wide problem. Is there any thought put into having clients use separate players for xlink resolved Ad periods vs standard periods? Apple's recently unveiled approach uses this idea to maintain the main player's timeline while still capitalizing on player qos heuristics & late ad pod resolution.
How is DASH-IF thinking of communicating to the player that the xlink resolved period is an ad? Is there anything in the standard for that? or would it be up to us to use events and embed metadata into the period?

Please let us know the answers to these questions.

Thanks,

Ronak

Hi Ronak! Sorry for the delay to your questions some initial thoughts I would have:

Our ads will have to be personalized for each user. I may not see the same ads you see. If the player is resolving xlinks to get to ads, we have to have a way to inject customer identifying information into the requests. Please keep in mind this can involve cookies or special headers (Authorization or otherwise). How is DASH-IF influencing the community (Exoplayer, ShakaPlayer, etc.) to accommodate such a thing?

There is a direct expectation that request customization will be needed to take common / cached responses and make them user specific. Within the frameworks of DASH that exist the potential mechanism is the UrlQueryInfo/ExtUrlQueryInfo/ExtUrlHeaderInfo elements. These are described in Annex I of the DASH specification and detail directives on how the player should copy information from either a source query string or source header response to subsequent http request query strings or headers.

As a direct example of this application from the Hulu deployment ecosystem: We will generate user specific manifest urls which contain session specific query parameters and then have them stripped by CDN / services at request time so that the manifests served are common to all users. The manifests then carry the ExtUrlQueryInfo element which directs the player to copy the user specific values from the original url to the subsequent requests for xlink / patching. This way the client is receiving cached responses and sending unique requests. We prefer query over headers due to CORS restrictions in some deployment environments.

We did a small prototype to see how Ads using Xlink would work, and we see that it influences the player's timeline (duration gets longer as each ad is resolved). We are concerned this only makes it harder for us to do things like resume playback positions for the user and would be an industry wide problem. Is there any thought put into having clients use separate players for xlink resolved Ad periods vs standard periods? Apple's recently unveiled approach uses this idea to maintain the main player's timeline while still capitalizing on player qos heuristics & late ad pod resolution.

You are correct that this now means the timeline exposed by a player to an application UI can now change with respect to duration during playback. Currently we have not discussed recommending player exposure behavior differences in the group since it would solve the signaling issue for SGAI but not for SSAI and we want to ensure interoperability between the two forms. Put another way, whether a stream is performing SSAI or SGAI, it would make sense for the player to expose the same stream model, though we recognize in SGAI it may change over time unlike SSAI.

Initial thoughts on proper differentiation and signaling have been looking at introducing a custom descriptor scheme that could be used to differentiate "main content" and "inserted content". It could then be possible for players to either expose the "main content" timeline independent of the stream interleaving, or expose proper stream chapter tagging so that applications can compute content vs stream time for tracking purposes. Other descriptive elements like AssetIdentifier could be used but as that may only exist once it may be too limiting of a mechanism.

How is DASH-IF thinking of communicating to the player that the xlink resolved period is an ad? Is there anything in the standard for that? or would it be up to us to use events and embed metadata into the period?

Covered above a bit, but to also highlight some implementation details from the Hulu player that can help provide information reference:

Our players model an interface similar to a HTMLMediaElement called HuluMediaElement with extensions and customizations to expose deeper information about the underlying stream

One extension is HuluMediaElement.manifest which is an object that exposes the high level manifest timeline and descriptor data:

media.manifest = {
periods: [
    {
        start: <mediaTime>,
        duration: <mediaTime>,
        isRemote: <boolean>,
        descriptors: [{ // all descriptor element types
            schemeIdUri: <string>,
            value: <string>
        }]
    }
]
}

Each of our periods contains a custom descriptor scheme which has one of three values: "content", "bumper", "ad"
Our application UI sometimes operates in content time and sometimes operates in media time, it depends on the asset type and subscriber entitlement, so with the stream exposed as a single timeline with proper scheme decoration the application can easily cross convert between stream and content time, even as the ad payloads change during the session
To aid state management we do expose "xlink resolving" and "xlink resolved" event messages that allow the application to know when underlying changes to the stream composition have been made along with hints about what the change was.

This is not to say this is the right or recommended approach, but an example of how this variability in timeline and signaling of changes can be done.

some things I think would be good for you to add to the spec:

Standard way to signal "this is an ad" that would work cross-player.
We can use Events to embed metadata about the ads; i.e. show a button with this text; show this picture. Is that the recommendation?

Having all this defined in the spec; is easier to influence 3P devices & players supporting streams that are powered this way.

@technogeek00

Other things I can think of:

How should fast forwarding be handled? i.e. if a user skips past all of the xlink periods, how would we force resolution of one of the ad periods?
How will the protocol help with communicating whether the ad could be skippable or not?

On the first two:

Standard way to signal "this is an ad" that would work cross-player.

Good callout, this is underspecified as is and should be straightforward

We can use Events to embed metadata about the ads; i.e. show a button with this text; show this picture. Is that the recommendation?

Yep that is the recommendation as it stands.

How should fast forwarding be handled? i.e. if a user skips past all of the xlink periods, how would we force resolution of one of the ad periods? How will the protocol help with communicating whether the ad could be skippable or not?

These are good questions and I think they are addressed together, this may tie in with signalling what is and isnt an ad as well. perhaps there is a signal of intent by the author around skip enforcement. Typically this is delegated to application logic above the player, but providing this functionality would also align with HLS Interstitial work.

Hi @Ronak2121 , thanks for raising these points I created an issue to extend the use of AssetIdentifier https://github.com/Dash-Industry-Forum/AdInsertion/issues/84 to use in MPD Events, that would allow explicit signalling that a piece of the content is an ad by using an MPD Event, in addition I setup a sample stream with AssetIdentifier with ad-id https://pl8q5ug7b6.execute-api.eu-central-1.amazonaws.com/7.mpd

@technogeek00 @ronak2121 If I understood correctly, Hulu has been following HLS interstitial way of resolving XLink. Using separate players for content and ad. Signaling of content and ad is done using Event data in MPD. Also Another thing I was trying on Shaka Player was If Remote Period (referring XLink onRequest) has duration then I would have duration of entire playback in the start and would resolve xLink when needed and there would not be need for separate players like HLS interstitial.

If I understood correctly, Hulu has been following HLS interstitial way of resolving XLink. Using separate players for content and ad.

Minor correction, the Hulu player is DASH based and performs X-Link within a single player instance with one buffer per elementary stream type. The same concept can be extended to HLS Interstitials so that you do not use two players, its actually very impractical to use two players.

Also Another thing I was trying on Shaka Player was If Remote Period (referring XLink onRequest) has duration then I would have duration of entire playback in the start and would resolve xLink when needed and there would not be need for separate players like HLS interstitial.

The duration on the placeholder period with xlink decoration has no relation to the result, there is no requirement for it to be accurate. XLink may replace one period with multiple, the player must be built to dynamically adjust the timeline.

Thanks @technogeek00 for the info. I think once the player knows if its ad or content, content and ad timeline would be independent. This is what you mean right. Once ad starts, player timeline would reset to 0 - (break duration) and when ad completes, player timeline would resume from last viewing time. Thanks once again for helping me understand all these things. Hopefully will get chance to work with you.

@technogeek00 two things which are implicit but clarifying with you.

In the above comment you had mentioned that it would be unique request for xlink / patch so you have ExtUrlQueryInfo defined at Period level (xlink period). I did not find example for period level in dash spec doc (https://dashif.org/docs/DASH-TAC-v1.0.htm#_Toc532379025).
Also, there is not something like headerParamSource for query case in DASH spec. I think it would have been better if DASH spec allows dynamic values for query string as well. It seems like its allowing only hardcode values for query string. For both the cases, I have not found any examples in DASH spec. For Now, I have defined queryParamSource and implemented accordingly and defined ExtUrlQueryInfo at xlink period level for query template.

Hey all, sorry for being so late to the party, but I'd like to discuss my hesitance for relying on XLink for dynamic ad resolution, and also propose an alternative approach.

Benefit of XLink

I like the use of XLink in DASH as a means of lazy loading sections of the MPD that are not immediately needed for the user's current experience. An example of this is a gigantic multi-Period manifests (e.g. 10h dynamic manifest) where we only load the latest Period unless the user scrubs back. Another example is a manifest with many audio language/codec options where we only resolve AdaptationSet if user switches audio selection. In both these cases the remote element will not alter the timeline and therefore there is no risk for it invalidating any other presentation time statements elsewhere in the MPD (e.g. MPD@mediaPresentationDuration, next Period@start, etc.). Furthermore, section 5.5.3 Processing in ISO/IEC 23009-1 ed.5 indicates that the XLink resolution will not modify MPD parameters outside of the context of the XLink element being resolved.

Issues Applying to DAI

Dynamic ad insertion will inevitably lead to different timings being inserted/replaced into the manifest, which will invalidate other timing information declared by the MPD, and thus leads to statements such as above being a MUST:

The duration on the placeholder period with xlink decoration has no relation to the result, there is no requirement for it to be accurate.

In my opinion I feel this is a less than ideal solution. I don't actually see any reference to this interpretation in the spec and I would assume that an MPD is still valid after all XLinks are resolved. If any player has already implemented my interpretation then this new interpretation is a breaking change.

Furthermore, XLink now has a dual responsibility (lazy load vs. splicing in discontinuous content), which leads to the need to indicate what "type of XLink" is being included in the MPD:

Each of our periods contains a custom descriptor scheme which has one of three values: "content", "bumper", "ad"

[...]

To aid state management we do expose "xlink resolving" and "xlink resolved" event messages that allow the application to know when underlying changes to the stream composition have been made along with hints about what the change was.

I believe this leads to complexity in MPD handling for the DASH client.

There are other limitations; consider a Period replacement that had valid underlying fallback content and a situation where the XLink DAI resolution leads to a large under-fill. The rules of MPD Assembly state that the whole Period is replaced, but now we've lost a lot of timeline that could have been filled with fallback content, and this could be fatal in a dynamic MPD scenario.

And as mentioned elsewhere, to add more metadata about ad processing, we have to pull in multiple other DASH concepts (e.g. essential/supplemental properties, other event schemas, etc.).

XLink has success in DASH and it feels like it should fit for this use-case... Don't get me wrong, all of my concerns have solutions... But this all seems a little reminiscent of class Square extends Rectangle (just a little humor, no offense meant).

Apple HLS Interstitials

Apple's solution to SGAI was to create a specification dedicated to solving the problem. It has some nice properties in that it can define extra ad rules and also explicitly keeps ad timeline separated from primary content timeline. If an implementor has explicit control over system resources to properly optimize the solution they may choose to use a "dual player" like solution, or otherwise they can use a single timeline as is assumed with XLink, but with a clarity in the MPD timeline truth vs any computed timeline as a result of dynamic asset resolution.

DASH Interstitials

Similar to HLS EXT-X-DATERANGE, DASH has the EventStream element that provides extra event information at the MPD level with accurate timing, and importantly pre-existing specification to signal custom schemes for interpretation.

I suggest a formalization of a new schemeIdUri for interstitial content (e.g. https://dashif.org/guidelines/dashif#interstitial). The initial schema can mostly take ideas from HLS; however, we have room to add concepts and have more DASH-centric specifiers.

I won't define anything exactly now. Below I provide an EventStream that is intended to be used as a conversation starter.

<EventStream
    schemeIdUri="https://dashif.org/guidelines/dashif#interstitial"
    xmlns:interstitial="https://dashif.org/guidelines/dashif#interstitial"
    xmlns:up="urn:mpeg:dash:schema:urlparam:2014">

    <!-- duration can indicate how much timeline the Event should consume -->
    <Event duration="0" id="1" presentationTime="60">
        <!--
            Allows for flexible insertion of parameters from the MPD URL. This is useful
            for when we want to provide personalized requests while keeping high
            cacheability of the MPD.

            Ideally I'd like to just include one `EssentialProperty` at the root of the
            `EventStream` that follows `up`; however, I'm not sure that is allowed. I'm
            also not sure if `up` can be extended to apply for `interstitial:AssetList`
            below as is, or if it is better to just re-define it within the
            `interstitial` namespace.
        -->
        <up:UrlQueryInfo
            queryTemplate="ad_params=$query:personalized_ad_params$"
            useMPDUrlQuery="true"/>
        <!--
            The AssetList can link to a JSON object similar to the HLS format which helps
            with interoperability.

            We can also introduce some URL templating to allow for more client driven
            information. Here I give examples intending to mean the following:
            - $EventID$ - The id of the containing Event.
            - $StartOffset$ - How far into the interstitial the DASH client is starting.

            These are just some ideas. I also believe that "Annex I: Flexible Insertion
            of URL Parameters" should also apply here.
        -->
        <interstitial:AssetList>
            <![CDATA[https://example.com/interstitial.json?format=dash&id=$EventID$&offset=$StartOffset$]]>
        </interstitial:AssetList>
        <!--
            Similar to HLS X-SNAP; however, we can add more information, such as what is
            the maximum duration we will allow the interstitial to snap to, which is
            provided by `roundingLimit` (duration subject to timescale). Another idea is
            to add a new enum case such as `PERIOD` which could be used in cases where
            the MPD is conditioned to already be multi-Period and the interstitial is
            intended to replace the content. Setting case `PERIOD` with a rounding limit
            could ensure that the whole Period is replaced; however, in the case of under
            fill, that some primary content will remain.
        -->
        <interstitial:Snap roundingLimit="1">OUT,IN</interstitial:Snap>
        <!--
            Restrict mimics the HLS definitition which is nice as it allows for ad rules
            to be defined in the server in a consistent way.
        -->
        <interstitial:Restrict>SKIP,JUMP</interstitial:Restrict>
    </Event>

</EventStream>

Some of the benefits I see are:

Backwards compatible: DASH clients that don't understand the scheme will just ignore it.
Ads are MPD: The remote AssetList defines actual MPD assets which is a simplification as implementors do not need to maintain any separated storage or JIT manifest XML construction to deliver just Period elements (as required by XLink).
MPD integrity: The MPD remains valid as any timeline changes for the DASH client are separated from the MPD timeline. Internally the DASH client may have to deal with timeline changes but that is computed on top of the truth for primary content derived from the MPD. Put another way, the responsibility of MPD resolution can be separated from timeline computation / post-processing.
Simple stitcher implementation: EventStream with SCTE35 messages are already commonly used to signal ad opportunities. This makes adoption of this simple as a server-side stitcher can simply replace the SCTE35 EventStream with an interstitial version based on the signaling and not have to worry about complexities of Period splitting.
XLink compatibility: This approach can be used in conjunction with XLink without any conflict. It also allows XLink to maintain the single purpose of lazy loading components of the primary MPD.
HLS Interoperability: The approach is quite in line with HLS and the interstitial server could share many components with one implemented for HLS.

Disclaimer

I have yet to implement a POC with the suggestion as it is quite fresh in my mind. I am quite keen to hear opinions on this though and can look to get something put together to validate the approach. I hope to at least spark a conversation on the topic. I'd be happy to discuss any ideas at one of the Friday ad insertion taskforce meetings (though I have yet to attend one).

@technogeek00 @ronak2121 any feedback would be greatly appreciated.

Dash-Industry-Forum / AdInsertion