search

Dash-Industry-Forum / DASH-IF-IOP

DASH-IF Interoperability Points issue tracker and document source code
33 stars 8 forks source link

UTC time corresponding to the first frame #170

Open haudiobe opened 6 years ago

haudiobe commented 6 years ago

Submitter: Will Law Email: wilaw@akamai.com There are many use-cases with dynamic (live) streams in which the player needs to know the wallclock time corresponding to the media presentation time. Some example use cases are:

  1. Synchronization of live stream playback across devices, including with broadcast devices
  2. Clipping - live to vod clip generation at in/out points defined in UTC
  3. Synchronizing sports scores, real-time data (player position, heart rate, statistics etc) with the media stream.
  4. Compensating for drift of live stream over time with respect to wallclock time.

The player can use AST to calculate when a segment becomes available, but this does not tell it when that segment was encoded. The difference will be the sum of the encoding time + any transfer time to the packager that creates the manifest. This difference is not currently signaled anywhere in the manifest, with the consequence that the above use cases cannot be cleanly addressed with DASH.

I would like to propose that a new attribute be added at the MPD level called "@UTCatEncodeStart". Its definition is "the UTC time corresponding to the first frame of content produced by the dynamic stream". The player can calculate the UTC time associated with any segment by simply adding the presentation time to this new value.

Consider the example of encoder that has as its input a camera that is filming a UTC clock. At precisely 10:00:00 on the clock the encoder begins operation. It has a 2s delay while encoding, so at 10:00:02 is produces the first frame that is showing 10:00:00 on the clock. The packager is making 5s segments. It waits to receive the first 5s of content which occurs at 10:00:07 and then produces a manifest with an @AST of 10:00:07, a @UTCatEncodeStart value of 10:00:00 and a representation @startNumber of 1. A player downloads the manifests and begins playback by requesting segment#1. Lets consider the case of any subsequent segment number, such as #3. The player knows that this segment becomes available at time (AST + (n-1)5) = 10:00:17 . It now also knows that the first frame of this segment should show the clock with a value of (UTCatEncodeStart +n5) = 10:00:15. It can then synchronize this video with external data that also references UTC, or execute any of the use cases given above.

The exact name used "UTCatEncodeStart" can be debated and changed to one that is perhaps clearer, but the functionality and purpose should remain the same.

Cheers Will

wilaw commented 6 years ago

My timing example above is incorrect. Here is the corrected version:

Consider the example of encoder that has as its input a camera that is filming a UTC clock. At precisely 10:00:00 on the clock the encoder begins operation. It has a 2s delay while encoding, so at 10:00:02 is produces the first frame that is showing 10:00:00 on the clock. The packager is making 5s segments. It produces a manifest with an @ast of 10:00:02, a @utcatencodestart value of 10:00:00 and a representation @startnumber of 1. A player downloads the manifests and begins playback by requesting segment#1 at time AST + 5 = 10:00:07. For any subsequent media presentation time of T, the equivalent wallclock for that frame is UTCatEncodeStart + T. So if the media time is 1min 45s, then the frame of video should show the clock with a time of 10:01:45. It can then synchronize this video with external data that also references UTC, or execute any of the use cases given above.

ZmGorynych commented 6 years ago

SCTE 214-1 already provides a solution in sec 11.2:

Correspondence of PeriodStart to the time of the asset may be signaled using Period.SupplementalProperty with @schemeIdUri="urn:scte:dash:assettime". The value of @value attribute shall be the timestamp corresponding to PeriodStart, as NPT or SMPTE relative timestamp, as defined in RFC 2326. Correspondence of PeriodStart to UTC time may be signaled using Period.SupplementalProperty with @schemeIdUri="urn:scte:dash:utctime". The value of @value attribute shall be the timestamp corresponding to PeriodStart, in format defined in RFC 3339. Note: The difference between the asset time and UTC time is that asset time is relative to the asset start, while UTC time is the UTC time corresponding to the acquisition time of the first sample of the period. Thus, asset time will show that a period starts at 42nd minute of an asset, while UTC time will show that the period starts with content captured on October 21, 2015 at 4:29am.