Negative cue times

[rjksmith]

Background

Data synchronisation work in OGC Testbed-19 has highlighted a requirement for negative cue times.

Issue

WebVMT supports a media start time which defines the start time and date that corresponds to the zero point in the media timeline. Data in a WebVMT file can be synchronised with another WebVMT file (and its media timeline) by adding a positive or negative offset to the media start time so that both files start at the same time and date. An equal and opposite offset should be added to each cue time in the modified file in order to maintain the correct timings.

If the media start time moves forwards in time, all the cue times in that WebVMT file move backwards which can result in negative cue times. HTML supports this, but the WebVMT timestamp in the 19 September 2023 Group Note does not.

Discussion

A possible workaround to avoid negative values is to set the cue time to zero instead. This has the same net effect since the current playback position cannot be less than zero. However, timing information for each negative cue is lost which means that the process is destructive and cannot easily be reversed. This undesirable consequence would be avoided if WebVMT supported negative cue times.

By contrast, HTML text track cue does support negative values for cue start time and cue end time. Hence, adding support for negative cue times to WebVMT is a simple change.

Proposal

Add negative cue time support to WebVMT to enable file synchronisation use cases to be handled with a non-destructive process.

[rjksmith]

Example: Comparison of Destructive and Non-Destructive Synchronisation Processes

WebVMT file before synchronisation process:

WEBVMT

MEDIA
start-time: 2024-01-03T12:00:00.000

00:00:05.000 -->
{ "pan-to":
  { "lat": 51.0070, "lng": -0.0020, "end": "00:00:25.000" }
}
{ "move-to":
  { "lat": 51.0130, "lng": -0.0015, "path": "drone1" }
}
{ "line-to":
  { "lat": 51.0090, "lng": -0.0017, "path": "drone1",
    "end": "00:00:10.000" }
}

Without Negative Cue Times

If the start time is moved forward by 13 seconds without using negative cues, the process is destructive and not easily reversible because timing information is lost.

WEBVMT

NOTE Start time moved forward by 13 seconds

MEDIA
start-time: 2024-01-03T12:00:13.000

NOTE Destructive process
Timing information lost, so process is not easily reversible
Cue start time was at 00:00:05.000
Line-to end time was at 00:00:10.000
Both values are now 00:00:00.000

00:00:00.000 -->
{ "pan-to":
  { "lat": 51.0070, "lng": -0.0020, "end": "00:00:12.000" }
}
{ "move-to":
  { "lat": 51.0130, "lng": -0.0015, "path": "drone1" }
}
{ "line-to":
  { "lat": 51.0090, "lng": -0.0017, "path": "drone1",
    "end": "00:00:00.000" }
}

With Negative Cue Times

If the start time is moved forward 13 seconds with negative cues, the process is non-destructive and completely reversible because no timing information is lost.

WEBVMT

NOTE Start time moved forward by 13 seconds

MEDIA
start-time: 2024-01-03T12:00:13.000

NOTE Non-destructive process
No timing information lost, so process is easily reversible
Cue start time offset by 13 seconds from original value
Line-to end time offset by 13 seconds from original value

-00:00:08.000 -->
{ "pan-to":
  { "lat": 51.0070, "lng": -0.0020, "end": "00:00:12.000" }
}
{ "move-to":
  { "lat": 51.0130, "lng": -0.0015, "path": "drone1" }
}
{ "line-to":
  { "lat": 51.0090, "lng": -0.0017, "path": "drone1",
    "end": "-00:00:03.000" }
}

[rjksmith]

Temporal Metadata Analysis Prior To Media Start Time

Use of negative cue times also enables temporal analysis of metadata prior to the media start time. For example, speed could be calculated for a WebVMT path prior to the media start time in order to improve context for timed metadata in the current media file - such as a racing clip.

In addition, media start time can be used to set the timeline offset of a media timeline which is already supported by HTML.