Getting current PTS in HLS livestream?

saml commented 4 years ago

Searched documentation and issues

Question

I want to detect if the current position is at or has passed a certain PTS in an HLS livestream so that I can perform arbitrary actions (such as applying different visual style). For example, if it's at 00:10:31 (or has already passed that mark because the player joined the livestream late), change the background image.

When the HLS manifest includes #EXT-X-PROGRAM-DATE-TIME, I can use the following to get current timestamp:

final int windowIndex = player.getCurrentWindowIndex();
final Timeline timeline = player.getCurrentTimeline();
final Timeline.Window window = timeline.getWindow(windowIndex, new Timeline.Window());

// This is not really PTS but good enough info to use:
window.windowStartTimeMs + player.getCurrentPosition()

In many cases HLS manifest author doesn't include #EXT-X-PROGRAM-DATE-TIME, and without such tag, it looks like I cannot get PTS (or timestamp) of the livestream. I tried the following:

// presentationStartTimeMs is C.TIME_UNSET unfortunately.
window.presentationStartTimeMs + player.getCurrentPosition()

But, window.presentationStartTimeMs is C.TIME_UNSET.

Segments do have PTS:

$ ffprobe -v error -of json -select_streams v -show_packets https://cph-p2p-msl.akamaized.net/hls/live/2000341/test/segment_1_20200803_1596481155.ts | jq .packets[] -c | head -1 | jq .
{
  "codec_type": "video",
  "stream_index": 0,
  "pts": 1292217244,
  "pts_time": "14357.969378",

pts_time, 14357.969378, is about 03:59:18 . And using a different player, I can verify that:

$ mpv --rebase-start-time=no --osd-level=3  https://cph-p2p-msl.akamaized.net/hls/live/2000341/test/segment_1_20200803_1596481155.ts

The video starts with 03:59:18 (with --rebase-start-time=no , mpv doesn't start with 00:00:00 but displays PTS as-is).

This livestream is from https://ottverse.com/free-hls-m3u8-test-urls/ . And I'm not sure how long it'll last. I used https://cph-p2p-msl.akamaized.net/hls/live/2000341/test/level_1.m3u8 (from master.m3u8).

Would it be a good idea to set window.presentationStartTimeMs to PTS of the first packet/frame of the current window? Or, I think adding that to Timeline.Period is better because a window may contain multiple periods including foreign segments (such as ad). And, if we only expose "start time" of a window, instead of a period, it'll be difficult to get the current position (PTS or some timestamp).

For example, window.windowStartTimeMs + player.getCurrentPosition() doesn't accurately describe where the player is at in the "main" HLS stream. If I have the following manifest:

#EXTINF:4.000000,
#EXT-X-PROGRAM-DATE-TIME:2020-07-30T20:00:00.000Z
output303.ts

#EXTINF:10,
#EXT-X-DISCONTINUITY
ad.ts

#EXT-X-DISCONTINUITY
#EXTINF:4.000000,
#EXT-X-PROGRAM-DATE-TIME:2020-07-30T20:00:04.000Z
output304.ts

I expect window.windowStartTimeMs + player.getCurrentPosition() to be around 2020-07-30T20:00:04 when it's playing output304.ts segment. But, since window hasn't changed (this particular short example wasn't livestream, but vod), window.windowStartTimeMs is still at 2020-07-30T20:00:00 and player.getCurrentPosition() includes those 10 seconds from ad.ts. So, window.windowStartTimeMs + player.getCurrentPosition() results in 2020-07-30T20:00:14 .

The question is, how do I get current PTS or current timestamp (based on #EXT-X-PROGRAM-DATE-TIME) in HLS even in the presence of discontinuity?

A full bug report captured from the device

N/A

Link to test content

https://cph-p2p-msl.akamaized.net/hls/live/2000341/test/level_1.m3u8 (I don't own this. It's from https://ottverse.com/free-hls-m3u8-test-urls/ .)
https://github.com/saml/exoplayer-example/blob/master/fixture/hls-with-audio/output-date-time.m3u8 (This is for testing #EXT-X-PROGRAM-DATE-TIME with an ad in the middle)

AquilesCanta commented 4 years ago

Thanks for the high quality issue, I appreciate there's enough information for me to work with. I think there are two questions here:

How do I get the current accumulative playback position in a live stream, when the media playlists don't have a EXT-X-PROGRAM-DATE-TIME.
How do I get the content position when there are stitched-in ads in the stream.

For 1: I think you can use something like:

timeline.getWindow(player.getCurrentWindowIndex(), new Timeline.Window()).positionInFirstPeriodUs / 1000 + player.getCurrentPosition() where positionInFirstPeriodUs will be the cumulative position of the window start.

For 2: I think the content is not spec compliant because it generates an ambiguous mapping between EXT-X-PROGRAM-DATE-TIME tags and media segments. Both EXT-X-PROGRAM-DATE-TIME tags say that output304.ts corresponds to different walltimes. The only reason it works is that ExoPlayer ignores the second EXT-X-PROGRAM-DATE-TIME tag. From the spec:

   The Server MUST NOT add any EXT-X-PROGRAM-DATE-TIME tag to a Playlist
   that would cause the mapping between program date and Media Segment
   to become ambiguous.

Aside, ExoPlayer doesn't know that you are stitching in ads. It's all content. What I advise you do is that you listen for Player.EventListener.onTimelineChanged and keep track of the content position yourself by subtracting the ad duration yourself. You can obtain the current playlist from the window. See Window.manifest. It may not be easy to do because the EXT-INF associated to the ad is not precise but there is nothing much you can do without using a more complex solution.

saml commented 4 years ago

@AquilesCanta Thank you!

(window.positionInFirstPeriodUs / 1000) + player.getCurrentPosition()

Gives me elapsed time since the player joined the livestream. For example, if the livestream has been going on for 30 minutes, and I open the player to view the livestream now, (window.positionInFirstPeriodUs / 1000) + player.getCurrentPosition() would give me ~10 seconds, instead of 30 minutes. I need the "30 minutes" to decide where in the livestream the player is at.

Thank you for correcting me about wall clock. I'll subtract ad duration if I insert ads to the livestream as you advised.

Since there's seems to be no way to get current PTS, I'll add #EXT-X-PROGRAM-DATE-TIME to the manifest and subtract ad duration, if any.

AquilesCanta commented 4 years ago

I need the "30 minutes" to decide where in the livestream the player is at.

And where is that information available?

saml commented 4 years ago

@AquilesCanta I could see in packets/frames of a segment:

# Displaying the first frame of a segment called output302.ts
$ ffprobe -v error -of json -select_streams v -show_packets output302.ts | jq .packets[] -c | head -1 | jq .
{
  "codec_type": "video",
  "stream_index": 0,
  "pts": 108852000,
  "pts_time": "1209.466667",
  "dts": 108846000,
  "dts_time": "1209.400000",
  "duration": 3000,
  "duration_time": "0.033333",
  "size": "6465",
  "pos": "564",
  "flags": "K_",
  "side_data_list": [
    {
      "side_data_type": "MPEGTS Stream ID"
    }
  ]
}

This pts_time (1209.466667 this is in seconds) is something I want to read. If there's something like Timeline.Window.startPtsUs, I could do window.getStartPtsMs() + player.getCurrentPosition().

google / ExoPlayer