Playing HLS frame-accurate?

[matanmarciano]

I'm playing live HLS (which I have control on the server-side) and I'm trying to trigger events on the client-side on some predetermined timestamps (according to the HLS timeline). If I'm working with 'SinglePeriodTimeline.windowStartTime + player.getCurrentPosition()', is it guaranteed to be deterministic and frame-accurate? If so, is it also safe to use 'SinglePeriodTimeline.windowStartTime + player.getCurrentPosition()'' for accurate seekTo() (again, content-dependent and should be frame-accurate)? Note: these timestamps are determined in frame-exact accuracy according to the stream's content (scene changes).

[kim-vde]

For listening You can use player.getCurrentPosition() to know whether the position required has been reached. This is not deterministic or frame-accurate though, as the position is recomputed approximately every 40 ms. The event will be fired when the frame is shown or very shortly after.

If you need more precision, you can use a VideoFrameMetadataListener (see SimpleExoPlayer.setVideoFrameMetadataListener()), which gives you the frame release time in nanoseconds.

SinglePeriodTimeline.windowStartTime is retrieved from the EXT-X-PROGRAM-DATE-TIME tag. If not provided, it is set to Long.MIN_VALUE + 1.

Note that, if you use HLS discontinuities, the timestamps will no longer match with what you set server-side.

For seeking Seeking to an exact frame should work, assuming that the SeekParameters is set to EXACT, which is the default, and that the timestamp corresponds to a sample timestamp (you cannot seek in the middle of a sample).

[matanmarciano]

Tnx for answering @kim-vde , onVideoFrameAboutToBeRendered raised with presentationTimeUs that calculated by your internal logic and releaseTimeNs that is the current nano time. so both of these values not really what I am looking for.

I need to start playing/seek to some point (triggered by server-side). Maybe can I get the pts value (from the MPEG TS packet) of the current decoded/rendered frame?

[kim-vde]

I am not sure I understand what you are trying to do.

You know the timestamp in advance, right? Do you want to trigger an event when the playback has reached this timestamp? To which position do you want to seek? Why do you need frame accuracy?

[matanmarciano]

OK I will try to provide some deep explanation. We have a live stream (HLS) of football (just for example:)). there was a game clock/streaming clock (for most is 90min, 00:00:00 to 01:30:00). When I launch the exoplayer with this stream, I located a few seconds back to the game clock..lets call to this "delay" as X. I want to change (in real time) this X value..it means to be X ms back from the "game clock". I need this a pretty accurately so if I use 2 exoplayer instances at the same activity and "move" them to the position that is back X ms from the game clock and pause immediately, I will see the same frame. I can leave with difference of a few frames..

[kim-vde]

Ok, thanks for the clarifications. If I understand correctly, you want to seek to liveTime - X when a user starts playing the video.

I think you could use what you suggested originally. You might have a difference of a few frames, but not more.

[matanmarciano]

So actually it is not working.. Please follow the senario:

one activity with two Exoplayer instances. when the activity is start I pull X value from server side. I prepare the exo#1...play for a while and after a ~2min I prepare the exo#2.. now, I have two exo instances that plays the same stream but not located at the same position.. so I have some different thread that need to "fix" them to be at the same X..

[kim-vde]

What you are trying to do is still not clear to me. My understanding is that you are trying to execute the following steps:

• Create one activity with 2 player instances
• Retrieve the value x = liveTime - delta at which to start from the server
• Prepare and start the first player instance from time x
• Wait for a while
• Prepare and start the second player instance from the current position of the first player

If this is right, then you will need some more advanced mechanisms to have the 2 players synchronised at the exact same frame. Indeed, even if you manage to synchronize the 2 players at the start of playback, they might get unsynchronized over time. Issues #4880 and #2855 are probably interesting to look at.

[matanmarciano]

OK many thanks for your answers. I will take a look