getPlayheadTimeAsDate and getPresentationStartTimeAsDate available for VOD

[stefanovualto]

Have you read the FAQ and checked for duplicate open issues? Yes

Is your feature request related to a problem? Please describe. I need to retrieve timestamps from a VOD stream. Because I want to display the UTC time contained in the stream into the UI.

Describe the solution you'd like it would be very helpful to be able to use getPlayheadTimeAsDate and getPresentationStartTimeAsDate as I do for LIVE.

Describe alternatives you've considered I am actually hacking with the getManifest which doesn't work for HLS and I think I will run into issues if there are discontinuities (might be an implementation detail, I am new in shaka player).

Thank you in advance for considering it or helping me find the best way to do it.

[TheModMaker]

For VOD content, there isn't really UTC timestamps within the media, there are just media timestamps. For example, a segment says to be played at 37 seconds. This can't really be mapped to a wall-clock time since it is just an offset from the start of the video. You could just use video.currentTime and add some offset you know about. But there isn't really anything in the manifest that would map the 37 seconds to any wall-clock time (for VOD).

The only thing I could think of is if you used absolute epoch timestamps in the media and then used presentationTimeOffset to adjust the segments back to 0 for the VOD. You could use this as a mapping to a wall-clock time; but this seems content-specific. Could you give us some example content and what you'd expect the timestamps to be?

[stefanovualto]

@TheModMaker Thank you for your answer.

Here are the content that I use for my tests:

• https://d2540nayx53mpm.cloudfront.net/vuworkflow/content/vod/9a5c02a3-c55c-493f-9d49-a7a2c508c126/9a5c02a3-c55c-493f-9d49-a7a2c508c126_nodrm_23eac2df-7c70-4040-a632-324ea444a622.ism/.m3u8
• https://d2540nayx53mpm.cloudfront.net/vuworkflow/content/vod/9a5c02a3-c55c-493f-9d49-a7a2c508c126/9a5c02a3-c55c-493f-9d49-a7a2c508c126_nodrm_23eac2df-7c70-4040-a632-324ea444a622.ism/.mpd

What I currently do (in dash only) is to find the timestampOffset in the current playing segment that can be found in the current variant from getManifest. It's hack workaround that doesn't work in HLS (getManifest isn't available. Sorry is that's a newbie question, but why?).

Do you think that even if it's content specific there is something that could be added if available?

Thanks in advance for your help and enlightenment.

[TheModMaker]

If you're using Safari, we play HLS using src=, so the browser handles all playback. You can set the streaming.useNativeHlsOnSafari configuration to false to force us to handle playback. Note that in this mode, we can't play FairPlay content.

[stefanovualto]

Thank you @TheModMaker ,

You are right, I tried it already.

But with fairplay and Raw AAC, MP3, etc (without an MP4 container) not supported, it doesn't seem a viable option.

To be honest, I thought of using HLS only on Safari, not sure that I have other choices if I want to support DRM too.

What do you think it is more likely to happen:

• Adding the support for fairplay + Raw AAC, MP3, etc (without an MP4 container)
• Adding getPlayheadTimeAsDate and getPresentationStartTimeAsDate support in VOD

Thank you for your help it is very appreciated.

[joeyparrish]

What do you think it is more likely to happen:

• Adding the support for fairplay + Raw AAC, MP3, etc (without an MP4 container)
• Adding getPlayheadTimeAsDate and getPresentationStartTimeAsDate support in VOD

We can't support FairPlay without using Apple's built-in HLS playback because Apple hasn't given us any other way to play FairPlay content. They only document how to use FairPlay with src= HLS playback, not with MediaSource-based playback. Further, Apple has still not made MediaSource available on iOS. We would very much like to support FairPlay in MediaSource-based playbacks, but it's out of our control.

And as @TheModMaker mentioned before, getPlayheadTimeAsDate and getPresentationStartTimeAsDate make no sense with VOD. These things both rely on the concept of a presentation start time, but there is no such thing in VOD content. A VOD presentation has no limits on its availability, since it is "Video On Demand". So the video timestamps have no relation to wall clock time. You can start and stop the video whenever you like.

Does this help to clarify things for you?

[stefanovualto]

Thank you @joeyparrish ,

But I was wondering if the manifest LIVE or VOD contains #EXT-X-PROGRAM-DATE-TIME would it possible to retrieve this information in a way through shaka (without having to parse and hack it my self from the manifest with an extra request)?

It could be using the shaka HLS parser for example (Which I tried but it failed because of the unsupported Raw AAC, MP3, etc (without an MP4 container))... Like that, we won't have to add this extra layer for playback that doesn't need it...

Sorry for bothering you about that issue, but don't you think that the use case is relevant when a LIVE stream becomes VOD, that the timestamp is kept in the stream?

[joeyparrish]

@stefanovualto, sorry, but getPlayheadTimeAsDate() and getPresentationStartTimeAsDate() will not be supported in VOD, even for HLS content with EXT-X-PROGRAM-DATE-TIME. We don't model start time in VOD at this time. It is, of course, possible, but not in our current manifest/timeline structure.

If you can explain more clearly why you would want this, we are definitely willing to consider it for the future.

[joeyparrish]

If you need to reopen this issue, just put @shaka-bot reopen in a comment. Thanks!

[stefanovualto]

@shaka-bot reopen

@joeyparrish thank you for letting the door open on a possible implementation of it. I will try to explain my use case and hope for it to make sense.

The context

We are providing a LIVE and VOD streaming service. In that context we offer a LIVE to VOD solution to provide catchup on-demand of live events.

Concrete use case

Working with governments they stream LIVE events daily. And give the chance to re-stream them in VOD after. As they stream in LIVE they create metadata link to the current stream, like speaker interventions, subjects discussed, etc... This metadata are based on the time when they occur. Then when the VOD stream is generated it usually get trimmed, then using a time based metadata information doesn;t require any rework to match the beginning of the stream, even in clipped meeting created for social media.

I am not sure the use case is explained well enough, but from my point of view I clearly see values in giving the opportunity to get timecodes from a VOD stream.

Let me know if this convices you, if not I will ask a colleague with an English mother tongue to advocate for this feature 😄

Thnks again for your work, shaka is a great player 🥇 !

[joeyparrish]

I think I see. You want to correlate some external stream of events to the VOD playback, based on the date and time metadata of the original HLS live stream. Is that a fair summary?

[stefanovualto]

Yes, I think that summarize it well.

[joeyparrish]

Okay. Thanks for discussing it and for being patient with us.

I'm placing the issue on the backlog for now, but I'm also tagging with ~"help wanted"~ "contributions welcome" to indicate that we would be happy to entertain a PR on this if someone else has time to work on it before we do.

The first step would probably be parsing EXT-X-PROGRAM-DATE-TIME in the HLS parser. Then we would need a place for this information to go.

One option is the PresentationTimeline, as the stream's start time. However, this could complicate things a little bit while the stream is still live. Our PresentationTimeline is used to define what's available for seeking, and without clock synchronization with the server, we need to base that timeline on the list of segments in the playlist rather than on the PROGRAM-DATE-TIME tag. If we switch to using that tag instead, the lack of clock sync in HLS will cause big issues for viewers.

If we only use the PROGRAM-DATE-TIME tag for VOD, this could work, though. The PresentationTimeline class makes a few assumptions about this.presentationStartTime_ being null for VOD. If we replace those with explicit checks against this.isLive() instead, we could safely set the start time for VOD without disrupting the timeline.

Then we would only have to change the isLive() check in Player's getPresentationStartTimeAsDate(), and the rest should just work.

[markjaniczak]

We have a very similar use case to @stefanovualto. In addition to a stream of events, we also have a stream of image thumbnails which correlate to frames at specific seek times. We'd like to be able to align those frame previews with their respective playback positions in the VOD.