search

webtiming / timingobject

Defines the HTML5 Timing object at the basis of multi-device synchronization matters
25 stars 4 forks source link

Report state of a timingsrc #33

Open chrisguttandin opened 1 year ago

chrisguttandin commented 1 year ago

Currently there is no way to know how good a media element controlled by a TimingObject is actually playing in sync. The media element might not have the audio/video data to play the currently requested samples/frames or it might not be able to play for some other reason like autoplay restrictions.

It would be great to get some sort of report on how the well the playback is actually in line with what the TimingObject dictates.

ingararntzen commented 1 year ago

Hi Christoph.

The mediasync wrapper referenced in timingsrc does provide some information about this. Also, any decent implementation would need to maintain such information internally, so it makes sense to export this as well.

https://webtiming.github.io/timingsrc/doc/api_mediasync.html

However, the mediasync wrapper is a best effort proof-of-concept implementation - so its quite possible that the api doc is not up date. Maybe @Snarkdoof can comment on this?

chrisguttandin commented 1 year ago

I did make an effort to implement the media element synchronization as well over here. It holds this info internally as well. At least as accurate as it can be when using the media element. However I don't really know how to expose it.

I hesitated so far to add a "sync" event or something similar since that would imply that there is a universally valid definition of something being in-sync or not. I guess it means different things for different applications and use cases.

On the other hand emitting an event every time the playbackRate or currentTime of the media element gets tweaked would be very noisy.

Therefore I think a getter would be best. It could be called at whatever rate necessary. It could expose the actual offset and each application could decide how to interpret that value. But I have no idea where to put it. Extending the MediaElement with another property/method is maybe not the best idea. Similarly adding it to the TimingObject isn't possible either because it has no idea about the synchronized MediaElement.

A different approach would be to use the bufferedamountlow event on the RTCDataChannel as a source of inspiration here. It can be configured by setting the bufferedAmountLowThreshold. Additionally there is also a bufferedAmount property to read the current value.

Copying the exact same behavior would mean to add an event and two properties to the MediaElement and any other object which accepts a timingsrc. I'm not sure if that's a good idea.

ingararntzen commented 1 year ago

In our implementation this state would be associated with the media sync wrapper object.

I think it should be ok to emit an event every time one adjusts the media player. You should not adjust the player too often. A getter which tells you the current diff between timing object offset and media offset would be good.

I think it would be wrong to associate this status information with the timing object though. The timing object does not know for what purpose it is being used. If the media player handles sync with timing object internally, then it make sense for the media player to provide status information about sync, in my opinion.