Support Media Source Extensions and EME

[SergioCrisostomo]

This is a feature request question.

W3C defined specs for Media Source Extensions and Encrypted Media Extensions. This specification allows JavaScript to dynamically construct streams for <audio>. It defines a MediaSource object that can serve as a source of media data for an HTMLMediaElement, instead of just passing a URL.

There are more advanced players like DASH, HLS, Shaka, etc. These are very complex video players that can play audio if the audio format is following their standard (dash .mpd manifest, or HLS .m3u8 manifest). But there is a use case for more simple and generic uses where the app using has its own logic for audio streams.

I think it would be good to be able to support the W3C standard in its most simple form, ie:

• enabling the use of MSE
• enabling Clear Key encryption events and callbacks

In practical terms this means:

• adding a callback for EME events encrypted, message, keystatuseschange
• adding mechanism to set the length display so we can show the track length
• enable seek functionality
• (maybe, not tested) correct the loaded progress bar to show properly the chunks that are loaded

I suggest minimal additions in the H5 player, as far as possible putting the MSE/EME logic on the consumer of the player. These changes would expand the use case of this player to consuming audio streams.

I am happy to implement this if you think might be useful. Happy to hear your thoughts.

[lhz516]

Very interesting!

I did some research on MSE and still not fully understand how plain audio and SME work together. For example, will audio.currentTime = 123 still work with SME? Is it hard to support both plain audio and SME?

Another concern I have is that will this feature significantly increase the bundle size of the library? How do you make the "minimal additions" to this library? What top-level API (props) changes are you proposing?

[SergioCrisostomo]

Happy you found it interesting!

The audio API is unchanged, audio.currentTime will still work without changes. What we don't have (unless the consumer sends in that info as a prop is the total length/duration of the file, since it will be downloaded in chunks in different files.

I don't think the bundle size will increase significantly. For example the current ended event has this footprint:

audio.addEventListener('ended', (e) => {
  this.props.onEnded && this.props.onEnded(e)
})

API changes:

• We would need to add the same for encrypted, message, keystatuseschange.
• We would need to receive duration as a prop (the consumer should provide this data to the component) and either implement in or ignore duration and render the this.props.duration value passed to the component. Very litle change needed here also.

The seek and progress bar status are internal changes. For the progress bar we need to know the chunks sizes, so basically split the progress bar in parts. The length of each chunk should also be provided by the consumer.

These should be small changes, the consumer should take the heavy load.

If you are positive to this still let me know and I can send a PR so you can see changes it requires.

[lhz516]

Thanks for the explanation.

Look like the work of adding new props is straight forward. The hardest part is loading chunks and connect them with the progress bar. The behavior would be very different from the existing progress bar. So I have one thing to discuss before you create a PR. Are you planning to create a new progress bar component and use it with SME; or modify the existing ProgressBar to support both?

[SergioCrisostomo]

Quite true, I am not sure the MSE api fires progress events the same way. I did check now and seems like it behaves the same way so the progress bar logic could stay untouched.

[lhz516]

I see. Sounds good to me! I'll give a review once you create a PR. Thanks for the initiative!