search

w3c / media-source

Media Source Extensions
https://w3c.github.io/media-source/
Other
268 stars 57 forks source link

Potential interop problems around appendBuffer()->QuotaExceededError #201

Open wolenetz opened 6 years ago

wolenetz commented 6 years ago

While Chrome, Firefox and Edge all appear to support pushing back on the web app and yielding a QuotaExceededError exception when coded frame eviction doesn't evict enough to make room for the appendBuffer() being attempted, Safari does not.

From comment https://trac.webkit.org/changeset/172657/webkit, it appears there may be some lurking interop issue. Comment from that change:

For now we DO NOT throw a QUOTA_EXCEEDED_ERR when we are not able to free up enough space to remain below the prescribed quota, because some big name, widely deployed, code bases ignore the error and continue appending data as though the failed append succeeded, leading to a corrupted bitstream and failure to play.

@eric-carlson, @jernoble - In case there indeed is some lurking interop issue and not just an implementation issue, I've filed this spec bug. Please comment, especially if there is some blocking issue with how this is spec'ed.

jyavenard commented 6 years ago

I'm puzzled on why it would lead to a corrupted bitstream.

If the eviction didn't work for the first appendBuffer, why would it not reject on the second appendBuffer as well.

But assuming it did manage to evict data between the first appendBuffer and the second (like playback has continued)

There's already provision for this case in the Coded Frame Processing algorithm. The first segment got refused due to QUOTA_EXCEEDED_ERR. The second appendBuffer appended would fall into step 6. https://w3c.github.io/media-source/index.html#sourcebuffer-coded-frame-processing

"If last decode timestamp for track buffer is set and the difference between decode timestamp and last decode timestamp is greater than 2 times last frame duration"

which has a consequence of doing "Set the need random access point flag on all track buffers to true."

The 2nd appendBuffer would certainly add frames with a dts greater than the last one + 2xframe duration.

And how would not firing QUOTA_EXCEEDED_ERR change anything on the internal handling?

Having said that: To me the only major downside of QuotaExceededError is that it's a synchronous behaviour when the entire MSE is pretty much designed to be asynchronous.

It would have been much better to have appendBuffer fire an "error" event with QuotaExceededError attached to it.

It would have prevented from an implementation perspective to have synchronous cross-thread/process communication needed.

wolenetz commented 6 years ago

I suspect if one does a "one byte at a time" (or other sub-media-segment chunked) sequence of appendBuffers(), then most appends will not begin at a media or initialization segment boundary, causing parse (decode) error on append if the app ignores that some of the appends in the sequence failed.

I think the comment in WebKit may have had validity back when apps didn't expect QuotaExceededError could happen, but that has very likely changed since then (again, given Chrome, Edge and Firefox can give this exception).

wolenetz commented 3 years ago

Are apps still struggling with some implementations not providing the QUOTA_EXCEEDED_ERR? If so, this seems an implementation issue.