search

w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
987 stars 136 forks source link

https://w3c.github.io/webcodecs/#dom-videoframe-videoframe-data-init should validate the plane layout according the coded width and height #702

Open youennf opened 1 year ago

youennf commented 1 year ago

https://w3c.github.io/webcodecs/#dom-videoframe-videoframe-data-init step 9 is computing the plane layout according parsedRect which is computed from visibleWidth/visibleHeight. It seems codedWidth and codedHeight should be used instead, which would be consistent with checking whether the buffer is large enough. See also https://bugs.webkit.org/show_bug.cgi?id=256848.

youennf commented 1 year ago

Switching to using codedWidth/codedHeight seems to align with what Chrome does AIUI: https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/webcodecs/video_frame.cc;l=849;drc=a3636d1ffd79769d79cb736dcf0066eeef408762

sandersdan commented 1 year ago

Agreed, combinedLayout is only used to run the check in step 11, which should be making sure that the buffer is large enough to contain the entire coded region of the frame. It's not necessary to compute anything related to the visible region here, but Compute Layout and Allocation Size algorithm is shared with copyTo() which does.

The complexity comes from having to support explicit and implicit layouts. The actual check just needs to ensure that the buffer contains the full planeCodedHeight * planeStride bytes for every plane.

It could be worth making this more obvious because using planeCodedWidth * planeSampleBytes for the last row instead of planeStride might be logical but would differ from the spec intention.