w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
978 stars 136 forks source link

Should spec validate buffer size by codedSize when constructing VideoFrame from a buffer #516

Open ChunMinChang opened 2 years ago

ChunMinChang commented 2 years ago

When constructing VideoFrame from the buffer, the buffer size (a.k.a. data.byteLength) must be bigger than or equal to the allocationSize of result of Compute Layout and Allocation Size, in step 11. However, if the (codedWidth, codedHeight) is (3, 3) and visibleRect’s (x, y, width, height) is (0, 0, 1, 1), giving a buffer with 2 x 2 x 4 size that contains 2-pixel width, 2-pixel height data in RGBA format can pass that check. Is this expected? Is it allowed to construct a 3 x 3 RGBA buffer with 1 x 1 visible rect by a 2 x 2 RGBA buffer data?

FYI, Chromium refuses to construct a VideoFrame in this case:

Uncaught TypeError: Failed to construct 'VideoFrame': data is not large enough.

// test case
let data = new Uint8Array([
  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
]);
let frame = new VideoFrame(data, {
  timestamp: 10,
  codedWidth: 3,
  codedHeight: 3,
  visibleRect: { x: 0, y: 0, width: 1, height: 1 },
  format: "RGBA",
});
sandersdan commented 2 years ago

The spec is supposed to leave room for an implementation to apply a crop to the data that still includes the whole visible rect, but it shouldn't be assuming that behavior. The app can always just pass a smaller coded size to crop the data on the right/bottom instead.

The byteLength check should be against the coded size, not parsedRect.

The Chrome implementation otherwise matches the spec, c.f. visible rect validation and layout validation.