w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
975 stars 137 forks source link

Add a way of specifying frame duration to FLAC encoder config #595

Closed Yahweasel closed 1 year ago

Yahweasel commented 1 year ago

This is a followup to #405. #551 added a frameDuration attribute to the Opus encoder configuration, and closed #405 since Opus is the only codec with this tuning knob supported by any browser.

I'm the author of the polyfill: https://github.com/ennuicastr/libavjs-webcodecs-polyfill . The polyfill does support FLAC encoding.

I'll be adding support for this tuning knob to the Opus encoder in libavjs-webcodecs-polyfill Soon™, but I'd like to also add it to FLAC, and ideally do so following the spec instead of deviating from the spec with the possibility of Chrome later deviating in a different way. That is, I'd like to see similar for FLAC.

dalecurtis commented 1 year ago

We'll want to update the flac codec registration with this information then.

Yahweasel commented 1 year ago

That would be my preference, yes :). AFAIK the same basic style as Opus should work fine (optional field with FLAC-specific details, optional frameDuration field in that)

dalecurtis commented 1 year ago

That sgtm, I can't seem to find a spec link for specifying duration like we have for opus though. Do you know if that's documented anywhere?

Yahweasel commented 1 year ago

https://xiph.org/flac/format.html#frame_header

In FLAC, any number of samples is technically valid in a frame. For FLAC to correctly support end of file and such situations, it has to be: the way FLAC handles EOF is a partial frame, not a full frame with some extra indicator of how much data to drop. There are certain frame sizes that are prioritized in the sense that they require less space in the header to specify them (namely, the multiples of 576 or 256 samples), but all other frame sizes can also be specified. This is of course contrary to Opus, where only certain frame sizes are valid.

Well, asterisk: the number of samples has to fit in 16 bits :)

dalecurtis commented 1 year ago

How is this specified to the flac encoder? Is it just through the samples parameter to FLAC__stream_encoder_process?

Yahweasel commented 1 year ago

https://xiph.org/flac/api/group__flac__stream__encoder.html#gac35cb1b5614464658262e684c4ac3a2f ( FLAC__stream_encoder_set_blocksize )

Default is encoder's preference, which of course should be the default here too.

dalecurtis commented 1 year ago

Thanks. Looks like we'd also want to provide compression level. Maybe bits per sample, though this could be locked in w/ the first encoded chunk.

Yahweasel commented 1 year ago

For the sake of completeness, compression level certainly could be added, but I'd consider that vastly less important. FLAC was created in a time when FLAC's highest compression was actually challenging for most machines; nowadays, my watch could probably compress a dozen FLAC streams at max compression all at once ;) . On the other hand, frames can have mandatory durations mandated by the surrounding file format or protocol, so without that, certain uses of FLAC are simply inaccessible via the API.

bdrtc commented 1 year ago

the frameDuration discussion here correspond to blockSize described in section 5.1 of draft-ietf-cellar-flac, and the valid size also describe there: minimum block size of 16 samples, and a maximum block size of 65535 samples, default value should be 0 (let the encoder estimate a blocksize by default).

for the compression level: it's described in FLAC api doc,the valid value is within the range of 0 (fastest, least compression) to 8 (slowest, most compression), default value is 5.