search

w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
941 stars 132 forks source link

Should VideoEncoderConfig cloning remove parameters that are not useful for a given codec? #681

Open youennf opened 1 year ago

youennf commented 1 year ago

According https://w3c.github.io/webcodecs/#clone-configuration, the configuration should be cloned, hence all members of VideoEncoderSupport should be copied.

Chrome in https://wpt.live/webcodecs/video-encoder-config.https.any.html assumes that the avc member will not be cloned if the codec is vp8. Safari simply clones all members it knows.

The note in https://w3c.github.io/webcodecs/#dom-videoencoder-isconfigsupported says that only recognised dictionary members will be copied. recognised could be interpreted both ways. It would be good to clarify the intent here.

sandersdan commented 1 year ago

The intent was to support feature detection. More generally, the returned configuration should be the user agent's interpretation of the supplied configuration.

I would say that the spec as written implies that all known IDL properties should be copied.

For the purpose of a developer debugging WebCodecs operation, I'd say that Chrome's behavior is perhaps more helpful.

Based on that, I would vote for Chrome's behavior, but it might not have an obvious algorithm to specify.

youennf commented 1 year ago

The intent was to support feature detection

I would think feature detection is done at WebIDL level, parameters not recognised will not be copied. The only case would be a feature supported for one codec but not for another codec. Is that a real use case?

Overall, we might end up in bad interop here between browsers, which might be more hurtful than anything else. I would vote for either defining an algorithm or keeping it simple like the spec is today and wait for web developer feedback.

sandersdan commented 1 year ago

The only case would be a feature supported for one codec but not for another codec. Is that a real use case?

I suppose that's the key question. Right now I don't think we have any options that make sense to support only in some codecs, except for the explicitly codec-specific options. I can imagine bitrate control getting complex enough to have variability, but that hasn't happened yet.

I think I'm comfortable with either fixing Chrome to match the current spec, or changing the spec to exclude coded-specific options that do not apply to codec.

youennf commented 1 year ago

I think I'm comfortable with either fixing Chrome to match the current spec

I think I would prefer this at it is simpler to spec and implement. We can always revisit whenever we see a real need.

chrisn commented 1 year ago

In the last Media WG meeting, we discussed adding hints for screen content coding tools, which would be applied in codecs that support them. Would this affect cloning? Maybe not, as it's just a hint?

Djuffin commented 1 year ago

I think feature detection won't suffer if we just clone every field known to the user agent. Codec specific fields that belong to a different codec don't have any effect anyway.

As for hints, they need to be copied if the user agent knows about them regardless if they had any real effect on configuration of the underlying encoder.

Djuffin commented 12 months ago

Media WG discussion on 7/11/2023:

  1. Make spec more explicit that even fields that don't affect configuration (like codec specific fields) need to be copied even if they don't affect configuration in this particular case.
  2. File a bug against Chromium
youennf commented 10 months ago

WPT PR up for review.