web-platform-tests / interop

web-platform-tests Interop project
https://wpt.fyi/interop
315 stars 28 forks source link

WebRTC peer connections and codecs #531

Closed jgraham closed 9 months ago

jgraham commented 1 year ago

Description

Video conferencing requires interoperability beyond compatible runtime environments, as it needs to solve negotiation and establishment of runtime audiovisual communication with other user agents directly. Sites and platforms have different codec needs, requiring APIs for customization.

Specification

https://w3c.github.io/webrtc-pc/

Open Issues

No response

Tests

Current Implementations

Standards Positions

No response

Browser bug reports

No response

Developer discussions

No response

Polls & Surveys

No response

Existing Usage

No response

Workarounds

No response

Accessibility Impact

No response

Privacy Impact

No response

Other

Note that WebRTC in general does not show up as a priority on developer surveys as it's generally used by a small number of sites that specialize in video conferencing and related services. However those sites are often very popular and compatibility issues highly visible to users (see e.g. webcompat.com issue reports)

jgraham commented 1 year ago

CC @jan-ivar

foolip commented 1 year ago

For survey data and web developer demand, in preliminary results from State of HTML 2023, WebRTC was a somewhat common response to the freeform question "Which existing HTML features or browser APIs are you unable to use because of browser differences or lack of support?"

alvestrand commented 1 year ago

The common requirement in WebRTC is that browsers MUST support H.264 Constrained Baseline and VP8. The space of possible H.264 profiles is quite confusing, and there are parameters (max-fs and max-fr) that are usually not generated or parsed for VP8.

Things that could be worked on:

There's a lot of other behaviors that could be tested, but are not so directly related to codecs.

alvestrand commented 1 year ago

Note that https://github.com/web-platform-tests/wpt/pull/36487 - a proposal that allows observing bits on the wire - would make testing codec bitstreams easier.

foolip commented 1 year ago

@jgraham can you confirm if the existing tests are enough to be useful here, or if testing of specific codec bitstreams using a mocked peer is necessary?

fippo commented 1 year ago

Since codecs beyond the MTI VP8 and H264 are optional... can we add tests for codec support gated by assert_precondition? Essentially extensions to https://wpt.live/webrtc/protocol/video-codecs.https.html

gsnedders commented 1 year ago

Since codecs beyond the MTI VP8 and H264 are optional... can we add tests for codec support gated by assert_precondition?

375 had a long debate about whether we want tests with assert_precondition in Interop.

jensimmons commented 1 year ago

Tests Anything with “codec” in https://wpt.fyi/results/webrtc Anything with “codec” in https://wpt.fyi/results/webrtc-extensions Anything with “codec” in https://wpt.fyi/results/webrtc/simulcast

Do you mean this?

Any way to create a better link to the specific tests you are suggesting?

jan-ivar commented 1 year ago

Do you mean this?

Yes, thanks for the direct links!

As to scope, I don't think we meant to preclude "codec" tests yet to be written or improved, if there's interest — in practice we find our tests grow as more implementors implement — AFAIK our WebRTC team(s) haven't participated in interop-20xx before, so we're open to input on how to best organize this.

Anything with “codec” in https://wpt.fyi/results/webrtc/simulcast ... (but there's nothing here)

Sorry, that should have read anything with "h264", "vp8" or "vp9" (we can leave out "vp9" if need be). I originally had "anything codec-related" in any of these specs, but the format of the nomination process encouraged being specific.

jan-ivar commented 1 year ago

There's a lot of other behaviors that could be tested, but are not so directly related to codecs.

I'm open to broadening the scope if there's interest. Just happy to have WebRTC in the running. 🙂

gsnedders commented 1 year ago

Any way to create a better link to the specific tests you are suggesting?

https://wpt.fyi/results/?label=master&label=experimental&aligned&q=%28path%3A%2Fwebrtc%20or%20path%3A%2Fwebrtc-extensions%29%20codec

nairnandu commented 9 months ago

Thank you for proposing WebRTC peer connections and codecs for inclusion in Interop 2024.

We wanted to let you know that this proposal was not selected to be part of Interop 2024. This is because we got many more proposals than we could include in this year's project. Note that individual vendors may nevertheless choose to advance work in this area during the forthcoming year. We would welcome this proposal being resubmitted again next year, if necessary.

For an overview of our process, see proposal selection. Thank you again for contributing to Interop 2024!

Posted on behalf of the Interop team.