search

w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
955 stars 135 forks source link

Encoder fails if a MS Teams videocall is open #422

Closed mcarrozzino closed 2 years ago

mcarrozzino commented 2 years ago

I'm experimenting problems with WebCodecs encoders when a videocall on MSTeams is active. Apparently (but I am not completely sure) the problem seems to be caused by the fact that, being the camera taken by MSTeams, the encoder in my webpage gets no image and goes into the "closed" status, and from that point on it fails encoding. However, in this case also other encoders in my application, that do not rely on webcam images but on other video streams, fail: basically all instances of the encoder do not work.

[EDIT: this happens also with SKYPE, but in both cases it is not related to the camera being "taken", rather to the fact that Skype/Teams are actually encoding/streaming. As a matter of fact, if I launch a video call without the other peer answering, but with the camera taken by Skype/Teams, my web page encoders still work. If instead the Skype/Teams video call is active and I launch my webpage, all the encoders fail. It's like Teams/Skype get the ownership of some resources that cause my encoders to fail.]

1) Why does this happen? 2) Is there a way to avoid it? 3) In general, when an encoder enters the "closed" status, is it possible to "reopen" it ? 4) How is it possible that, when an instance of the encoder fails, all the other instances fail as well? 5) I get no error message during init, the only error messages I get is for encoding on a closed codec: is there a way to detect the impossibility to use the encoder before entering in the actual encoding loop?

sandersdan commented 2 years ago

If you are reporting a bug in the Chrome implementation of WebCodecs, please file a bug at https://bugs.chromium.org/p/chromium/issues/entry.

In general there isn't enough information here to diagnose the problem. Are these other applications browser-based, and if not, what OS are you running? This can affect the number of simultaneous consumers a camera can support.

In WebCodecs, and encoder cannot transition from configured to closed without either explicitly calling close() or emitting an error via the error callback. Assuming there was an error, what is the error message?

A closed codec cannot be re-opened. It must be recreated.

It does not make sense that all encoders would fail together, unless they are all backed by a faulty hardware implementation. It is possible that they are being shut down due to inactivity at around the same time, though. In either case there would be an error callback with a message.

There is no way to establish with certainty that a codec is in an operable state. However, if you are getting an exception related to being in a closed state, it means that you ignored an error callback.

mcarrozzino commented 2 years ago

Thank you a lot @sandersdan.

In general there isn't enough information here to diagnose the problem. I'll try to add more context. First of all, this happens on Windows 10, on both Chrome v95 and Opera v82. This behaviour takes place when there's an active video call (using the camera) on Teams Desktop App or Skype Desktop App. It doesn't happen if there is an active video call on Zoom, and, in general, it doesn't seem to happen when there is a video call running on a web application inside a browser. So apparently the problem is not using the camera but, rather, encoding/streaming the camera...

Assuming there was an error, what is the error message? You are right: I did not notice the error returned by the init callback. Unfortunately it's not very descriptive: the callback returns: "Encoding error" after which I get a bunch of errors such as: Uncaught DOMException: Failed to execute 'encode' on 'VideoEncoder': Cannot call 'encode' on a closed codec.

It does not make sense that all encoders would fail together You are probably right from the perspective of timing. However, what happens is that, if I am in one of the bad conditions above described (i.e. active video call on a desktop application like Teams/Skype) all the encoders I am using fail (not only the one encoding the camera, but also another one encoding another video stream and also another one which I use for screen sharing). So probably the do not fail together, but they do fail ALL. If instead I'm not in a "bad condition" they all work properly.

Hope this additional info can help in finding a solution.

mcarrozzino commented 2 years ago

Hello @sandersdan, any hint on this issue?

chcunningham commented 2 years ago

Hi @mcarrozzino, unfortunately most folks are OOO until early-mid January for the holidays.

This issue sounds like a Chromium bug, so I've filed that here: https://bugs.chromium.org/p/chromium/issues/detail?id=1281018 - lets move the discussion there. Could you add a small sample script to that bug for us to reproduce the issue?