Jitsi browser audio issues for 2022-01-01 release

[msporny]

This issue is here to track Jitsi browser audio issues with the latest (2022-01-01) release of Jitsi for the CCG.

[msporny]

Reported by @kayaelle:

Sound issues keep creeping back. In. Safari seems to handle it the best for Mac users. One attendee tried:

• Chrome 97.0.4692.99 (Official Build) (64-bit) • Firefox 96.0.2 (64-bit) • MS Edge 97.0.1072.69 (Official build) (64-bit)

[msporny]

Reported by @kayaelle:

I don’t typically use safari but I’ve also experienced intermittent sound (hearing) issues on chrome & Firefox even when clearing session storage) and Safari seems to work fine. The sound issues reported to me had to do with hearing.

[msporny]

Reported by @vsnt:

I joined late today, and had some issues others have noted. Started out on Chrome, ended up listening via Safari. Adding comments to this thread. On my regular chrome - which I need to upgrade

• joined late, and didn't hear any sound
• exited and joined again, still didn't hear any sound

On Safari

• side chat does not scroll
• audio from Mike dropped several times, but I think others experienced this and it was a bandwidth issue <-- confirmed by multiple parties, seemed to be a bandwidth issue from Mike's computer

[TallTed]

Standalone Jitsi Meet app may be a more viable option than a tab in any browser. (I've never had an audio issue with it, only CPU spikes when someone is screen-sharing, which spikes I've also seen with Jitsi in a browser tab.) It does have its own issues, such as forcing app updates, but I haven't hit any as bad as the audio issues described above.

[davidlehn]

As a data point that I haven't fully researched, in recent Mac Chrome v97 when I do an initial meeting load I sometimes will hear jitsi beep boops, and see blue blobs of people talking, but not hear their audio. The console will have lots of errors like this or similar (and others I didn't capture):

Logger.js:154 2022-xx-xxTxx:xx:xx.xxxZ [features/base/media] Failed to play audio track! retry: 3 ; Error: NotAllowedError: play() failed because the user didn't interact with the document first. https://goo.gl/xX8pDD

Their shortlink goes here: https://developer.chrome.com/blog/autoplay/. That info is from some time ago so I'm guessing similar policies are being more strictly enforced in more recent browser releases since this wasn't a problem before.

While I haven't done proper experiments, I think this is in fact happening when I load a jitsi meeting URL without interacting with the page. Reloading doesn't help (or maybe does after a few times?). But if I do click in the window while jitsi loads it seems to address the interaction issue errors immediately. And that interaction seems be cached with some timeout of hours or days. So this can be difficult to test during regular use.

[msporny]

What follows are a few upgrades made to the Jitsi system that will hopefully fix some of the audio issues that some of us have been experiencing over the past two months.

Most of these problems have been caused by changes to how browsers manage audio permissions when playing back audio. Some of these problems are a result of new browser bugs introduced as a result of these changes. When and how web pages are allowed to call certain APIs will be an area of experimentation by the browser vendors over the next couple of years as they try to tighten down the privacy and security profile of web browsers.

Meeting Join Screen

A meeting join screen has been enabled where you can select your microphone, speakers, and camera before joining any CCG meeting. We had this disabled before because it was not necessary.

New upgrades to Chrome now require an individual to interact with a web page before that web page can play sound to that person. This was done to prevent ads from auto-playing audio commercials, but the side effect was that Jitsi is now blocked (in some cases, depending on your browser version and OS platform) from playing audio when you don't interact with the page first.

Placing this join screen before a call ensures that everyone will interact with the web page, thus enabling Jitsi to get access to the audio device and play back what other people are saying on the call.

If this still doesn't work for you, your audio drops out half-way through the call, or you talk but other people can't hear you -- just re-load the page and that should fix the audio for you in most of the cases. Unfortunately, since these are largely sporadic browser bugs, a Jitsi upgrade won't help us here... but a page refresh might fix the problem for you.

Transcription Upgrades

The auto-transcription feature is working much better now that we're using a more financially costly audio AI model. Google has a great business model here -- "Those are some pretty words you just said... it'd be a real shame if something were to happen to 'em, pal.") :P

Err, I mean, Google is wonderful and there is nothing wrong with charging good money for a service that provides great value.

The transcriptions seem to be good enough to replace human scribes at this point. The quality is not as good as a /good/ human scribe, and the bots capture every single word that is said (probably too much), but the days of human transcription seem to be numbered.

I've optimized the transcription bot so it stops recording every single utterance, so one and two word quips are not recorded now. That was most of the "clean up" work required for auto-transcribed minutes these days... which takes less time (at least for me) than dealing w/ a human scribe (the output from meeting to meeting is far more consistent now).

We still have a few issues with the system... like, the fact that Apple devices don't seem to care about the Jitsi server telling them to "PLEASE STOP FIRE-HOSING ME WITH SCREEN SHARE DATA!!!"... So, doing a dual-screen Apple 8K desktop screen share w/ the system still might bring it to its knees. When you screen share, please just share a reasonably sized HD-quality window instead of all two screens of your ultra-wide dual monitor 4K desktop setup at home. :P

We're regularly doing calls with 30 people on them now and the system load seems to be stable, even with screen sharing, screen recording, and auto-transcription enabled.

If you have additional concerns or problems with Jitsi, please do log the issues in here.

[kayaelle]

No browser audio issues today at VC-EDU.

We did lose recording while screen sharing. We restarted the recording but no transcription. Question: does the restart of the recording replace the original recording?

[msporny]

Question: does the restart of the recording replace the original recording?

On the server, it does not. The script that puts the recording in place will use the latest largest file from the day for the work item channel that exists.

We are at a point now where we could automate all of this, but time pressures are keeping me away from writing the (fairly simple) code to do that.

[msporny]

I'm going to close this issue as we've been operating for 9+ months now w/o major issues.

There are still the Apple macOS bugs that crop up from time to time, such as Safari WebRTC audio channel not categorizing itself correctly leading to the individual not being auto-transcribed, but it's clear that Apple isn't prioritizing fixing those bugs at this point.

I'm also hesitant to upgrade the infrastructure because it seems to be working well enough for most people and an upgrade might cause more chaos instead of fixing things.

Closing, folks should feel free to raise a new issue if they're regularly experiencing bugs with Jitsi that prevent them from participating in meetings.

[TallTed]

There are still the Apple macOS bugs that crop up from time to time, such as Safari WebRTC audio channel not categorizing itself correctly leading to the individual not being auto-transcribed, but it's clear that Apple isn't prioritizing fixing those bugs at this point.

I can't speak to anyone else's capabilities on this front, but for myself --

Past experience has shown that the more people report issue like this — through Apple Feedback for non-developers, through their Bug Reporter for developers — the higher the priority gets, especially if it's causing data loss, which I think would include failure to record Safari users when users of other browsers are recorded.

This is the first I've heard that there was a known cause — like "Safari WebRTC audio channel not categorizing itself correctly leading to the individual not being auto-transcribed" — which I would report if I had sufficient detail to do so.

Apple doesn't make it easy if at all possible for developers to see reports made by others — but if someone who's analyzed the issue sufficiently to know about the WebRTC categorization shared enough detail that others could (1) confirm that the same issue was happening to them and (2) log a sufficiently detailed report that Apple can act on it — this would help raise the priority of their resolution.

All of which is to say — if you're a macOS user and/or developer who has such info, please share the details!