[WebRTC] self and peer voice visualizer levels do not match.

[canny[bot]]

When in a webrtc voice region, the voice visualizers (green waves) do not appear when your own avatar speaks. The voice visualizers work properly when other avatars speak. This issue does not reproduce when in a Vivox region.

1. Login with 2 webrtc voice viewers to secondlife://Aditi/secondlife/webRTC1/128/128/23 on the WebRTC viewer
2. UserA: Observe that the white voice dot is visible above your avatar's head (as expected)
3. UserA: Click the "Speak" button in the toolbar to start transmitting voice and speak
4. UserA: Observe that when you are speaking, the voice visualizers (green waves) are not visible above your avatars head or in the Nearby Chat speakers list
5. UserB: Observe that UserA's voice visualizers appear as expected.

[canny[bot]]

This issue has been linked to a Canny post: [WebRTC] Voice visualizers are not visible :tada:

[maestrolinden]

I can reproduce this on the latest webrtc voice build, 'Second Life Test 7.1.4.8414833386 (64bit)', on both Mac and Win10. The current server build I reproduce it on is WebRTC-Voice 2024-03-25.8414560025.

[maestrolinden]

I can't reproduce this issue on MacOS or Win10 with WebRTC viewer 7.1.4.8610505473 (64bit). Changing the volume in either a webRTC version (WebRTC-Voice 2024-04-09.8610484336) or a Vivox region (Second Life Server 2024-03-18.8333615376). In all cases, I see the visualizers with appropriate-looking volume levels when either my own agent speaks or another agent speaks. Likewise, I don't see visualizers on agents who are not speaking. Passes QA

[roxanneskelly]

Repro:

• Log in to region with WebRTC voice outside of a parcel that has parcel-only voice.
• Unmute (PTT down)
• Walk across parcel boundary.
• continue talking.
• No voice bars...

[roxanneskelly]

This is a race condition in webrtc related to how we're using webrtc.

We use a custom audio processor to grab the audio data and determine audio level. This is done because the custom processor runs after the automatic gain control, echo cancellation, noise reduction, etc. and we want bars to reflect what other users are hearing.

Along with that, we bring up and shut down webrtc audio connections in an overlapped manner, where one may be in procesws of coming up at the same time where the other one is shutting down. These connections use the same audio processing module, which calls our audio processor.

When shutting down a connection, a connection shuts down custom audio processing (and other processing) if all of its streams are muted. If the new connection has already unmuted to talk when it's coming up, custom processing will get shut down, hence voice bars go away. Other audio processing proceeds, hence audio is available.

The solutions here are:

1. Wait for the first connection to shut down before the other one comes up. This won't work as we bring up multiple connections for cross-region voice.
2. Use multiple audio processing modules, one for each connection. This requires a unique audio connection factory for each connection, which gets cumbersome and may not work.
3. Build a patch for WebRTC that does the 'right' thing and doesn't disable custom audio processing until all streams on all connections are muted.
4. Disabled the 'disabling' of custom audio processing with a WebRTC patch. (hacky)

The plan of action is to explore 3 for a bit, and do that unless it's too complicated. Otherwise do 4.

[roxanneskelly]

Chose option 4 (disable disabling of custom audio processing) as option 3 (fixing it for real) was a fair bit more extensive and for patches to webrtc, surgical fixes are preferred.

[DmitrySProductengine]

Failed QA. Verified on the Second Life Release 7.1.9.9604591078 (64bit) on Win10/Win11/OSX in the scope of IQA #264. Tested on Agni: Region: WebRTC Voice 1 Sim: Pop Rocks 2024-05-08.9009748043 Voice: Secondlife WebRTC Gateway

Tested on Aditi: Region: webRTC1 Sim: WebRTC-Voice 2024-06-12.9490472357 Voice: Secondlife WebRTC Gateway

This behavior is partially reproducible. In my opinion, the current reproducing is related to the microphone volume level and its sensitivity.

The behavior was reproduced on the built-in microphone on a Mac and an external microphone on a laptop with Win10/11.

To test, I used an online radio at medium volume or a conversation with a medium voice. If the microphone volume level on a Mac was at the default level (in the middle), then the user on a Mac (user A from the issue steps) did not see his own conversation indication. At the same time, the user on Win10/11 saw the indication and heard a voice (radio or voice). If you raise your voice during a conversation, your own indication also appears.

Workaround: slightly increase the microphone volume in the viewer settings.

Also, if this behavior is not reproducible at the default settings, you can slightly reduce the microphone volume. With a high probability it will start reproducing (User A will not see an indication of his conversation, but User B will).

This behavior is not reproduced in regions with Vivox.

Perhaps this is a new expected behavior (for the user to increase the microphone volume themselves), or perhaps it is possible to increase the microphone volume by default slightly. Technically, there is an indication of your own conversation, but due to microphone volume settings, it is not always displayed correctly.

See attached screenshots:

1. User A - OSX - voice visualizers (green waves) are not visible

   

2. User B - Win10 - voice visualizers appear as expected

   

3. User A - default mic volume

   

4. User A - increase mic volume

   

[DmitrySProductengine]

Additional info: The behavior of changing the microphone volume level looks as if, apart from the visual display, the user's volume does not change for the users who are listening to it (at least in Nearby chat). Changing the minimum and maximum values ​​does not change the user's volume. Unfortunately, I didn’t have time to fully investigate this behavior, but perhaps it should be filed as a new bug.

[maestrolinden]

Additional info: The behavior of changing the microphone volume level looks as if, apart from the visual display, the user's volume does not change for the users who are listening to it (at least in Nearby chat). Changing the minimum and maximum values ​​does not change the user's volume. Unfortunately, I didn’t have time to fully investigate this behavior, but perhaps it should be filed as a new bug.

@DmitrySProductengine I just tested the effect of the "Mic Volume" slider in 7.1.9.9604591078 on Mac and Windows, but I'm unable to reproduce your bug. My test setup was:

• Both viewers running 7.1.9.9604591078 on SLS 2024-06-11.9458617693 (Vivox mode simulator)
  • One viewer is Win10 22H2, the other viewer on Mac OS X 14.5.0
• Both viewers are in same region, with:
  • Voice operating in nearby chat mode (not on a p2p or group call)
  • "Hear Sound and Media From: Camera Position"
  • Each viewer's camera facing the other viewer's avatar, with approximately 5m distance between the camera and the other avatar. Both avatars and cameras were stationary during my test.
• For each viewer, I went to the "Mic Volume" slider and tried 0%, 25%, 50%, 75%, and 100% slider values
  • Each time the slider was moved, the loudness meter to the right of the "Mic Volume" slider appeared to increase/decrease properly as I spoke into the mic. At 0%, the first box of the loudness meter barely registered, and at 100%, the loudness meter would go into the red
  • After applying the updated "Mic Volume" setting, the viewer would reconnect to nearby chat, and the other viewer would hear the updated loudness. At 75% and 100% "Mic Volume" settings, the audio was noticeably clipped and distorted.

If you're still seeing a failure with "Mic Volume", could you please file a bug with test setup details?

EDIT: I tried again on WebRTC-Voice 2024-06-12.9490472357 (webRTC mode), and I can confirm the bug is as you describe in that build. While adjusting "Mic Volume" changes the loudness meter and also the speaker's own voice indicator show that their voice is loud (including red lines when volume is 100%), any listening avatars see no real effect of this setting - the voice indicator and audible voice stream don't seem to change at all. While "Automatic Gain Control" is enabled by default and might affect this behavior, disabling it on both viewers does not change my results.

I filed #1828 about this bug.

[roxanneskelly]

adjusted the sensitivity and fixed #1828

[DmitrySProductengine]

@maestrolinden - thank you for investigating and filing the bug. While investigating, I observed a new feature "Automatic Gain Control". Having found information about it, when it was turned on, such a result could well have been expected (automatic voice normalization). But even when this feature was disabled, changing the microphone level did not lead to anything. It was already the end of the working day for us and I decided not to fill the incompletely investigated behavior - but to draw attention to it with a comment.

[roxanneskelly]

The AGC issue has been fixed as well. I will note that many times the system audio driver config (for windows, mac, etc.) has some processing that happens as well. Now, when you turn that off, there may be a difference.

[igorlproductengine]

Failed QA Verified on Second Life Release 7.1.9.9688089989 (64bit) in the scope of https://github.com/secondlife/iqa/issues/270 on Win10/OSX systems.

Test Environment: Grid: Aditi WebRTC Region: WebRTC1 Simulator: WebRTC-Voice 2024-06-24.9652733849 Voice Server Version: Secondlife WebRTC Gateway

Steps to Reproduce:

1. Log in to the WebRTC region with two WebRTC voice viewers at secondlife://Aditi/secondlife/webRTC1/128/128/23.
2. User A: Ensure the white voice dot is visible above the avatar's head (as expected).
3. Set the mic volume to 100% in the Sound & Media tab in the preferences floater.
4. Click the "Speak" button in the toolbar to start transmitting voice and speak.
5. User B: Observe the voice activity indicator for User A.

Observed Behavior:

When User A speaks, they see a red voice activity indicator above their avatar's head. User B only sees a green voice activity indicator for User A.

Expected Behavior:

The red and green voice activity indicators should be displayed correctly for all users, indicating User A's voice activity.

Additional Observations:

This issue only reproduces in WebRTC regions. In Vivox regions, the voice activity indicators work as expected.

[DmitrySProductengine]

Investigated on the Second Life Release 7.1.9.10495892574 (64bit) on Win11/OSX in the scope of IQA #312. Current behavior on WebRTC regions has changed. With all default settings, the voice visual levels are off the charts (constantly turns red during normal conversation). Tested on Aditi:

1. WebRTC:
   secondlife://Aditi/secondlife/webRTC2/128/128/23 Sim: WebRTC-Voice 2024-08-15.10397494850 Voice: Secondlife WebRTC Gateway

See attached screenshots:

1. Actual - 7.1.9.10495892574 - OSX - WebRTC

   

2. Actual - 7.1.9.10495892574 - Win11 - WebRTC

This issue is not reproduced on Vivox regions (the voice visual levels are correct).