FrankerFaceZ Audio Compressor Interferes with Live Caption

[xuchkang171]

When using Live Caption in Chrome with the FrankerFaceZ audio compressor feature enabled, the captions become unreliable and flicker. This manifests as:

• Repeated Switching: The displayed text of Live Caption rapidly changes between two slightly different versions. This may include:
  • Variations in starting points: One version starting a word or a few words earlier than the other.
  • Differences in punctuation
• Overall Instability: The flickering behavior makes it difficult to follow the captions.

Reproduction Steps

1. Start a Twitch stream or video with Live Caption enabled.
2. Enable the FrankerFaceZ audio compressor feature.
3. Observe the flickering and inconsistent captions.
4. Disable the FrankerFaceZ audio compressor and refresh the page.
5. Verify that Live Caption is now working correctly without flickering.

Possible Cause

It is suspected that the FrankerFaceZ audio compressor might be interfering with the audio processing used by Live Caption. One possible explanation is that the extension might be creating an additional audio track that is playing simultaneously, causing confusion for Live Caption.

Additional Information

• Operating System: Windows 10
• Google Chrome Version: 128.0.6613.138 (Official Build) (64-bit)
• FrankerFaceZ Chrome extension Version: 1.57
  • Player » Compressor: Default settings.

Expected Behavior

Live Caption should display accurate and stable captions regardless of FrankerFaceZ features being used.

[SirStendec]

Duplicate of https://github.com/FrankerFaceZ/FrankerFaceZ/issues/1524

Pasted response:

This is a bug with Chrome, not with FrankerFaceZ. We are using the web audio API in a normal way, so if the browser can't handle that correctly, there's nothing for us to do about it. Just don't use the feature if you need live captions to work.

As for needing to reset the player before it starts working again, that has to do with a limitation of the web audio API. Once we have switched the audio pipeline from the standard