Proper surround sound playback with video

[ThreeDeeJay]

Is your feature request related to a usage problem (not a bug)? Please describe. To my surprise, FreeTube supports 5.1 surround playback, but requires Settings > Default Video Format > Audio Formats so higher audio bitrates are selectable. However, this will freeze video playback since it's an audio-only mode.

Describe the solution you'd like to see implemented Allow selecting highest/surround bitrates during regular video playback

Describe alternatives you've considered Not really solutions but here are other methods of YouTube surround playback I've tried:

• Downloading video using JDownloader and playing it locally
• Kodi media center using YouTube addon

Screenshots https://i.imgur.com/r20Uqfb.mp4 Recorded using virtual surround so anyone can listen to the surround effect on regular stereo headphones.

Additional context On a side note, YouTube also supports spatial audio, which requires less channels (4) yet is capable of 3 dimensions (with height) thanks to the Ambisonics format, which is supported by OpenAL Soft, an API that can render it on headphones in case VR/360 support ever gets implemented and plugins like SoundField, which allow decoding it into virtual surround.

[ThreeDeeJay]

Here's the original 5.1 video I used to record the virtual surround clip, just in case: https://youtu.be/PqVCPE8_ntE

[GilgusMaximus]

It surely looks like a good idea, but with whatever the player is doing with qualities and stuff, my intuition will be that this will go very bad. So if someone attempts to add more complex quality selection to support this, please contact us first in the dev channel to clear things up or write a detailed pr

[mbirth]

This sounds a lot like it depends on https://github.com/FreeTubeApp/FreeTube/issues/1460 for video-vr , adding OmniTone and https://github.com/videojs/videojs-vr/issues/69

[ThreeDeeJay]

While I'm not particularly experienced in the code, I propose OpenAL Soft for spatial audio rendering. It already supports First-Order Ambisonics, which is the format used by YouTube. OpenAL Soft is also capable of rendering virtual surround, which allows hearing the surround effect on headphones like this, without needing a speaker setup or proprietary hardware/software.

[ThreeDeeJay]

Something I noticed the other day. Opus audio codec track is the default (and arguably highest quality) BUT it doesn't seem to be used for surround tracks, only some AAC, AC-3 and EC-3 have 6 channels (5.1 surround) So perhaps all we need is to be able to manually switch/select specific audio tracks (specifically one with 6 channels). Here are some confirmed 5.1 surround videos for testing: https://youtu.be/Yd7pZRhAcW8?list=PLVDbhbTh1d8fFGCQuz0hsIdV2HvnvgQR3

[github-actions[bot]]

This issue is stale because it has been open 28 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[ThreeDeeJay]

Forgot to mention, there are Opus tracks with 6 channels, but I've only seen them in VR/360 videos with spatial audio (4 channels for First Order Ambisonics + 2 channels for head-locked stereo) which could be used for surround but would require decoding ambisonics first.

[absidue]

Just leaving some comments here.

Using OpenAL Soft is not possible, as it's native code and FreeTube is written in JavaScript.

Also from my testing and the chromium docs, AC-3 and EC-3 aren't supported by Electron, so we can't play them in FreeTube (FreeTube uses Electron, which is a set of patches on top of Chromium).

[ThreeDeeJay]

@absidue Then in that case, it seems the best way to get surround is adding an option to select a specific audio track that does support surround sound like AAC. Because like I mentioned, FreeTube can already output surround sound, but only when setting Default Video Format to Audio Formats (just tested this video which I assume uses AAC in 0.19.1 beta), but then that plays audio only with a still thumbnail instead of video.