Nested Scene: Audio sources will double the volume of output

flaeri commented 2 years ago

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

27.1.3

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://gist.githubusercontent.com/flaeri/514baa3a403e488e25ab2f6a542ea3d6/raw/2af1c96866e53867fb4171e29d5dddef33b14baa/2022-01-22%252011-49-16.txt

OBS Studio Crash Log URL

No response

Expected Behavior

When having an audio source in a scene, and that scene is nested into a different scene, which is also containing the same ref to an audio source, I would expect the volume in the output to remain the same as if there was only a single ref to the audio source.

Current Behavior

When having a scene with an audio device, and a nested scene containing the same audio source (as a reference), then the audio in output will double in volume.

Scene: mic
  source: mic1
scene: nestMic
  Source: mic1 (ref)
  Source: Mic (nested scene "mic")

Steps to Reproduce

Start with a clean scene collection/profile, so that you dont have any audio source (global or local).
Create a scene called "audio1", and add an "audio output capture", pointed at your desktop audio
Create a new scene called "main", and add the same source as above (as ref), and add a nested scene pointing to "audio1".
Use a tonegenerator as the only audio on that device (for instance: https://www.szynalski.com/tone-generator/) to produce a consistent audio level output
Start recording. Wait 5 sec, and enable/disable the nested scene
Inspect the output recording for volume changes when higing/showing the nested scene (eye)

Anything else we should know?

Example output. The jump is when enabling/showing the nested scene:

Example file demonstrating the volume change: https://user-images.githubusercontent.com/50419942/150635923-74e69184-1c59-46a9-9262-dfc47686d8a2.mp4

Happens on latest beta as well. I figured I would do a writeup for this, as I'm not confident that this is the intended behavior, and may have heard thru the grapevine that this should already be accounted for.

Happens on the following: 25.0.8, 23.2.1, 22.0.2

jp9000 commented 2 years ago

Audio mixing is hierarchical, so this is sort of intended, each source will output audio to its parent. For example, if you duplicate a reference to a source multiple times in the same scene, that will also double the audio.

WizardCM commented 2 years ago

Sure, but if we want it to be officially intentional then it should be accurately reflected in the audio mixer.

jp9000 commented 2 years ago

Either case, this isn't really a bug. This is pretty much just how the system works internally.

mufunyo commented 2 years ago

Issues don't have to be bugs though. In this case there's an intended internal behaviour that isn't clearly shown to the user in the audio meters.

WizardCM commented 2 years ago

After some digging, it looks like the UI displays it this way because it uses the source_activate signal to check & add a source to the Audio Mixer. Obviously, this signal only runs once per source, so they're only added once to the mixer.

This means that the Audio Mixer is actually tied to sources not scene items, which is also why it (correctly) lists items from any nested scene items. Neat side effect. Also means that if you toggle visibility of either scene item for the same source (even a nested scene), audio monitoring works uninterrupted (and de-duplicated) while stream audio gets doubled when both are enabled.

I'm unsure about the correct fix for this - I think it'd be logical to show either

one entry for each active scene item (including from nested scenes) or
just a ⚠ indicator in the mixer that has a tooltip for sources that appear more than once, something like "This source is loaded x times in the current scene, which can result in boosted audio"

In summary:

Output audio is per-scene-item (including nested)
Monitoring audio is per-source
Audio Mixer displays per-source

gxalpha commented 2 years ago

I think idea 2 would be better here, I see some problems with 1:

Having a source multiple times could suggest you can set different volumes per scene item, which as far as I can tell isn't possible (the volume is tied to the source, not the item).
The audio mixer tends to eat more performance than it should, and adding more entries could worsen that.

Warchamp7 commented 2 years ago

How difficult would it be for the audio to only come through once as long as 1 or more copies of the source are visible?

kamalmostafa commented 2 years ago

In my opinion, if this isn't a "bug", then it is fundamentally the wrong design choice.

Doubling the audio volume for multiple copies of a source ... (a) is not the behavior I expected. (b) makes my use case impossible to achieve. (c) forces a pretty weird audio behavior that I cannot imagine any use case actually would want. (d) does the exact opposite of what the mixer UI suggests it does.

I propose revisiting the design decision to double (or triple, etc.) the audio in this situation -- or really any situation! Is there actually any use case which would ever want doubled-audio when a source appears multiple times in a scene? If not, just change the behavior!... No UI change needed at all. Are we really expecting a deluge of bug reports saying "I was actually relying on having amplified audio in this case" ? :)

My particular use case: I've got multiple rtsp/rtmp cameras, only one of which supplies audio. I want (simply) to always hear the audio from that one "main" camera's source (as if it were a "global audio source" but there's no way to make it one within OBS?). Sometimes that main camera's full-screen view scene is actually active (so I get its audio). Sometimes a different camera's scene is active (I do not get main's audio; but I can work around that by placing an off-screen copy of the main view in that scene -- annoying, but workable). But sometimes a crop of the main camera is visible in a PiP window (and now I'll get double-main-audio, given the necessity of the off-screen copy workaround). Sometimes I might even put a zoom-crop of the main cam in the main view's PiP window (so I will I get double-audio no matter what; no workaround). None of this is sensible.

In lieu of just changing the behavior ... My use case would alternately be solvable if the Source's "visibility" (the eyeball icon) was decoupled from the Sources "audio enabled" status. I.e. Another click-icon, adjacent to the visibility-eyeball which is a tri-state audio-enable selector: auto / enabled / muted (where "auto" means "tied to visibility", like the current behavior). That would at least be simpler than multiple separate-volume instances of a sources appearing in the mixer, and would supply a range of behaviors instead of forcing this one (pretty weird) behavior.

paulh-aja commented 2 years ago

I'm not sure if this is the same issue but here is the workflow I was doing when running into what seemed to be the same problem:

Add a video clip source
Copy the video clip source and Paste as Reference
Crop out an isolated area of the pasted reference source.

The idea was that I wanted to play a video clip, and crop and zoom in on an isolated area of that source, showing it as an overlay over the original. Here's an example of what I was doing with the video:

However I only wanted to hear the audio from one of these sources at a time. It seemed odd to me that it was possible to toggle the visibility of either the original video or the pasted reference source, but not mute the audio of either the original or the pasted reference.

RytoEX commented 2 years ago

Duplicate of #2871

datamattsson commented 6 months ago

It doesn't matter if the scene is nested or not. Multiple crops of the same AV device in the same scene increases the volume. #10325

obsproject / obs-studio