Feature Idea: Sending a submix instead of input

[AtmanActive]

Hi Jesse,

I just discovered SonoBus and I am blown away. Congratulations on coding such a fantastic piece of software.

I'm into internet-band-distributed-studio stuff and am exploring all possible software/hardware combinations to record and mix with my friends all over the world. SonoBus fits right in!

Now, recently, as a band, we started using one DAW that is remote to all of us which sits on a powerful machine, is controlled via VNC and enables us to arrange and mix from any device and location. Let's call this DAW a CloudDAW in this discussion. We are relying on Audiomovers ListenTo to transport the audio for us.

The new idea that dawned on us the other day is how great it would be to be able to record musicians without rendering, transferring files and preparing anything. SonoBus is almost there with this idea, but not 100%, yet. So, the idea is that our CloudDAW sends both master mix and SMTPE to SonoBus. That works as expected right now. Then, all participants have that SMTPE track set to -inf volume so they don't really hear it, but, SonoBus is still able to record it. That works as expected right now. Then, we press play on the CloudDAW and the musician rec on the SonoBus and SonoBus records all participants, including the SMTPE track. In that way we can have a timestamp of where in the DAW that recorded track should be.

The only thing missing in SonoBus right now is the ability to send a submix to all participants, as opposed to input-only, as it is right now. So, let's imagine a checkbox next to each participant and when I click on that checkbox, then, network-wise, both my input AND that channel are sent. In that way, every other participant can hear CloudDAW+Singer in exact time Singer hears it. And then, all other participants just need to set the CloudDAW channel to -inf volume to avoid double sound.

In this way, with the Send-SubMix option, all band members could be scattered around the globe, be working on one and just one CloudDAW and record any of the band members (one by one only) in real-time, but still making sure everyone hears everything in perfect sync.

Please let me know if I should explain further.

Thanks.

Cheers!

Keep up the good work.

[strk]

Sounds like a way to make it easy to create feedback loops, doesn't it ? But I've felt the need to "forward audio" to another connected party as well, since I can connect to 2 people but they can't connect between them, so such a feature could allow me to send them not only my audio but ALSO the audio from the other party. For my case such "mix" would need to be per-user. Here's the scenario:

 Actors:
     - Alice
     - Bob
     - Charlie (me)

 Intended setup:
     - Alice only sends audio to Charlie
     - Alice only receives audio from Charlie
     - Bob only sends audio to Charlie
     - Bob only receives audio from Charlie
     - Charlie receives audio from both Alice and Bob
     - Charlie sends Bob a mix of Charlie + Alice sound
     - Charlie sends Alice a mix of Charlie + Bob sound

@AtmanActive would the above setup also work for your case ?

I think my case can be already implemented with current SonoBus by running 3 instances.

The SonoBus channel will have these users:

  - Alice
  - Bob
  - Charlie
  - CharlieMix4Alice
  - CharlieMix4Bob

Charlie will run 3 SonoBus instances:

  - Charlie: receives audio only from Alice and Bob, sends audio only to CharlieMix4Alice and CharlieMix4Bob
  - CharlieMix4Alice: receives audio only from Charlie and Bob, sends audio only to Alice
  - CharlieMix4Bob: receivs audio only from Charlie and Alice, sends audio only to Bob

[strk]

Sorry, I forgot to mention that in my solution I need to do audio routing externally to SonoBus because SonoBus will not allow the submix idea you depicted, @AtmanActive :)

But having an audio router externally (like JackAudio on Linux) should allow the aforementioned routing, whereas CharlieMix4Alice and CharlieMix4Bob would receive input from an externally defined mix, taking the audio from the main "Charlie" instance.

[AtmanActive]

@strk : your case looks one step more complex than mine. You need to be able to send TWO OR MORE submixes to two (or more) different participants. While for my remote-recording-with-cloudDAW concept I just need to be able to send a submix of input + ONE incoming channel to EVERYONE. So it's just a simple mix-in-this-selected-incoming-stream-with-my-input-when-sending functionality.

[AtmanActive]

I was also thinking about bridge-ing this with two instances and some routing between them, but, as the idea is that each musician would need to be able to do the same, then it gets too complex for me (fiddling with other people's computers and setups). Right now, we are recording by installing Reaper, TeamViewer, Mumble and Audiomovers on musician's computer, then using Reaper to record, Mumble for communication, Audiomovers for DAW monitoring and TeamViewer to remotely control the whole thing.

[AtmanActive]

While we're on the subject, it just dawned on me that one even easier way to have SonoBus record audio with a timecode would be this:

• SonoBus VST plug-in reads timecode from the DAW (or allows MIDI input for MTC)
• SonoBus sends this stream together with audio, inaudible, of course
• SonoBus creates broadcast WAV on record which has this timecode embedded in the file

Et voila!

Of course, the ability to send a submix is still essential as, without it, nobody but the singer can hear the singing in correct time. Simply because there are random/various latencies reaching each participant for DAW track and singer's track.

[AtmanActive]

Sounds like a way to make it easy to create feedback loops, doesn't it ?

It sure does. But, aren't feedback loops common concern in studios and stage? Hopefully, every musician is aware of that by now. Besides, in this remote-record-with-cloudDAW concept only ONE participant would use a submix and only while in a recording session. Let's imagine a situation where there are three participants: Singer, Friend and CloudDAW. Now, Singer enables SubMix, sending their own input+CloudDAW back. CloudDAW has it's own receiving from anyone disabled anyway. Now, even if Friend would now (erroneously) enable SubMix on their end, now sending their microphone+CloudDAW to everyone, nothing bad would happen, they would both just hear echoed CloudDAW.

[AtmanActive]

But I've felt the need to "forward audio" to another connected party as well, since I can connect to 2 people but they can't connect between them, so such a feature could allow me to send them not only my audio but ALSO the audio from the other party.

If this would happen to me I would look for a way to fix their modems/routers or introduce a VPN like ZeroTier One or something, but I would never route their audio through me just because they can't open ports properly.

[strk]

@strk : your case looks one step more complex than mine. You need to be able to send TWO OR MORE submixes to two (or more) different participants. While for my remote-recording-with-cloudDAW concept I just need to be able to send a submix of input + ONE incoming channel to EVERYONE. So it's just a simple mix-in-this-selected-incoming-stream-with-my-input-when-sending functionality.

To be honest I find your use case a bit overkill. You'd be using a lot of bandwidth to solve a problem which is not really a problem... You could have your base start with a count-in and use the count-in to align audio at the end, isn't that simpler and cheaper on network ?

[AtmanActive]

You could have your base start with a count-in and use the count-in to align audio at the end, isn't that simpler and cheaper on network ?

You're absolutely right. It is.

We could use a count-in on the main playback to synchronize, BUT, that would mean that each and every recorded piece would need to be manually added to the arrangement, with manually adjusting where it should sit in time. So we can't really review the recording until we do the manual insert-into-DAW operation. On the other hand, If we would have an SMTPE track recorded in parallel, or a broadcast WAV with timecode embedded, then, CloudDAW could import those recorded pieces AUTOMATICALLY and place them EXACTLY where they need to be. This can, then, be easily automated with the use of Syncthing and a few scripts. Hence, this would enable a frictionless, random recording from any point in time, without the need to render/import/prepare anything. It would work just as good as when one is recording with a local DAW. Only this time with remote recording and remote listeners and band members.

But, whichever method we use, be it manual or automatic inserting of recorded pieces back into the DAW, the latency problem still remains: on that recording session, all of the other band members can't hear proper singing while it is being recorded because for them, singer's voice and CloudDAW's playback come in different latencies as compared to how they happen at Singer's computer.

[AtmanActive]

I just took version 1.3.13 beta for a spin, and wow! What a progress. Incredible! Multichannel is already here! So, now, SonoBus is even closer to be able to work in this scenario. The only thing missing is the ability to choose one of the already incoming tracks as a source for one of my own multichannel inputs.

Why would I ever want to send an incoming track together with my microphone to everyone else? So that everyone else can hear my singing in the exact right timing, perfectly synchronized to that track.

And now, with this new multichannel capability, it would be even better than a submix simply because every participant could make their own mix of my voice versus the playback. Fantastic!

[essej]

Yes, I was about to ask/tell you to try the beta! The latest is currently at 1.3.18: https://www.dropbox.com/s/roeus0zh2wkj1qo/sonobus-1.3.18-mac.dmg?dl=1 https://www.dropbox.com/s/9srwgfvvpwqm53p/sonobus-1.3.18-win.exe?dl=1 NEW IOS BETA - REGISTER HERE (new link) - https://testflight.apple.com/join/o8Ju1jnm (and available to compile on the develop branch here)

You are right, just being able to select received user audio as an option to send out as an additional send group in multichannel would be the easiest way to accomplish this. Cool idea! This would also let you do some interesting chain playing/recording use cases. I will need to add a default muted state for a multichannel group, so that in a common scenario people wouldn't get crazy double-signal-at-different latencies heard by default unless they really intended to. So that submix send from someone would be muted by default on the receive side.

[essej]

What is your clouddaw, anyway? Just curious...

[AtmanActive]

Reaper.

[AtmanActive]

Just wanted to share my personal progress on this subject:

https://github.com/AtmanActive/ReaConnect