EnvelopSound / EnvelopForLive

Free, open-source tools for Ambisonic 3D panning within Max for Live 10+
http://www.envelop.us/
GNU General Public License v2.0
479 stars 59 forks source link

Signal degradation #11

Closed roddylindsay closed 7 years ago

roddylindsay commented 8 years ago

From @cameronangeli

No specific bugs, but we're having a degradation of signal. It's either a clocking issue or the encoder can't handle the amount of info we're sending and translating to our speaker arrangements. I'm leaning towards an encoder issue, because we're seeing it regardless of whether we are on one or two machines.

Here's the arrangement we used:

1) Performance Machine (Macbook Pro) running Ableton Live 9 64 bit: We are using all 16 envelop audio sources: 1-4 = 2 stereo pairs that are used as the core, 4 corner virtual anchors placed at 0 elevation in the corners. Drums and bass piped to these anchors so you hear them from all ground level speakers; 5-8 = 2 stereo pairs that are used for send effects piped into static anchors placed in the roof in a square - Reaktor effects like a 4 ch rotate delay, or 8 ch random delay tap, or front to back reverb; 9-14 = 3 stereo pairs for individual stereo sounds that can be panned; 15 & 16 = mono sounds that can be panned.

This machine is networked using ethernet on a static IP address (192.168.50.12 - or .13 or .14 depending on who is performing) to the encoder machine, which also has a static IP address (192.168.50.20).

All envelop M4L plugins are set to connect to the IP address: 192.168.50.20.

We are using Dante Virtual Sound Card 32x32 48K to send audio from ableton digitally over ethernet. So the UDP messages, and 16 channels of audio are sent from the performance machine, to the:

2) Encoding Machine (mac mini 2016 8GB Ram). We are also running Dante Virtual Sound Card, and Dante Controller on this machine. The channels from the performance machine are mapped to the input channels of the encoding machine. In order for the signal to work properly, we also had to initially use Dante Via to identify audio sources at Dante capable.

Max 7 is running on this machine, which is running our adapted version of Envelop. We have changed the IP address for the client and server to be 192.168.50.20. We have created 11 speaker presets - which includes the Savage arena arrangement.

That arrangement is 16 speakers = 4 JBL powered subs at 0 degrees in the corners, 4 3-way Mackie stacks at 2 degrees in the corners, 4 JBL eon 515XTs at about 10 degrees in the corners, and 4 JBL eon 515xts that are mounted at about 50 degrees, rotated 45 degrees from the corners to fill the gaps and add height.

The Mac Mini is connected to a PreSonus 16ch audio interface. This only has 8 analog outs, so we have expanded using ADAT to a behringer 8+8 unit, giving us the other 8 outs.

When it's working, the panning sounds amazing. however, at mostly random times, the sounds will chunk up. It seems like there is channel bleed, and effects propagating through the glitch. On my laptop, the degradation is not too terrible, and lasts for a few seconds when it happens, then fixes itself. But on my buddy's machine, it will progressively get worse to the point where audio stops. We will restart Max on the encoding machine, and all is solved. It also appears that the more we throw at the encoder, the more we notice this random degradation. for example, using the send effects, which will add audio to 4 channels all at once will instigate the issue.

the encoder machine is running at about 15-20% DSP. and the laptops ableton CPU usage never goes above 15%.

We have added a few other things to the Server to help us manage the setup: 1) we added a master volume and meter that allows us to control the volume of all 16 tracks before they enter the encoder. 2) We added sub groups to control the volume of each of our speaker types (like a fader for the subs, fader for the 3-ways, fader for the low eons, and fader for the roof eons) 3) in order to deal with the speaker placement distance discrepancy, we added individual delays to each speaker. We never used this because it sounded fine, but technically, we should have delayed the eons on the room, since they were closer to the DJ than the corner speakers. 4) we disabled the built in reverb because it uses up a bunch of the DSP, and really doesn't add too much to the experience. We prefer to have the effects routed as we did with Reaktor.

We also have two other small scale setups - one in my studio with 8 speakers in a cub arrangement, and one in my buddy's studio with 8 speakers, 4 in the corners and 4 up high rotated 45 degrees. The planning seems to translate very well between all three setups and the binaural option.

There are a few tests that we need to run to further identify why we might be having this issue: 1) we need to do a full analog routing (no dante). based on the gear I have, I could test up to 16 channels out of ableton, to an 8 channel speaker arrangement in my studio. to try the 16 channel setup would require me to find a big enough space. 2) i'd like to get an external clock to sync everything to.

I know this is a lot, and i assume we have taken the envelop software to places that you did not intend, but i thought you could use the feedback to help make this next iteration as good as can be.

ramagottfried commented 7 years ago

My sense is that this is probably either related to the Dante setup or that your Mac mini is not powerful enough to handle the processing. If you can reproduce the problem without the Dante, definitely let us know and we can take a closer look.

The more channels you have the higher the CPU usage will be, so you'll always need to benchmark your patches/live sessions to make sure you're computer can handle it. Remember that you should always test your system well in advance of a show with the actual hardware you are going to use!

This system should be quite capable of being expanded to much higher channel counts. I used something similar with a 75+ speaker MADI/Ethersound based HOA system at Ircam with no problems. However, I was using a quad-core MacPro tower (circa 2012).

The new version with the fx rack will probably end up being more expensive CPU-wise as people start adding more spatial effects, so this issue of managing computer power will be one to keep an eye on.

ramagottfried commented 7 years ago

... although, rereading if your CPU wasn't over 20% on the server then probably it's not the Mac Mini, you can usually run Max up to 70% or so before it starts clicking. So, I do think it could be something like clock drift, but really hard to say without seeing the system.

It's also possible that many rapid panning messages could cause the CPU to spike which might not be visible from the client side monitor if it's a fast spike. But usually if a problem is caused by CPU spikes, it would go away when then level returns to normal... anyway, do let us know if you can isolate it.

cameronangeli commented 7 years ago

Hey Guys,

I finally had a chance to get back to some testing. In an attempt to isolate the issue, I have moved back to my Mac Pro (mid 2010), 2 x 2.4 GHz Intel, 16GB machine.

Test 1: Macbook pro (running the ableton set) & Mac Pro (running Max w/ Envelop Server and Client). 16 channels of audio routed over Dante virtual sound card. DVS as the input device on the Client, and Mackie Onyx as the output device. 8 speaker arrangement (4 speakers at 0 degrees elevation, and 4 speakers at 50 degrees elevation, rotated 45 degrees azimuth to make a cube that's been twisted). I have put automation on all envelop tracks - 7 stereo & 2 mono - so that every sound is moving at random around the room. CPU usage of ableton is at 6-10% on the macbook pro. DSP on the the server (running on the mac pro) is usually around 17-22%. In six separate sessions lasting 10 minutes, there was at least one degradation event lasting 2-5 seconds in duration. When the degradation event is not happening, there is no noticeable issues with the performance. all sounds appear to be mapping perfectly.

Test 2: Mac Pro running the ableton set and Envelop Max Server & Client. 16 channels of audio routed from ableton to client through Loopback. Output device is Mackie Onyx. 8 speaker arrangement, same as above. DSP on the client jumps to 30% then drops down to 20ish%. CPU usage on ableton is 27-30%. With all 16 channels being panned, the client gets overloaded, and the HOA sound field, Object amplitude, and Object locations visuals begin to lose accuracy. If i cut back on the number of sounds that are being panned, the accuracy returns. In some cases, the DSP meter actually freezes until i back off of the panning. This does not affect the sound quality. But like in test 1, there will still be a degradation within about 10 minutes of continuous use.

Test 3: with the same arrangement as test 2, I bypassed the encoder by hardwiring the max server patcher so that the 16 channels coming from the adc~ are routed directly to the dac~. since there are only 8 direct outs on this system, each pair of channels in, is mapped to one speaker out. Obviously, there won't be any panning information implemented on the sounds. But what this did confirm, is that there is no degradation after over 20 minutes of continuous playback. I then proceed to move the wiring through each step of the server workflow: 1) after the first "p meter" 2) after the "poly~ env.16..." 3) after the second "p meter" 4) after the "hoa.3d.optim~". The degradation did not return - even with all the UDP panning data still flowing - until the connections that came from the "hoa.3d.optim~".

I then tried to switch from "maxRe" to "basic" mode, and thought i might be getting an improvement, but that kept freezing the client/server visuals. so it was to get a reasonable comparison of the two modes.

So the degradation seems to be introduced in the hoa stages. If you're interested, I will try to get a recording of the degradation. Are you planning to stick with the hoa encoder? or will the next version switch to a different toolset?

thanks!

Best,

Cameron

ramagottfried commented 7 years ago

Excellent testing, thanks Cameron! Can you send me a patch that I can use to reproduce the problem on my system?

If if is the hoa.3d.optim~ object, can you try removing that, and send the audio directly out to the decoder and see if the problem goes away?

If the sound is okay before the optim~ that means that the encoding is okay, but maybe there's a bug in the HOA optimization (which would be really too bad). If we can isolate it for sure, then we can ask the developer to fix this. The new version is still using the hoalib.

Thanks for your work on this! It's really the only way to iron out bugs like this.

cameronangeli commented 7 years ago

sure. I just e-mailed a link to a zip of the project to the address on your website. I will run some more tests to remove the optimization as well.

ramagottfried commented 7 years ago

thanks cameron -- yes, best to keep simplifying the patch until it's very clear what's causing the problem and then we can send it to the hoa-lib devs.

I'll put a test patch together to start with and post it here in a second.

ramagottfried commented 7 years ago

actually a good test to start would be to use the hoa.3d.optim~ help patch for a while and see if that starts degrading. I'm running a sine wave through it right now, will come back and check it in 15-20

ramagottfried commented 7 years ago

so far no degradation that I can hear testing the hoa.3d.optim~ help patch.

There was some distortion with the sine wave at first, but after testing the direct signal I found that the sine wave was distorting by itself, after reducing the gain on the signal the distortion goes away.

I'll let it run for a while and see what happens. Let me know if you're able to isolate the issue further in a repeatable way.

cameronangeli commented 7 years ago

when i bypassed the optimizer, it seems like the degradation is reduced by a little more than half. I ran for an hour, and had three points where degradation occurred. the instances were smaller in comparison as well. I was able to get a recording of one of them with my iphone. It's right at the beginning of the clip for about 3 or 4 seconds: https://www.dropbox.com/s/4p7peodr8tfrn6r/Envelop%20Degradation.m4a?dl=0

I also tried to overdrive the signals on each channel to get the HOA sound field to be clipping the red - no noticeable impact/connection to the degradation.

for reference of my setup, i've recorded a video: https://www.dropbox.com/s/t41d8j7kt388bzc/File%20Oct%2009%2C%2012%2001%2047%20PM.mov?dl=0

there is no degradation in this video, but what it illustrates, is that the meter that I've placed on the server patcher is not affected by too much panning, while the meters on the client patcher seem to freeze if your moving more than 8 channels at a time. My guess is because we're sending too much UDP data back and forth. Moving the meters to the server patcher, without the need for UDP sends, might reduce the traffic and keep the UI working better. Not sure if this has an impact on the degradation though.

ramagottfried commented 7 years ago

Thanks, that's super helpful to hear what it sounds like. To my ears it sounds like it might be a vector/buffer size issue. Can you try increasing your buffer size to 2048 in Jack and see if that fixes it? (you'll have to quit max and ableton before you change that)

Re: GUI speed -- I'm not super concerned with the client meters, or speed of the graphic display, the most important thing is that the sound is correct. There is a lot of data flying around as you said, so the graphics are being slowed down on purpose to avoid slowing down the data getting to the DSP part of the patch.

ramagottfried commented 7 years ago

Sorry, I forgot you are running the server on a separate computer, in that case please try increasing the vector size and IO buffer on the server to 1024 and see if that helps. If not, then try increasing the vector size in Ableton also.

And just in case it's not already, I suggest using overdrive on all instances of max -- the GUI on the client is referring to the server, but the client itself should also be in overdrive mode.

cameronangeli commented 7 years ago

Well!!! it appears that the buffer rate is the solution for one computer. I put the ableton buffer to 2048, and both client (in/out) buffers to 2048, and i played for 30 minutes, no issues. This would also explain why it was different for various machines.

I will give it a try with the two machine setup when i get a chance. glad i got that recording to you!!!

ramagottfried commented 7 years ago

great!! very happy to hear that! maybe try reducing it to 1024 or 512 and see if one of those might work also, what was it set to before?

cameronangeli commented 7 years ago

Ableton was at 512 before. Client was at 1024. Was already in overdrive.