musikinformatik / SuperDirt

Tidal Audio Engine
GNU General Public License v2.0
518 stars 75 forks source link

let's sort out the panning scheme #36

Open kindohm opened 7 years ago

kindohm commented 7 years ago

There "stereo image" of a sample is lost when played in SuperDirt. For example, a sample that inherently contains right and left panning sounds like a "mono" sample without any panning in SuperDirt. Example:

https://soundcloud.com/kindohm/stereo-image-difference/s-Oh8f6

The above is a recording of the sample played in SuperDirt, then in Classic Dirt. The sample has a panning oscillation that should be audible in any audio player. In SuperDirt, it sounds like a "mono" sample that doesn't pan. In classic Dirt, the panning is audible.

telephon commented 7 years ago

Ah that is unintended, it must have been a misunderstanding from past conversations. You can easily play around with different channel mappings as you go:


(
DirtPan.defaultMixingFunction = #{ |channels|
    channels.flop.collect { |ch, i| ch[i] ?? { DC.ar(0) } }
};
~dirt.loadSynthDefs;
)

(
DirtPan.defaultMixingFunction = #{ |channels|
    var which = MouseX.kr(0, 1);
    var first, second;
    first = channels.first.size.collect({ |i| channels @@ i @ i });
    second = channels.collect({ |x, i| x.rotate(i.neg) });
    LinXFade2.ar(first, second, which * 2 - 1)
    //first;
    //channels @@ 0
};
~dirt.loadSynthDefs;
)

// back to "normal"
(
DirtPan.defaultMixingFunction = #{ |channels|
    channels.postcs;
    channels.sum
};
~dirt.loadSynthDefs;
)

there are a lot of possible solutions, maybe we find an optimal default.

bgold-cosmos commented 7 years ago

I believe this is a result of Classic Dirt's behavior being a little confusing, and due to "panning" having an ambiguous definition for stereo signals

IIRC, Classic Dirt will play stereo samples in stereo, but only if pan is unspecified (or maybe also if it's exactly 0.5). Any other pan value will cause it to mix down to mono, then pan.

I believe SuperDirt, by default, will just mix down to mono then pan with any input. If you want something else, like @telephon says there are a number of possible options. In my own SuperDirt what I did was to insert something like this into the dirt_sample_%_% SynthDef in core-synths.scd:

            if (sound.numChannels == 1) {
                sound = DirtPan.ar(sound, numChannels, pan);
            } {
                sound = Balance2.ar(sound[0], sound[1], pan*2-1);
                sound = DirtPan.ar(sound, numChannels, 0.5, 1, {|x| x});
            };

Basically, interpret pan as a balance for stereo samples and preserve that by using the custom mixing function {|x|, x} as an argument to DirtPan. This is still different from Classic Dirt's behavior, but it works for me :)

yaxu commented 7 years ago

Yes I think there would be a better default to the one that classic dirt has. Personally though I'd want 0 to be hard left and 1 to be hard right, and stereo image to only be clearly preserved around the centre. Does that make sense for others?

telephon commented 7 years ago

yes - how would it translate to multichannel output / multichannel in + multichannel output ?

yaxu commented 7 years ago

I can't see how this generalises to multichannel outputs or soundfiles.. 2 in/out seems a special case. But maybe we can make it generalise with an additional parameter, panspread and/or panmode to control the behaviour?

On a tangent, it'd be nice if a ring configuration went from -1 to 1, and if this worked for stereo as well, so -1 to 1 went from right to left, and then back again. Then jux would work for both stereo and multichannel. I think we discussed this before.

telephon commented 7 years ago

On a tangent, it'd be nice if a ring configuration went from -1 to 1, and if this worked for stereo as well, so -1 to 1 went from right to left, and then back again. Then jux would work for both stereo and multichannel. I think we discussed this before.

Currently, both stereo and multichannel panning works between 0 and 1, in stereo it is a left -> right pan, in multichannel it is a full round. How should it be mapped? Do you mean we should interpret -1 for stereo as 1?

kindohm commented 7 years ago

I really can't speak to scenarios where channels > 2. In the case of a 2-channel (L and R) setup, I think pan normally just decreases amplitude on side opposite to the pan. If pan ranges from 0..1:

pan "0" : 100% left, 0% right pan "0.25" : 100% left, 50% right pan "0.75" : 50% left, 100% right

This will work for any min/max/range of pan, whether it's 0..1 or -1..1.

But like I said, I really have no idea how this translates to a setup with more than two channels.

telephon commented 7 years ago

Let's say you have a stereo sound file with a sawtooth wave on the left and a sine on the right. Hen panning it, would the sine ever be audible on the left? And the sawtooth on the right?

kindohm commented 7 years ago

@telephon no. In a two-channel setup, Pan is the expression of the amplitude of either side; it will never blend the two channels. I've never used any audio editing/recording/mixing software that does something different with "panning".

Again, let's say pan can range from 0..1

pan "0" = sawtooth (left channel) 100% amplitude, sine (right channel) 0% amplitude pan "0.125" = sawtooth 100%, sine 25% pan "0.25" = sawtooth 100%, sine 50% pan "0.5" = sawtooth 100%, sine 100% pan "0.75" = sawtooth 50%, sine 100% pan "0.825" = sawtooth 25%, sine 100% pan "1" = sawtooth 0%, sine 100%

telephon commented 7 years ago

well, in the standard mixing, I would have thought that e.g. with pan 0.25, you'd have sawtooth 75 % and sine 25 %?

telephon commented 7 years ago

(Btw. the reason we are discussing here is that we try to find a solution that is more general than the conventions – that naturally involves trade offs.)

kindohm commented 7 years ago

@telephon that is not my understanding or expectation of how pan would work. I would expect when panning to one side that the amplitude remains at 100% on that side. I just confirmed this behavior on both my physical mixer and the mixer in my audio software.

No problem with the discussion; it's good to figure things out :) I am arguing strongly for convention in this case. If there is a more general problem to solve that would benefit from a non-conventional approach, then so be it, but I think it would go against what most Tidal users would expect.

My personal preference would be a conventional panning implementation by default, and perhaps an option for a less-conventional, blended panning implementation when more than two channels are used. My opinion is that this is what most new Tidal users would expect.

telephon commented 7 years ago

@kindohm yes, that would be great if convention turns out to be a special case of a more general scheme. Right now I suspect that this is difficult, the possibilities branch out a bit too much.

I didn't know that on mixers the panning scheme is "hemispherical". It's a good convention, but at least one problem seems to me that it has no meaning outside stereo, has it?

I'm currently rewriting DirtPan so that you can fully define your own defaults. Still, it would be nice if we had a list of named schemes.

bgold-cosmos commented 7 years ago

I suspect the main combinations people use in practice are these:

I don't know if people work with multichannel inputs -> multichannel out, but I've never run into it.

telephon commented 7 years ago

@bgold-cosmos yes, that seems to be about what I have come up in my notes on paper. Although for the 2-2 case I see that sometimes you really want to only have the left channel when you pan to 0. So already here there are are four possibilities: mix+pan, balance, hemispherical and caterpillar like, as discussed above.

I don't know if people work with multichannel inputs -> multichannel out, but I've never run into it.

It is done a lot in electroacoustic music, and I think that tidal is a valuable language in these areas. In any case it is quite idiomatic in supercollider to think in terms of multiple channels mapped in various ways.

telephon commented 7 years ago

@muellmusik how would you do it?

muellmusik commented 7 years ago

Um, I'm not sure exactly what 'it' is here, but a couple of observations:

telephon commented 7 years ago

@muellmusic: thanks for chiming in (pun intended). The meaning of 'it' is: to find a good configurable default of how to pan an N-channel input to an M-channel output, where the most common cases would be N = 1 and N = 2, and M = 2, and where:

There are a couple of parameters that could be interesting enough to expose them:

Tidal has a pan range from 0 to 1 and if I understood him correctly @yaxu suggested it might be a good idea to extend that range to -1 to 1, but to wrap in the stereo case, fore and back.

muellmusik commented 7 years ago

My suggestion would be the following:

  1. Where N == M hard assign.
  2. Where N != M splay without wrapping. By default maximise spread (meaning that for N > 1 first and last input channels should be hard panned to the first and last output channels) and evenly space (meaning that for odd values of N, you have a centre panned channel, including for mono).
  3. Provide a straightforward way to do other common options, i.e. mix down, wrap, etc.

My rationale for this is that it supports the most common cases out of the box (stereo or mono to stereo), but generalises well to different values of N and M. It requires no special treatments (note that 1. is actually just a particular case of 2., and does not require a separate implementation). Splaying, while perhaps not a 'standard' approach, is very useful in live coding once you wrap your mind around it. Apologies if this seems obvious, and hope it is of some use!

telephon commented 7 years ago

In this approach, what would a pan parameter do? Would it move the channels in a ring? Would that include the stereo case?

muellmusik commented 7 years ago

pan could essentially be a centre in a generalised sense. So spread would mean width of the splay, and pan centring the splay. In the mono case spread should be ignored (or at least not used, as it complicates levels), and pan/centre is just location.

Yes, include the stereo case. My instinct is that the default should clip position rather than wrap as in a ring, since the case of panning a stereo field is more likely than surround systems. But a ring should be made very easy to do, maybe just a flag.

2 cents, anyway.

telephon commented 7 years ago

actually, so:

Maybe a width parameter could somehow come close to what @kindohm described, but I'm not sure.

muellmusik commented 7 years ago

I would suggest spread 0 would mean a mixdown, panned to the location indicated by centre. Spread 1 is maximum spread. Sorry the comment above about levels has more to do with width (i.e. is a source panned across more than two adjacent inputs at any given position), so ignore that. -1 could be a special case of hard assigning channels until you run out of inputs (wrapping outputs?), but is that a likely generalisable use case?

I'd suggest mocking up a few interfaces and seeing how they feel in use.

telephon commented 7 years ago

I think I've found a solution that has both behaviours. A parameter splay specifies whether you are spreading to the full number of channels or just to the maximum of th einput channels. This can be useful if you prefer to keep a 1-1 relationship between in and out: #40.

DirtSplayAz : UGen {

    *ar { arg numChannels, signals, spread = 1, pan = 0.0, mul = 1, splay = 1, width = 2, orientation = 0.5;
        var n, pos, channels;
        n = signals.size;
        if(n == 0) { Error("DirtSplay input has not even one channel. Can't pan no channel, sorry.").throw };
        spread = spread * splay.linlin(0, 1, n / numChannels, 1);
        pos = if(n == 1) { pan } { [ pan - spread, pan + spread ].resamp1(n) };
        channels = PanAz.ar(numChannels, signals, pos: pos, level: mul, width: width, orientation: orientation);
        ^channels.flop.collect(Mix(_))
    }

}
telephon commented 7 years ago

@kindohm is this solved by https://github.com/musikinformatik/SuperDirt/tree/topic-panning ?

kindohm commented 7 years ago

@telephon probably, but how do I test? I have superdirt installed currently. How do I overwrite my superdirt quark with this branch? Just clone the branch and copy-paste the files over the quark's files?

bgold-cosmos commented 7 years ago

I've done something like this:

telephon commented 7 years ago

Yes, that's the correct way.

kindohm commented 7 years ago

trying to test. but I can't get any folders of samples to get loaded.

telephon commented 7 years ago

Now merged to 0.9-dev.

kindohm commented 7 years ago

This still doesn't work as I'd expect. In a 2-channel environment, I would expect panning to one side would decrease the amplitude of the other side. |=| pan "0" would result in full amplitude of the left channel and zero amplitude in the right channel. |=| pan "1" results in full amplitude on the right and zero amplitude on the left.

Here's a (crude) example of panning working this way in Audacity. I've loaded a two-channel sample in Audacity with different wave data in both channels. Notice how the opposite side decreases in amplitude as I pan to one side:

https://www.youtube.com/watch?v=OhR_rh-frO8

telephon commented 7 years ago

ok, well, given the discussion we had in this thread, I had supposed that this pure amplitude panning is a special case that would need to be implemented separately.

The problem is that there are so many different situations in "panning" that you need more than one method. But I'll keep in mind the fix-channel-amplitude-pan that you are suggesting. Optimally, it would be a parameter.

yaxu commented 7 years ago

Playing stereo samples in multichannel is currently not working in 0.9-dev for me.

For example this is silent: d1 $ sound "hh"

My settings are below. If I reduce output channels to two, playback is fine.

(
// configure the sound server: here you could add hardware specific options
// see http://doc.sccode.org/Classes/ServerOptions.html
s.options.numBuffers = 1024 * 16; // increase this if you need to load more samples
s.options.memSize = 8192 * 16; // increase this if you get "alloc failed" messages
s.options.maxNodes = 1024 * 32; // increase this if you are getting drop outs and the message "too many nodes"
s.options.numOutputBusChannels = 4; // set this to your hardware output channel size, if necessary
s.options.numInputBusChannels = 2; // set this to your hardware output channel size, if necessary
// boot the server and start SuperDirt
s.waitForBoot {
        ~dirt = SuperDirt(4, s); // two output channels, increase if you want to pan across more channels
        ~dirt.loadSoundFiles("/home/alex/Dropbox/projects/dirt/samples/*");   // load samples (path can be passed in)
        ~dirt.loadSoundFiles("/home/alex/wavs/dirt-jv1080/samples/*");   // load samples (path can be passed in)
        ~dirt.loadSoundFiles("/home/alex/wavs/dirt-impulse/samples/*");   // load samples (path can be passed in)

        s.sync; // wait for samples to be read
        ~dirt.start(57120, [0,0,0,0]);   // start listening on port 57120, create two busses each sending audio to channel 0

};

s.latency = 0.3; // increase this if you get "late" messages
);
yaxu commented 7 years ago

Playback also works in master.

telephon commented 7 years ago

ok, I'll check instantly.

yaxu commented 7 years ago

Thanks telephon, no real problem to use superdirt 0.8 but a nice opportunity to explore the work above! But re-reading it, I realised I should perhaps be working with https://github.com/musikinformatik/SuperDirt/tree/topic-panning anyway?

telephon commented 7 years ago

Isn't that already merged into 0.9? I think it is.

I'm running (in 0.9-dev) with this example, looks fine on the meter view:

(
~dirt.free;
// configure the sound server: here you could add hardware specific options
// see http://doc.sccode.org/Classes/ServerOptions.html
s.options.numBuffers = 1024 * 16; // increase this if you need to load more samples
s.options.memSize = 8192 * 16; // increase this if you get "alloc failed" messages
s.options.maxNodes = 1024 * 32; // increase this if you are getting drop outs and the message "too many nodes"
s.options.numOutputBusChannels = 4; // set this to your hardware output channel size, if necessary
s.options.numInputBusChannels = 2; // set this to your hardware output channel size, if necessary

DirtPan.defaultPanningFunction = #{ | signals, numChannels, pan, mul |
    var channels, inNumChannels;
    var spread, width, splay, orientation;
    //pan.poll;
    width = \panwidth.ir(2);
    spread = \spread.ir(1);
    orientation = \orientation.ir(0.5);
    if(numChannels > 2) {
        signals.collect { |x, i|
            PanAz.ar(numChannels, x, (i / numChannels * 2 * spread) + pan + 1, width: width, orientation: orientation)
        }.sum;
    } {
        //DirtSplay2.ar(signals, \spread.ir(1), pan, mul)
        DirtPanFixed2.ar(signals, \spread.ir(1), pan, mul)
    }
};

// boot the server and start SuperDirt
s.waitForBoot {
        ~dirt = SuperDirt(4, s); // two output channels, increase if you want to pan across more channels
       // ~dirt.loadSoundFiles("/home/alex/Dropbox/projects/dirt/samples/*");   // load samples (path can be passed in)
        //~dirt.loadSoundFiles("/home/alex/wavs/dirt-jv1080/samples/*");   // load samples (path can be passed in)
        //~dirt.loadSoundFiles("/home/alex/wavs/dirt-impulse/samples/*");   // load samples (path can be passed in)
~dirt.loadSoundFiles;
        s.sync; // wait for samples to be read
        ~dirt.start(57120, [0,0,0,0]);   // start listening on port 57120, create two busses each sending audio to channel 0

};

s.latency = 0.3; // increase this if you get "late" messages

)
fdragovic commented 7 years ago

not sure if it helps, but has it got anything to do with the tidal side not being merged because of the spread naming conflict (https://github.com/tidalcycles/Tidal/pull/160) ? also I hadn't noticed that last message, span seems ok? (:

telephon commented 7 years ago

Hm, yes, it all is a bit shaky still. It should work without the tidal side (pan alone should be ok), just missing some features. The problem is that it is not well tested, and I am always unsure about the specification. E.g. on a multichannel system tidal pan 0: should this mean center in a ring, or first channel?

telephon commented 7 years ago

But the parameter should certainly be renamed!

fdragovic commented 7 years ago

I guess it would be kind of useful to keep a center at 0 regardless of how many speakers there are, seems to me like that would keep the same tidal code if we change speaker setups?

fdragovic commented 7 years ago

correction, I typed in the name of the stereo sample wrong! works as expected in stereo!

telephon commented 7 years ago

https://github.com/musikinformatik/SuperDirt/commit/c6f05bdb1c23f8b1ebcdc8705d6f73b961f3d639

telephon commented 7 years ago

can you try with a fresh version of 0.9-dev if it still works?

fdragovic commented 7 years ago

now it's playing from channel 4 onwards according to the meter... I set 8 soundcard channels and 8 dirt channels to play around

edit: mono as well btw!

fdragovic commented 7 years ago

ok now it's working, dont know what I did except restart superdirt! playing around still and I'll report back!

yaxu commented 7 years ago

For stereo we have long had a default of 0.5 for centre, 0 hard left and 1 hard right. It would be nice if the negative numbers also went from 0hard left back to-1hard right. This would then generalise to multichannel in a ring configuration from-1around and back to1. We could keep0.5as the default centre front for compatibility with the common stereo case. Thenjux` would still work in multichannel (it doesn't currently - both channels end up in the same speaker).

fdragovic commented 7 years ago

@yaxu oh ok, I wasnt taking into account that jux uses pan as well!

@telephon here's a video of the meter... both mono and stereo samples are spread across a couple of channels with some empty ones in between it seems? other than that duplication it seems like it's panning around ok though... https://www.youtube.com/watch?v=EoHOeCL-Kh8

yaxu commented 7 years ago

@telephon yes it works now, danke schön!

yaxu commented 7 years ago

If it makes most sense, it would of course be straightforward to change jux et al to use a different range, it is not cast in stone.