kewlniss / CSharpSynthForUnity

Ported CSharpSynth Project from http://csharpsynthproject.codeplex.com/ to work in Unity
MIT License
78 stars 19 forks source link

Spatial sound #4

Open PEPSoares opened 5 years ago

PEPSoares commented 5 years ago

Great stuff first of all!

I had a problem while parsing the files because of float separator ".", my parse was expecting "," If anyone gets this problem use. CultureInfo.CurrentCulture = CultureInfo.InvariantCulture; If you don't mind I'll try to change Parse with TryParse which I think will solve the problem.
Also, I wasn't able to use spatial sound with this. Is this a limitation ?

Thanks

JimmyCushnie commented 4 years ago

I'm working on a major overhaul of this library, which I call LogicSynth, for my project. I will open source LogicSynth when it is complete, probably later this year. In my new library I figured out the spatial audio issue and I'd like to share my findings here.

The way we play generated audio in Unity is with OnAudioFilterRead. This method gives you an array of floats with the currently playing audio data, and you modify that array to add effects -- or, in our case, to play entirely new audio.

Unity does all its fancy spatial audio fun BEFORE it gives the float array to us, not after we pass it back. If the source doesn't have a clip, the float array will all be 0. But if it has a clip, the float array will be data from that clip which is scaled appropriately for the spatial audio fun.

Problem: we've been replacing the supplied data with our synthesized data. This removes all the spatial audio data.

Solution: set the source's audio clip to a wave with all of its points at peak amplitude. Unity will scale those points based on the spatial audio fun, and we can then multiply our synthesized audio data by this scaling value.

Here's how I do this in LogicSynth:

[RequireComponent(typeof(AudioSource))]
public class SynthPlayer : MonoBehaviour
{
    private AudioSource Source;

    private Synthesizer Synthesizer;
    private float[] sampleBuffer;

    private void Awake()
    {
        Source = GetComponent<AudioSource>();

        Synthesizer = new Synthesizer();
        sampleBuffer = new float[Synthesizer.BufferSize];

        Source.clip = GenerateSpatialSamplingClip();

        AudioClip GenerateSpatialSamplingClip()
        {
            float[] samples = new float[Synthesizer.SampleRate * Synthesizer.Channels];

            for (int i = 0; i < samples.Length; i++)
                samples[i] = 1f;

            var clip = AudioClip.Create("Spatial Sampling Clip", Synthesizer.SampleRate, Synthesizer.Channels, Synthesizer.SampleRate, false);
            clip.SetData(samples, 0);
            return clip;
        }
    }

    private void OnAudioFilterRead(float[] data, int channels)
    {
        Synthesizer.GetNext(sampleBuffer);

        for (int i = 0; i < data.Length; i++)
            data[i] = sampleBuffer[i] * data[i];
    }
}