timing irregularities with transport-synced buffer playback

[jinpavg]

1. In Max, the patch plays back very steady on the beat, synced to the transport
2. In Unity, it's quite off-kilter, every note is a little bit before or after the beat, and every few seconds (10-20), there's a hitch or stutter, like the beat is resetting
3. In the StepSequence.cs script, I have two versions of the transport stuff -- the one that is active is using the old method of accessing a transport, and the one that is commented uses Transport.Global -- both of these systems lead to the glitchy playback, but the Transport.Global version gives the ArgumentException: GCHandle value belongs to a different domain error in the console when entering game mode the second time.

Here is RNBO in Max recorded into Ableton vs RNBO in Unity recorded into Ableton:

If you access the Unity Profiler, you can see peaks during the glitches:

Find attached, the patch I used to build the plugin, and then via dropbox the unity project itself.

https://www.dropbox.com/s/c6o0iyeeqm4t3wy/RNBOSeqTest.zip?dl=0 ariaStepSequencerV2-debug.maxpat.zip

[jinpavg]

Another data point -- if you toggle DSP buffer size to "best latency" in Project Settings > Audio, the timing seems much improved from beat to beat, but there is still the glitch/stutter step every so many seconds.

[jinpavg]

Another data point -- if I build the Unity project, I no longer get the glitch/stutter step, but timing from beat to beat is still irregular. I think there might be a few things of note here:

1. there's this stutter step in the editor, not in the build
2. there's bad drift in both editor and build
3. We're getting the GCHandle error on restart (a known editor-only issue?)
4. We could disallow (or at least warn) against accessing RNBO's transport through the Plugin methods (ie Plugin.SetBeatTime()) if you are also doing so via Transport.Global

[arokhsar]

With regards to the glitch/stutter issue, I found that it only happens when using the global transport. This code produces no glitch/stutters:

void Start()
{
    AriaStepSequencerHelper = AriaStepSequencerV2debugHelper.FindById(instanceIndex);
    AriaStepSequencerHandle = AriaStepSequencerHelper.Plugin;
    //        AriaStepSequencerV2debugHandle.RegisterGlobalTransport(Transport.Global);
    //        Transport.Global.Tempo = tempo;
    AriaStepSequencerHandle.SetTransportRunning(true);
    AriaStepSequencerHandle.SetTempo(tempo);

}

But if you comment out the last two lines and bring back the two lines before it, the glitches return.

Note that this doesn't impact the timing problem.

[x37v]

@arokhsar that is very helpful information, likely there is something slightly wrong in the global transport computation

[jinpavg]

a couple of interesting things to add here. ..

1. the glitch/stutter does not occur for me on Windows, even using Transport.Global, though I can reliably repro on macOS
2. the drifting timing seems to be an event/transport issue, not an audio glitch. The screenshot below compares two Unity plugins recorded into Ableton -- the first blue track is the transport-synced patch attached to this ticket, and the second green track is exclusively signal-rate, with all the timing handled in gen. You can see that this second device lines up nicely with ableton's 120bpm grid:

[x37v]

@jinpavg I just pushed an update to main that i'm hoping will fix this.. can you update and try again?

[jinpavg]

In testing this, I'm getting an issue when running the plugin in-editor:

I get this in a fresh 3D URP project, with the current RNBOTypes package and this attached package made from the patch earlier in this thread using the latest from main --

RNBOSeqFixTest.zip

Here's the script to control that plugin:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Cycling74.RNBOTypes;
​
public class StepSeqFix : MonoBehaviour
{
    RNBOSeqFixTestHelper seqHelper;
​
    [SerializeField] float tempo = 120f;
    [SerializeField] float beatTime = 0f;
    [SerializeField] double[] steps = { 1d, 1d, 1d, 1d, 1d, 1d, 1d, 1d};
​
    readonly System.UInt32 sequence = RNBOSeqFixTestHandle.Tag("listSampleOne");
​
    [SerializeField] int instanceIndex = 1;
    [SerializeField] AudioClip buffer;

    // Start is called before the first frame update
    void Start()
    {
        seqHelper = RNBOSeqFixTestHelper.FindById(instanceIndex);
​
        if (buffer)
        {
            float[] samples = new float[buffer.samples * buffer.channels];
            buffer.GetData(samples, 0);
            seqHelper.Plugin.LoadDataRef("sampleOne", samples, buffer.channels, buffer.frequency);
        }
​
        RNBOSeqFixTestHandle.RegisterGlobalTransport(Transport.Global);
        Transport.Global.Tempo = tempo;
        Transport.Global.BeatTime = beatTime;
        //Transport.Global.Running = true;
​
        // seqHelper.Plugin.SetTempo(tempo);
        // seqHelper.Plugin.SetBeatTime(beatTime);
​
        seqHelper.Plugin.SendMessage(sequence, steps); 
​
    }
}

[jinpavg]

Ok! So the crashing is no longer happening. But my experience in terms of the timing issue and audio glitch in editor remains the same --

1. Stutter glitch after ~10-20sec when running in editor, but not build
2. Timing irregularity, with audio triggering before or after beat, in editor and build, for example:

[x37v]

Timing irregularity, with audio triggering before or after beat, in editor and build, for example:

okay, so we've got jitter in the timing maybe.. i wonder, have you compared this to rnbo~'s output with transport sync? It would be worth knowing if it is a rnbo issue or a unity plugin transport issue.

[jinpavg]

Ah, yes... this image is RNBO in Max vs the same patch as a Unity plugin:

[x37v]

Ah, yes... this image is RNBO in Max vs the same patch as a Unity plugin:

okay, good to know!

[x37v]

@jinpavg one more test there, can you make sure that the buffer sizes for unity and max match when you do the comparison?

[jinpavg]

this is reproducible running in RNBO in Max with higher sigvs and iovs. Now #18385!

[x37v]

@jinpavg if it is a rnbo issue and not a unity audio plugin issue, should we close the ticket?

[jinpavg]

closing!