Syncing two advanced audio players

[robclouth]

I'm having a lot of difficulty syncing two AdvancedAudioPlayers. The documentation is confusing and there is only one example. Each player has the sync mode SyncMode_TempoAndBeat and original bpm and first beat ms set correctly.

The way my code works is that it automatically switches between two players as tracks are loaded. The problem is that they just aren't synced. I'm calling syncToMsElapsedSinceLastBeat to sync each player to the other but it just doesn't work. What am I missing?

Ideally I'd like to sync the tracks on a 4 bars quantisation, rather than beat. But the syncToQuantum thing is confusing.

class Player extends SuperpoweredWebAudio.AudioWorkletProcessor {
  tempo = 120;
  isLoaded = [false, false];

  onReady() {
    this.players = [
      new this.Superpowered.AdvancedAudioPlayer(
        this.samplerate,
        2,
        2,
        0,
        0.501,
        2,
        false
      ),
      new this.Superpowered.AdvancedAudioPlayer(
        this.samplerate,
        2,
        2,
        0,
        0.501,
        2,
        false
      ),
    ];

    this.players.forEach((player) => {
      player.outputSamplerate = 44100; // The player output sample rate in Hz.
      player.timeStretching = false; // Enable/disable time-stretching. Default: true.
      player.formantCorrection = 0.5; // Amount of formant correction, between 0 (none) and 1 (full). Default: 0.
      player.fixDoubleOrHalfBPM = true; // If true and playbackRate is above 1.4f or below 0.6f, it will sync the tempo as half or double. Default: false.
      player.defaultQuantum = 16; // Sets the quantum for quantized synchronization. Example: 4 means 4 beats.
      player.pitchShiftCents = 0; // Pitch shift cents, from -2400 (two octaves down) to 2400 (two octaves up). Use values representing notes (multiply of 100), between -1200 and 1200 for low CPU load. Default: 0 (no pitch shift).
      player.loopOnEOF = false; // If true, jumps back and continues playback. If false, playback stops. Default: false.
      player.reverseToForwardAtLoopStart = false; // If this is true with playing backwards and looping, then reaching the beginning of the loop will change playback direction to forwards. Default: false.
      player.timeStretchingSound = 1; // The sound parameter of the internal TimeStretching instance.
      player.syncMode =
        this.Superpowered.AdvancedAudioPlayer.SyncMode_TempoAndBeat;
    });

    this.nextPlayerIndex = 0;
    this.currPlayerIndex = 1;
  }

  onDestruct() {
    this.players.forEach((player) => player.destruct());
  }

  onMessageFromMainScope(message) {
    if (message.SuperpoweredLoaded) {
      const currPlayer = this.players[this.currPlayerIndex];

      const nextPlayer = this.players[this.nextPlayerIndex];
      this.isLoaded[this.nextPlayerIndex] = false;

      nextPlayer.openMemory(
        this.Superpowered.arrayBufferToWASM(message.SuperpoweredLoaded.buffer),
        false,
        false
      );

      nextPlayer.originalBPM = this.nextTrackData.tempo;
      nextPlayer.firstBeatMs = this.nextTrackData.firstBeatMs;
      nextPlayer.syncToBpm = this.tempo;
    } else if (message.type === "loadTrack") {
      SuperpoweredTrackLoader.downloadAndDecode(
        `/test.mp3`,
        this
      );
      this.nextTrackData = message.data;
    } else if (message.type === "isPlaying") {
      if (!this.players[0].isPlaying())
        this.players.forEach((player) => player.playSynchronized());
      else this.players.forEach((player) => player.pause());
    } else if (message.type === "tempo") {
      this.tempo = message.data;
      this.players.forEach((player) => (player.syncToBpm = this.tempo));
    }
  }

  processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
    const currPlayer = this.players[this.currPlayerIndex];

    const nextPlayer = this.players[this.nextPlayerIndex];
    if (
      nextPlayer.getLatestEvent() ===
      this.Superpowered.AdvancedAudioPlayer.PlayerEvent_Opened
    ) {
      this.isLoaded[this.nextPlayerIndex] = true;
      nextPlayer.syncToMsElapsedSinceLastBeat =
        currPlayer.getMsElapsedSinceLastBeat();
      nextPlayer.setPosition(
        this.nextTrackData.firstBeatMs,
        false,
        false,
        false,
        false
      );
      nextPlayer.playSynchronized();

      this.sendMessageToMainScope({
        type: "onLoaded",
        data: this.nextPlayerIndex,
      });

      const temp = this.nextPlayerIndex;
      this.nextPlayerIndex = this.currPlayerIndex;
      this.currPlayerIndex = temp;
    }

    let isSilence = true;

    // this.players[0].syncToPhase = this.players[1].getPhase();
    // this.players[1].syncToPhase = this.players[0].getPhase();
    this.players[0].syncToMsElapsedSinceLastBeat =
      this.players[1].getMsElapsedSinceLastBeat();
    this.players[1].syncToMsElapsedSinceLastBeat =
      this.players[0].getMsElapsedSinceLastBeat();
    for (let p = 0; p < 2; p++) {
      if (
        this.players[p].processStereo(
          outputBuffer.pointer,
          p > 0,
          buffersize,
          1
        )
      )
        isSilence = false;
    }

    if (isSilence) {
      this.Superpowered.memorySet(outputBuffer.pointer, 0, buffersize * 8);
    }
  }
}

[gaborszanto]

Quantum and phase synchronization is explained here: https://docs.superpowered.com/reference/latest/advanced-audio-player?lang=js Please let me know if it helps.

There is a fundamental problem in your implementation: do not make both players synchronizing to each other, because they will constantly try and fight. You should have one main player and let the other synchronize to it.

[robclouth]

https://github.com/superpoweredSDK/Low-Latency-Android-iOS-Linux-Windows-tvOS-macOS-Interactive-Audio-Platform/blob/master/Examples_Android/SuperpoweredCrossExample/app/src/main/cpp/CrossExample.cpp

Thanks for the reply. But in this example you're syncing both players to each other in the process method.

Did you miss a link to the explanation of the quantum syncing?

[gaborszanto]

https://docs.superpowered.com/reference/latest/advanced-audio-player?lang=js GitHub text editor doesn't work for me sometimes.

I'm not sure what your code is doing, but you have two players playing simultaneously? If one is paused, then there is no "flow" to sync to.

[robclouth]

Thanks for the link. Both players are playing. Do I need to set synctoquantum continuously in the process method? In the android cross example you are syncing both to each other always in the process method.

[gaborszanto]

Quantum can be set once, phase is the position inside the quantum, so it needs to be updated every time.

[robclouth]

Sorry this still isn't resolved and there are no examples or other issues for me to refer to for help.

There are two players A and B. Both are playing, I know this because I can hear them. Both are playing the same track. The BPM and firstBeatMs are accurate, I know this because the phase of each player individually is reported correctly. Both have:

player.outputSamplerate = 44100;
player.timeStretching = false;
player.formantCorrection = 0.5; 
player.fixDoubleOrHalfBPM = true;
player.syncMode =  this.Superpowered.AdvancedAudioPlayer.SyncMode_TempoAndBeat;
player.originalBPM = trackTempo;
player.firstBeatMs = trackFirstBeatMs;
player.syncToBpm = 120;
player.syncToQuantum = 4;

I'm trying to sync B to A. Currently I'm calling playerB.syncToPhase = playerA.getPhase(); every call of processAudio but it's not changing the phase of playerB. I've confirmed this by sending the phases back from the audio engine to the UI and showing the phase difference. It should be 0 but it's a random amount that depends on when I start playback of player B.

What are the exact steps I need to take?

Thank you