How to set a time buffer to the synth

Fanelia82 commented 1 month ago

Documentation Confirmation

I have not found an answer on the documentation

Question's Topic

spessasynth_lib

The Question

I'm using Spessasynth in order to create metronome sounds (i m using noteOn/noteOff, not a sequencer). But I have a hard time to produce a consistent rhythm. the calls are made on time (i have verified that) BUT sometime the lib need a little bit of time to process the request and, at random, i hear a delay.

I looked at the code and i couldn't find a place where introduce a buffer time in order to give time to process data.

Thanks for your great work

PS, i can help modifying the lib if you need help.

spessasus commented 1 month ago

Hi Fanelia,

Can you provide the metronome code you mentioned so I can reproduce this issue?

Fanelia82 commented 1 month ago

Hi

the code is a little complicated...

you can test the issue here

www.nuzic.org/App/

the project is open source and can be found here:

gitlab.com/nuzic/nuzic_app

the sequencer (part) used is from Tone.js and is in the file APP_Player_controller and is something like this:

part = new Tone.Part(function(time, value){
  //stuff here
  SF_manager.synth.noteOn //etc
   //other stuff here
}

compas_sequence.forEach(item=>{
    let pulseN = Math.round(item[0]*(PPM/60))
    part.add({ time: item[0], note: item[1], durationHz: 10, pulse: pulseN})
})

the entire logic is a little bit more complicated, because the app is doing other things other than processing sounds.

In the app I'm using a modified minimized version that I'm testing. The modifications consist in accepting note frequencies or midinote+cents on the calls noteON / noteOff in order to play microtones.

I'm currently testing the minimized version of the lib here: gitlab.com/nuzic/sf_lib

the "sequencer" here is simpler but still give some lag (randomly)

function TEST_play_a_sequence(PPM){
    let midiNote=36
    let cents=0
    let dt=Math.trunc((60000/PPM)*Math.pow(10, 5))/Math.pow(10, 5)
    let deltas=[]
    let max=10//milliseconds
    console.log("PPM:"+PPM+" dT:"+dt+" acceptable delay:"+max)
    _wait_play(0)
    function _wait_play(iter){
        if(iter<15){
            iter++
            //mod noteOn(channel, midiNote, cents, velocity)
            SF_manager.synth.noteOn(0,midiNote,cents,80)
            d = new Date();
            let s = d.getSeconds();
            let ms = d.getMilliseconds();
            //console.log(s+"."+ms)
            TEST.push((s*1000+ms))
            setTimeout(function () {
                SF_manager.synth.noteOff(0,midiNote,cents)
                _wait_play(iter)
            }, dt)
        }else{
            console.log("delta times")
            for(let i=0;i<TEST.length-1;i++){
                deltas.push([TEST[i+1]-TEST[i]])
                console.log("delta : "+deltas[i])
            }
            let max_dt=deltas.reduce((prop,item)=>{return (item>prop)?item:prop},0)
            let min_dt=deltas.reduce((prop,item)=>{return (item<prop)?item:prop},dt)
            let max_delay=((max_dt-dt)>(dt-min_dt))?(max_dt-dt):(dt-min_dt)
            let pass=(max_delay<max)
            console.log("max delay : "+max_delay+" ----> Pass : "+pass)
        }
    }
}

I know i have made a sort of a mess,our project originally used only Tone.js (no delay problem with only this) to both produce sounds and DOM events.

In order to add more instruments i tried various soundfont libs and i stumbled on your. I started testing and modifying on a local copy of my app in order to understand if we can work with your lib, so i ended putting the mod lib in a gitLab repo instead of properly forking from your repo. Sorry about that.

Still, if you are interested on my mods, i can help and merge with your code.

Again, Thanks for your time!

spessasus commented 1 month ago

Hi,

The app you've linked (nuzic.org/App) appears to work fine in my browser (Firefox), though i only clicked the metronome icon and play button since I don't know Spanish. The clicks were in sync

"SF_lib" appears to use an outdated and modified version of spessasynth_lib, so I can't confirm the bug on the latest version. For example compare "worklet_voice.js". Your version uses deepClone, while the latest version has the code rewritten to use classes. Please update to the latest version of spessasynth_lib without modifications as one of the updates may have fixed the issue you're encountering. I'm adding an "invalid" label for now.

spessasynth already supports the functionality you added into noteOn:

For example, use the pitch wheel combined with pitch wheel range Registered Parameter Number or fine tuning RPN to tune an entire channel in cents.
If you want to tune each note individually, look into MIDI Tuning Standard which spessasynth supports. It allows you to tune each specific note number precisely.

PS:

Here's a basic metronome code that I created with the latest version, I don't see any delays or lag.

import { Synthetizer } from "spessasynth_lib"

document.getElementById("go").onclick = async () => {
    const context = new AudioContext();
    const sf = document.getElementById("sf").files[0];
    await context.audioWorklet.addModule(new URL("worklet_processor.min.js", import.meta.url));
    const synth = new Synthetizer(context.destination, await sf.arrayBuffer());
    let i = 0;
    let counter = document.createElement("h2");
    document.body.append(counter);
    // woodblock
    synth.programChange(0, 115);
    setInterval(async () => {
        synth.noteOn(0, 64, 127);

        counter.innerText = "metronome " + i++;
        await new Promise(r => setTimeout(r, 100));
        synth.noteOff(0, 64);
    }, 500)
}

PS 2: If you want to know more about the code or its internal components, feel free to ask me about it here (or preferably in discussions), no need for reverse engineering :)

Fanelia82 commented 1 month ago

Thanks for the answer.

I attach a mp3 (renamed mp4 for compatibility issues with github) of a metronome BPM 160 created with the current nuzic App, it is unnerving the little delays that are present.

https://github.com/user-attachments/assets/da7e8a6e-b220-404e-958c-6d56c3ffdedf

Yes, we need to play every note with an independent tune, for this reason we didn t use the pitch wheel.

I didn't know SpessaSynth supported MIDI Tuning Standard, probably because i used a older version of it. Our app already calculate the needed frequency of the tuned note (microtonal, equal temperament), but i didn t find a way to use it with the lib. NoteOn only accept a midinote, not a frequency, for this reason i opted for the modification: synth.noteOn(channel, midiNote, cent,velocity) AND synth.noteOn_freq(channel, frequency,velocity) (Slower).

Please, can you suggest how to play a single note tuned individually with Spessasynth? I still fail to find the info in the wiki.

With that info I will try to use the original lib from your repo in the app, looking if the issue persist.

Still i think it can be a good option to introduce a synth parameter for a time_buffering, it can be beneficial especially for users with old computers.

Thanks!

spessasus commented 1 month ago

Hi, Here's the MIDI Tuning standard specification. Look into the section called "[SINGLE NOTE TUNING CHANGE (REAL-TIME)]"

To use it, simply use the systemExclusive method.

Here's a helper function I made for you:

/**
     * Tunes a given note to another one
     * @param originalNote {number} the MIDI note that will be affected, 0-127
     * @param program {number} program that the channel must be set to in order for tuning to apply 0-127
     * @param newNote {number} the new note tuning, will effectively replace the value provided at noteOn
     * @param cents {number} the additional cent tuning applied to the new note, 0-100
     */
    function tune(originalNote, program, newNote, cents)  {
        newNote += Math.floor(cents / 100);
        cents %= 100;
        const preciseCents = Math.floor(cents * 163.83);
        synth.systemExclusive([0x7F, 0x10, 0x08, 0x02, program, 1, originalNote, newNote, (preciseCents >> 7) & 0x7F, preciseCents & 0x7F])
    }

And here's a more complete example of using that function:

import { Synthetizer } from "spessasynth_lib"

document.getElementById("go").onclick = async () => {
    const context = new AudioContext();
    const sf = document.getElementById("sf").files[0];
    await context.audioWorklet.addModule(new URL("worklet_processor.min.js", import.meta.url));
    const synth = new Synthetizer(context.destination, await sf.arrayBuffer());

    /**
     * Tunes a given note to another one
     * @param originalNote {number} the MIDI note that will be affected, 0-127
     * @param program {number} program that the channel must be set to in order for tuning to apply 0-127
     * @param newNote {number} the new note tuning, will effectively replace the value provided at noteOn
     * @param cents {number} the additional cent tuning applied to the new note, 0-100
     */
    function tune(originalNote, program, newNote, cents)  {
        newNote += Math.floor(cents / 100);
        cents %= 100;
        const preciseCents = Math.floor(cents * 163.83);
        synth.systemExclusive([0x7F, 0x10, 0x08, 0x02, program, 1, originalNote, newNote, (preciseCents >> 7) & 0x7F, preciseCents & 0x7F])
    }
    await synth.isReady;
    synth.setLogLevel(true, true, true, true);

    let i = 0;
    synth.programChange(0, 80);
    setInterval(async () => {
        i += 10;
        tune(64, 80, 64, i);
        synth.noteOn(0, 64, 127);
        await new Promise(r => setTimeout(r, 10));
        synth.noteOff(0, 64);
    }, 100)
}

I hope this helps.

Fanelia82 commented 1 month ago

Thanks for your time,

still i think this metod will substitute a midi note with another shifted (i can be wrong, i need still to try this solution). That is ok but in reality what my program do is giving the user the capability to redefine what a note is, so it is possible to define a midinote 36[0 cent] and a 36[50] and a 36[80] and play them all together(strange, yes but...)

well, i will mess around with your suggestion and the lib next week.

Thank you very much for your help.

spessasus commented 1 month ago

That is ok but in reality what my program do is giving the user the capability to redefine what a note is, so it is possible to define a midinote 36[0 cent] and a 36[50] and a 36[80] and play them all together(strange, yes but...)

tune(1, 0, 36, 0);
tune(2, 0, 36, 50);
tune(3, 0, 36, 80);

synth.noteOn(0, 1, 127);
synth.noteOn(0, 2, 127);
synth.noteOn(0, 3, 127);

will work ;-)

Fanelia82 commented 1 month ago

Hi!

regarding my rythm problem, sadly using the new lib didn't change my issue. the calls for the noteOn functions were done in time, but the sound is randomly delayed. can it be a problem of the lib time resolution (TICS)?

Fanelia82 commented 3 weeks ago

HI!

regarding my problem, it was a compatibily problem between this lib and the lib Tone.js. i solved the problem simply not using Tone.js part in order to create a sequence of events.

i have created a specialized sequencer for my app, able to process note and cents, loops etc... and included in the minimized/nmodified version of the spessasynth lib that i'm currently using.

thanks for your advice @spessasus , your work is amazing!!!

PS: nuzic.org/App is available in english (see in app the settings) and the website too (i think). It is a project with the objective to explain music theory with numbers and simple arithmetic. The app is a tool for develop the theory further and for composers to create music without the classical limitation of music theory: for example it is great for creating unconventional rhythms, scales and even modify TET and using microtonal notes (in development)

spessasus commented 3 weeks ago

Glad to hear that your problem was resolved!

spessasus / SpessaSynth