time slowly drifts away from what it should be

[felixniemeyer]

FluidSynth version

FluidSynth runtime version 2.3.4 Copyright (C) 2000-2023 Peter Hanappe and others. Distributed under the LGPL license. SoundFont(R) is a registered trademark of Creative Technology Ltd.

FluidSynth executable version 2.3.4 Sample type=double

Describe the bug

I'm using fluidsynth to generate audio for which I have event timestamps so that I can train a neural network to detect certain audio events. I use mido to read the timestamps of relevant not_on messages and tempo_change messages.

In almost all files the event timestamps I get from my script and the rendered audio from fluidsynth are perfectly in sync. Accidentally, I came across one file, where that is not the case. It's almost sync in the beginning but around 1s diverged at the end of the song.

I did some investigations and with fluidsynth -v the last time change message is logged like this

fluidsynth: debug: tempo=845070, tick time=17.605625 msec, cur time=593518 msec, cur tick=33792

The tempo is never different in the whole file. If you do the math you also get a timestamp that is slightly off from the 593518 msec that fluidsynth logs.

>>> tempo * 33792 / 48 / 10**6
594.92928

So I think this is an issue with fluidsynth. There are roughly 180 tempo change messages, so I don't think it can be rounding issues (because offset is around 1000ms) - but maybe it is.

The file is a bit weird, maybe, because it repeats a tempo change at every bar.

I upload it here for you:
Jack.zip

Expected behavior

I'd expect the notes in the rendered wav to play at the time that is specified in the midi file.

Steps to reproduce

• download the troublesome file. Jack.zip
• execute fluidsynth -v -F investigate/manual.wav -ni -a file --gain 1 -O s16 -r 44100 .../Billy\ Joel/Captain\ Jack.mid
• check the last line that starts with fluidsynth: debug: tempo=845070, ...
• do the math based on the tick (ticks_per_beat = 48 and tempo is 845070 from tick 0 on)
• see if it matches the current time in ms from the fluidsynth debug line

[derselbst]

I can confirm your observation, thanks for the report. However, I must also admit that fluidsynth's MIDI player was never meant for being accurate in any sense. Its only purpose was to provide a convenient way to preview MIDIs supplied via the CLI combined with a certain Soundfont. There have been various additions to the player from various contributors over the past releases and there's not a single unit test. Hence, using this logic for training a neuronal network is a bad idea. And to be very frank, I neither have time nor interest to investigate and fix this myself, sorry.

For accurate rendering I highly recommend using a MIDI parsing library combined with fluidsynth's sequencer API: https://www.fluidsynth.org/api/Sequencer.html

This also allows you to enqueue custom callback events, rather than abusing tempo change events as far as I understand. But even if you did, timing will be accurate as you've calculated - I just checked it using ANMP.

[felixniemeyer]

@derselbst thanks for confirming my observation! And thanks for the recommendation of using sequencer API. I might try it this way.

It's interesting you mention ANMP - maybe it has some export functionality or command line way to convert midi to wav.

I also found a project called timidity++ that seems to be able to convert midi

[felixniemeyer]

I think this can be closed!