Seq66's MIDI timing completely falls apart at JACK buffer sizer larger than 128

[unfa]

Here's a 16th note blastbeat at 200 BPM I used to test this. I am using Geonkick to generate audio so the waveform should be identical every time (it plays internally-generated samples).

At buffer size 2048 the timing is completely broken:

At buffersize 128 - it seems good, but when you look closely at the waveform just before each transient you'll see that they differ and this is audible.

Here's 32 - audibly it's best so far, but we can see that it's still not perfect:

256 is the lowest usable imho but it's jittery:

For my work I usually need buffer of 1024 or 2048, as I do livestreaming and my DSP load is really high.

Here's 1024, which is the lowest feasible for me, but already will cause severe xruns:

It seems like the events are not properly scheduled withing each buffer and they get piled up at the end or start of each buffer cycle? I don't know how this works technically, but no other MIDI sequencer I ever used did this.

Here's the RaySession I used to test this. I'll require you to have RaySession, Carla and Geonkick installed to test: unfa live 2022-09-03.tar.gz

RaySession allows to very easily change the buffer size:

Hopefully this can help :)

PS: I've tried changing JACK PPQN value from 192 to 2400 but it had absolutely no effect.

[falkTX]

looking at the code for jack-midi, there is a single call to write data at https://github.com/ahlstromcj/seq66/blob/master/seq_rtmidi/src/midi_jack.cpp#L375 the s_offset value being used there is always 0, causing events to lose their timing and thus introducing jitter.

reducing buffer size reduces the jitter because the 0 value will be closer to the target, by virtue of the buffer being smaller. with a 8192 size buffer, a value of 0 means there are 8191 possibly incorrect values. with 128 there are only 127, with max deviation being 127 samples.

this is simply lack of event timing being present on the midi messages.

[ahlstromcj]

Thanks for the report and the input. I will investigate and get the fix into version 0.99.1.

I did try to duplicate this, and was able to get very choppy audio in qsynth by setting its audio buffer to 4096, in ALSA, JACK, and with other sequencers, but I think that's a different issue, and I was on the wrong track.

I will do a proper investigation soon.

Question: how would I impress event timing on the MIDI messages in JACK? They already have timestamps.

Thanks again!

[unfa]

Thanks! I was hoping maybe I'll be able to do a Livestream tomorrow with Seq66 to show it off, but I'll have to wait until this is fixed.

[ahlstromcj]

It will be awhile. I got geonkick and raysession installed, and extracted your tar.gz file to ~/Ray Sessions. But when I open the session, geonkick appears in Carla but qseq66 does not. (I also had to run "qseq66 --home ~/Ray Session/unfa..../config" separately to get rid of your MIDI ports and use what I had. So I still have some tinkering to figure out, and maybe a session issue to fix.) The saga continues....

[unfa]

Thanks! Yeah, I had some trouble making qseq66 run inside RaySession, though I think it was a bug in the latter.

[ahlstromcj]

So I gave up on your setup, and am making a manual setup, using either Carla or QjackCtl (trying both). I am stymied at trying to get any sound out of geonkick. I can try a kit and see the VU meter moving, but no sound. I have PulseAudo in play, but it doesn't matter if I direct the output to either of the PA Sources or to System playback.

I put all of the drums on ch 10 (9 re 0) just to simplify things.

Once that's working, how do you display the generated blastbeats?

Thanks!

[unfa]

I have recorded the audio to Audacity, but you could use x42 SiSco (Simple Scope) plugin to view the waveforms in realtime.

[ahlstromcj]

I have got close to being able to provide a sample offset. I can get rid of the major glitches when the frame size is 4096. However, I am pretty sure my calculation of "sample offset" is not quite right.

Where can I find the definition of "sample offset". It's not defined obviously in the jack2 code. How does one calculate it, given the current frame number, the framecount in the MIDI output callback, the tick value of the MIDI event, and other parameters? Thanks!

-------- Filipe Coelho 07:38 Sat 03 Sep --------

this is simply lack of event timing being present on the midi messages.

[ahlstromcj]

Is there any function call a JACK client can make to determine the periods (nperiods) value? Thanks.

[falkTX]

I am not sure what you are confused about.. the process call is an audio block of X number of samples, where X = buffer size. it is called at regular intervals, that follow the real passage of time.

so under 48kHz with 512 buffer size, we have processing with blocks of 512 samples, where roughly after 94 blocks 1 second will have passed (512 * 94 = 48128, which is bigger than the 48kHz sampling rate). the 48kHz in my example here means there are 48kHz samples in a second.

for JACK MIDI, the "frame" just refers to the offset within the process function it is called within. because the process is an audio block, you have 0 up to block-size-frames (512 in my example) of possible values the event is present on.

This is basically how all audio and plugin APIs work, when we deal with audio in blocks.

Sometimes events cross the process call boundary, where a note starts near the end of the block and goes on for a while until it stops in another block (and thus its duration is higher than 512 samples, which means 10.6ms at 48kHz).

[ahlstromcj]

I get all that. But where does the nperiods (e.g. 2 or 3) value come in to play? I'm not interested in audio at this point, just MIDI.

Here's what seq66 does currently with my attempted fixes:

- Adds the timestamp to the messages stored in the output port's ringbuffer.
- In the process callback, extract the timestamp, which will be in units of
  MIDI pulses.
- Use the timestamp, pos.frame_rate, pos.ticks_per_beat, and
  pos.beats_per_minute to translate the timestamp to a frame number.
- Mod the frame number to be within the nframes range as returned by the
  callback.

Does the nperiods value affect the frame numbers? I picked up bits and pieces of how JACK works by looking at source code and information from the internet, but I have not seen any coherent explanation of JACK MIDI framing.

Also, will jack_transport_query() work on it's own, or do I have to have JACK transport enabled in the application in order get proper jack_position_t values?

[falkTX]

what do you mean by "nperiods"? the alsa value? if so, it is completely irrelevant here.

JACK audio and MIDI it is all sample-accurate and in the same thread, same way as plugins APIs do it. the timing of the events is meant to be in sync with the audio, the "offset" means exactly the same regardless if using audio or midi. How you reach to a precise audio-relative accurate offset is up to you. and it is not something exactly unique to JACK, so there is bound to be some info online about it. (it is the same situation as when one tries to add an audio/midi offset to a gui-generated event, typically a calculation based on current-absolute-time vs when next audio callback is expected)

jack_transport_query always works, if you get valid BBT information or not depends if there is a transport master supplying that information at any given moment or not.

[ahlstromcj]

Thank you! Very helpful information. Pretty much every sequencer application I have examined seems to make the calculation in a different way :-D, and sometimes very obscure to track down. Peace.

[ahlstromcj]

So I have been thrashing around this issue for a long time now, and still cannot get rid of lurching notes at a cycle or periods count of 4096. What I have done is (1) replace JACK's ringbuffer with a midi_message ringbuffer, much easier to manage (and tweak); (2) In the process callback, calculate the offset from the timestamp and use that in the event-write JACK function; (3) If the offset is less than the last offset, belay the output of that event until the next process callback. The issues is that there is a lag (typically ~40 ms in my setup, but it can vary a lot) between putting an event in the ringbuffer and then retrieving it in the callback. Now, Seq66's JACK MIDI is based on the (flawed) RtMidi project, and I am afraid I may ultimately have to refactor radically... which I am very tempted to put off until Seq66v2. Non-sequencer uses a buffer as well, but it calculates note offs; but I can't test that because I can't build the app thanks to the GUI fltk extensions project being yanked by the author.

So, unfa, what sequencers have you used that work well? I've look at many projects, but obviously need some more research. A most daunting issue!

The latest can be found in the portfix branch, which also includes work on song-recording (issue #44 revisited) and adds a prettier way to display notes and triggers. If you want, you can build it and see if there's any improvement in "The Seq66 Rag" :-D.

[ahlstromcj]

Another possibility of error just occurred to me. Stay tuned.

[unfa]

I think Giada doesn't have this problem, but I haven't tested it in a long time. Ardour 7 has added a live-looping mode but the codebase may be quite large.

Maybe you'd like to join my community chat? There's tons of users and also developers (including Paul Davis of Ardour and falktx) - you can ask them directly and hopefully get some help!

You can use Rocket.Chat (open-source, self-hosted): https://chat.unfa.xyz Or Discord: https://discord.gg/d8x9aby (proprietary)

Both services are bridged together so you can talk to everyone, but not everyone has matching accounts on both sides.

[falkTX]

I have this working in ttymidi for quite a while. Messages from the serial get a timestamp when entering jack, and vice-versa. Code at https://github.com/moddevices/mod-ttymidi/blob/master/src/ttymidi.c

But realistically, since this is all in software and you do not have to deal with hardware and hardware timing communication, you dont need to care about this at all... When drawing a note, you already know what exact position that note has. You simply have to map between that position to sample-frames, to be happening on the right audio cycle.

Say you have a note at exactly the start of the song. That obviously matches to sample-frame 0 on the very first audio cycle (assuming audio rolls together with transport). Now you have a note somewhere later, say 4 beats later. Just a matter of calculating the frame position based on BPM/sample-rate and beats-per-bar etc. on 120 BPM with 48000 sample rate: we have 120 beats every minute (60*48000 samples), which is roughly 0.000041666.. beats every frame. this value is quite small, so typically it is easier to do calculations based on beat divisions. This is the typical stuff for sequencers. How that works out in the end heavily depends on the engine implementation, if you store time in beats or seconds or something else.

[ahlstromcj]

Currently, the events are timestamped with a pulse value which gets converted to a frame, then offset using the PPQN (or JACK ticks_per_beat) and the BPM. This conversion is done in the callback, which figures out the offset for that frame and uses it, unless the offset is out of order, when the event is the ringbuffer is left alone for the next cycle. Obviously the lag between insertion into the ringbuffer and extraction from the ringbuffer is an issue.

As for giada, I see it is based on the same (flawed) RtMidi JACK implementation. But I haven't yet figured out how to import an existing MIDI file into giada to test it.

[ahlstromcj]

I have had persistent problems at 4096 frames per period (cycle) with the fill-in drum pattern from the Peter_Gunn reconstructed MIDI file. The blastbeats at the end are syncopated! I ran some detailed tests, the results are in the new file contrib/tests/test_numbers.ods in the portfix branch. I also ran it with ttymidi.c's microsecond time method and the iffy "compensation" factor, with the same results. (Search for SEQ66_ENCODE_JACK_FRAME_TIME).

I managed to run the same pattern in ardour... and it was also syncopated.

Any other sequencers you've tried?

At this point, I am putting this one on the way back burner unless some inspiration bubbles up. I am working on a new library for MIDI starting from rtmidi, which will first cover basic MIDI and JACK, and I will use it to dig deeper at some point. I will make a new release with the current mitigation efforts plus some fixes and features. Thanks!

[mxmilkiib]

Possibly https://github.com/jcelerier/libremidi?

[ahlstromcj]

Dang, that library has the same defect, inherited from RtMidi, of always reserving a 0 offset.

I might rebuild for falkTx's microsecond frame time again soon and play some more with it. The big issue is that sometimes an event, which is by calculation meant for cycle C, is not actually processed until cycle C + 1. ("cycle" is one call of the process callback).

[mxmilkiib]

@jcelerier might you have a thought?

[jcelerier]

hmm it should be pretty easy to add a writeMessage(message, timestamp_in_samples) to the API in libremidi that will do the right thing with jack, doing that, thanks for the heads up :)

[ahlstromcj]

I actually had added a timestamp to the send_message() in my perverse implmentation of the rtmidi library. The issue is that there is a variable delay between putting the event in the ringbuffer and getting it out again.

However, your post made me go back to the ttymidi.c module and refurbish my implementation. It's a little different than the original, but works fairly well for the 4096 bufsize case. If more testing proves out, I'll feel a little better about making that 0.99.1 release. Not quite sure where I went wrong before when trying the ttymidi way.

I also looked at your libremidi update. Thanks!