Play Sample Track Mid-Song

[tresf]

Sample tracks generally tend to be long and are very useful for things like vocals.

When using the "Sample Track" feature, and fixing the arrangement, it's common for a composer to fast-forward into the vocal sample.

When fast-forwarding, the sample doesn't play. Any adjustments must be done as guess-work (assume where the sample is playing) or the song must be played from the beginning each time a change is made.

This is a request to fix the Sample Track behavior to allow playing mid-sample. This would also require the sample track to stop playing when the cursor is fast-forwarded, or else two concurrent versions of the sample may play, which would be an undesirable sound.

Related: #133 #701 #749 #756 #858

-Tres

[unfa]

It's good that you're bringing this up, Tres - I think it should be filed as a bug.

2014/1/24 Tres Finocchiaro notifications@github.com

Sample tracks generally tend to be long and are very useful for things like vocals.

When using the "Sample Track" feature, and fixing the arrangement, it's common for a composer to fast-forward into the vocal sample.

[image: image]https://f.cloud.github.com/assets/6345473/1997517/272b412c-8526-11e3-99ba-2a9512fb1785.png

When fast-forwarding, the sample doesn't play. Any adjustments must be done as guess-work (assume where the sample is playing) or the song must be played from the beginning each time a change is made.

-Tres

— Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/131 .

Tobiasz unfa

-----BEGIN GEEK CODE BLOCK----- Version: 3.1 GIT/MU/P d->-- s+:-(--)> a? C++(+++)>$ ULC+(++)>$ !P? L+++>++++$ E? W++>$ !N-? !o--? K-? !w-- O? !M-- V? PS++ PE++ !Y+ !PGP+? !t(+) 5? !X !R+ tv b+>+++ DI>+ D+ G e h-->- !r y--() ------END GEEK CODE BLOCK------

[Sti2nd]

@unfa How can an unimplemented feature be a bug?

The way to implement it would perhaps go hand in hand with the clock/timer? Now you only need the sample track to be a normal sound player which begins at the time the clock say.

[tresf]

It is filed in our issue tracker and it is categorized as a bug. This request is probably best referred to as an enhancement as it improves existing functionality and does not represent anything broken (per-say), but that categorization is made mostly by Jonathan and it's probably a moot point so as long as the feature eventually gets implemented.

(off topic warning) Reminder, there's no use arguing symantecs around the word "bug". In the software world, all code request changes are often referred to as bugs and tracked with "bug trackers". The term is a bit ambiguous once you work on a development team. Many people use the issue tracker for support requests as well which 90% of the time are neither a bug or an enhancement.

-Tres

[xsleonard]

FWIW, Renoise has this as a togglable 'Autoseek' defaulted false per sample in an instrument. But I'm not sure where that option would go in the sample track UI, if you made it optional.

[oeai]

this is something i was talking about in #690

[oeai]

finally i found how to add track! isn't it better to do it with drag'n'drop file from left bar? instead of add sample-track, add pattern, double-click on, choose file. and yes, it works in simple way, needs to play from start to get music working. and it feels like using it is more laggy than unrendered project (70mb wav file)

[diizy]

On 05/09/2014 08:17 PM, Stian Jørgensrud wrote:

@unfa https://github.com/unfa How can an unimplemented feature be a bug?

The way to implement it would perhaps go hand in hand with the clock/timer? Now you only need the sample track to be a normal sound player which begins at the time the clock say.

This is on my todo list, and it was recently discussed on the mailing list as well - we just need a way to reliably convert miditime to actual time regardless of tempo automation, then it will be possible to also implement reliable sampletrack playing from mid-track. I'm looking into it...

[modeler]

If it is of any interest, Qtractor and Ardour both have this feature; a user can add an audio file as a track and trim / loop it at any point. Personally, I find this an essential part of the production process and the one single thing that stops me using LMMS "for real".

[diizy]

Already being worked on.

[diizy]

Happy to announce I now have a working prototype for this functionality.

[tresf]

:+1: Fantastic news! 1.1 is going to be a killer update.

I can test this out this week if you'd like. Should I use the branch diizy/lmms/tree/sampletrack ?

[diizy]

On 05/21/2014 10:51 PM, Tres Finocchiaro wrote:

:+1: Fantastic news! 1.1 is going to be a killer update.

I can test this out this week if you'd like. Is the branch diizy/lmms/tree/sampletrack ?

Not yet - this is all still on my local repo only. I'm making some further improvements before they get committed...

See I think it's not enough that they play sample-accurately from any position in a way that respects tempo automation, I want to also make it so you can adjust the length (play time) of each sample, and I want the UI (ie. the waveform display that is painted on the track) to respect tempo automation too...

[tresf]

Fantastic. Let me know. @diizy, this feature is tremendously beneficial. I have some old remix competitions where the sample track was 3-4 minutes (no temp automation) where this would really shine.

749

[diizy]

On 05/21/2014 10:59 PM, Tres Finocchiaro wrote:

Fantastic. Let me know. @diizy https://github.com/diizy, this feature is tremendously beneficial. I have some old remix competitions where the sample track was 3-4 minutes (no temp automation) where this would really shine.

Ok @tresf, it's now in there, sampletrack branch. I've also put it up as a pull request here: https://github.com/LMMS/lmms/pull/749

[tresf]

So initial testing:

(Disclaimer: I didn't have Linux running on a physical machine at the time of doing this and audio through the VM wasn't good, so I did my testing on Apple.)

Track Tested: (warning, explicit lyrics)

• (original version for reference) Jarren Benton - Lean (Prod by Gunnar Kennedy)

Performance:

• Using version 1.0.2, my CPUs hover at about 104-114%
• Using version 1.0.90_samplefile, my CPUs hovers at about 130-150%.
• The MSEC timer seems to update very slowly with 1.0.90_samplefile (perhaps due to CPU usage).

Accuracy:

• When skipping forward, audio is off slightly. This may be more than just the sample track that is off but difficult to tell if the rest is on point when the CPU usage is so high. This isn't easily observable at the beginning of track but very obvious at the end.
• Audio is not off when playing straight-through, just when fast forwarding.

Here's a demo: (warning, explicit lyrics)

• http://youtu.be/p1jeWdzvcjA

Notes:

• Since there are may changes in this version versus 1.0.2, the performance issues may not be directly related to the sample track changes.

[diizy]

On 05/22/2014 07:12 AM, Tres Finocchiaro wrote:

So initial testing:

/(Disclaimer: I didn't have Linux running on a physical machine at the time of doing this so I did my testing on Apple.)/

Track Tested: Jarren Benton - Lean (Prod by Gunnar Kennedy)

Performance:

• Using version 1.0.2, my CPUs hover at about 104-114%
• Using version 1.0.90_samplefile, my CPUs hovers at about 130-150%.
• The MSEC timer seems to update very slow with 1.0.90_samplefile (perhaps due to CPU usage).

That is really odd. There are some optimizations that can be made but it definitely shouldn't cause that much increase...

I'll take a look at it.

[diizy]

Ok there are some unanticipated issues... I may have to redesign things a bit.

[diizy]

@tresf, made some updates, issues you mentioned should be mostly solved now.

[tresf]

Thanks. Will test a bit later today and get back asap.

[tresf]

@diizy,

Thank, I've tested the changes.

So it's still off by a significant amount. The youtube video from last night finally finished processing: http://youtu.be/p1jeWdzvcjA

When I bring the audio sample earlier in the timeline by a fraction of a second or so it corrects it, so by the time it is getting to the 105th measure the sample track play position is behind by about that much. (see the amount I've fast moved ahead the green area to compensate?) FYI, BPM is 150.

Note moving the sample track forward breaks normal playback, I just did the visual to help a bit with the amount of compensation needed at the 105th bar to correct the timing/latency issue.

[diizy]

On 05/22/2014 04:00 PM, Tres Finocchiaro wrote:

@diizy https://github.com/diizy,

Thank, I've tested the changes.

So it's still off by a significant amount. The youtube video from last night finally finished processing: http://youtu.be/p1jeWdzvcjA

Ok well I can't hear anything in that video, it's too quiet...

But I think I've figured out a potential problem... the framesPerTick calculation in engine.cpp returns a float value, which is then compared against an integer value in the song.cpp playback routine, meaning that in practice, the frame amount gets rounded upwards... so this causes a frame of offset at each tick, which might add up quite a bit for long tracks.

I'll try to unify the ticks->frames calculations so that we always get the same amount. I don't know if we need a float value anywhere, I think it might be ok to just convert it to an int type... I'll have to see if that breaks anything.

[tresf]

Whoops sorry, I have a bad habit clicking the "close" button, thinking it's a cancel button when discarding a comment.

Yeah the HDMI audio is full blast so I use a 3rd party tool to turn it down, it must have recorded that way.

Thanks for the quick response.

[diizy]

Ok, I have to do some replanning for the sampletrack logic. I think I'll be able to streamline the process quite a bit and make it work accurately.

[diizy]

Ok, well, it's starting to look like this will not make it for 1.1.

There's already so many changes made to core functionality (including song, engine, mixer) that it's probably safe to say that this is going to be pushed to 1.2. Although @tobydox would have the final say about that...

I have the latest changes in the sampletrack branch. Thing is, the timing is perfect when looping, but for some reason there's a slight off-sync that gets introduced whenever the position is changed manually.

I can't figure out why. For all I know, changing the position manually and looping back should be the same thing. I even made a change to how position is changed so that instead of changing it immediately (which may happen in an inappropriate time, since there are no mutexes being used here) so instead now the change is just scheduled if the song is playing, and made sure that any position changes happen at the beginning of the period.

That didn't help, although it's IMO a sensible change to make anyway, so that's ok. But I still can't figure out why the timing goes off-sync with manual changes, but stays perfect with loopbacks... the same code is used to calculate the timing in both cases, so it just doesn't make sense...

[tresf]

@diizy, I just noticed something...

Take a look at this waveform. The kick should hit at the beginning of the measure, but instead it's drawn a fraction of a second later. Are there perhaps markers in the gui that are giving it the wrong timestamp?

[diizy]

On 05/22/2014 11:21 PM, Tres Finocchiaro wrote:

@diizy https://github.com/diizy, I just noticed something...

Take a look at this waveform. The kick should hit at the beginning of the measure, but instead it's drawn a fraction of a second later. Are there perhaps markers in the gui that are giving it the wrong timestamp?

No... I'm pretty sure this is just inaccuracies in the drawing code.

[tresf]

Hmm... that's too coincidental.

I just dragged it to line up and it plays at the right spot. I'd wager a bet that if it's not the cause, then it is the effect of the same underlying bug.

[diizy]

On 05/23/2014 12:25 AM, Tres Finocchiaro wrote:

Hmm... that's too coincidental.

I just dragged it to line up and it plays at the right spot. I'd wager a bet that if it's not the cause, then it is the effect of the same underlying bug.

Is your branch up to date? I'm not seeing any kind of offset like that.

Try with a sample that you know to be timed correctly (eg. one you render from a known tempo in LMMS).

[tresf]

Yes, and now I've reproduced with Ubuntu. I stripped it down to just claps, one per beat. Eventually the waveform is off as well as the audio if it is fast forwarded using the mouse cursor.

Dragging the sample back into alignment fixes it for playing from cursor for the location that is in alignment. This can't be coincidental.

The command I used to grab the branch is:

cd
rm -rf lmms
git clone -b sampletrack https://github.com/diizy/lmms.git
# --> And then the standard cmake, make install, etc.

Edit: This is default 140BMP tempo. Default 44,100 Hz Wav created with LMMS.

[diizy]

On 05/23/2014 02:47 AM, Tres Finocchiaro wrote:

Yes, and now I've reproduced with Ubuntu. I stripped it down to just claps, one per beat. Eventually the waveform is off as well as the audio if it is fast forwarded.

Dragging the sample back into alignment fixes it. This can't be coincidental.

Try the latest source. From what I've tested, there's never been any offset with the waveform...

Try playing a beat with a bb-track in time with the sampletrack, that way you'll notice more easily if there is any problem or if it's just a graphical glitch.

[tresf]

What you are describing is what I'm doing. Should I take a video? You keep telling me to try the latest source but I literally built this from your branch less than an hour ago.

cd
rm -rf lmms
git clone -b sampletrack https://github.com/diizy/lmms.git
# --> And then the standard cmake, make install, etc.

[diizy]

On 05/23/2014 02:57 AM, Tres Finocchiaro wrote:

What you are describing is what I'm doing. Should I take a video? You keep telling me to try the latest source but I literally built this from your branch less than an hour ago.

I also updated it less than an hour ago. Just check that it's up to date... This is all pointless if we're not talking about the same code.

No, don't take a video. There are obviously some weird issues with the timing when moving the position by hand. I have no idea why though because the same doesn't seem to happen when the song loops.

Can you test the looping on long sample tracks? Get a sample track that you know to be aligned to the tempo, then try to loop it at various positions. Also try starting the playback from the middle, see if it's any different from when you move the position while playing.

[tresf]

I didn't try looping yet (no middle mouse button on this computer), but I did download the latest changes and the symptoms are the same.

This is a 44,100Hz 140bpm wav generated with LMMS with on clap per bar. Again, if I can move the sample track into alignment and it plays from the cursor fine. The visual offset is the same margin of error as the audio offset.

[diizy]

On 05/23/2014 03:26 AM, Tres Finocchiaro wrote:

I didn't try looping yet (no middle mouse button on this computer), but I did download the latest changes and the symptoms are the same.

This is a 44,100Hz 140bpm wav generated with LMMS with on clap per bar. Again, if I can move the sample track into alignment and it plays from the cursor fine. The visual offset is the same margin of error as the audio offset.

You don't need a middle mouse button, shift-right click does the same thing.

Also, just to be clear:

1 - song is stopped, move cursor to the middle of sample track, play song

2 - play song, then move cursor to the middle of sample track

Does the same thing happen in both cases?

[tresf]

You don't need a middle mouse button, shift-right click does the same thing.

Thx. Works with delete too. Great shortcut.

Yes, I just tested both scenarios as well as loop points and had what appeared to be identical results.

-Tres

[diizy]

Ok, I think I'm making some progress here... I've been going over the timing and frame counting and debugging them. I should have something that works better soon...

[diizy]

Ok, some progress: I now have the frame calculation working when tempo is static, but it still seems to break down with changing tempo... not sure why.

[diizy]

Ok, I finally got it.

The calculations should work correctly now, I think. There's one caveat: if you put tempo automation in the song so that there is no tempo set at the beginning of the song, this may mess up the calculations, because they can't get the proper value for the start of the song as it isn't written down, so this causes undefined behaviour...

This shouldn't be a problem though, as it's recommended to set all automations at song start regardless, and people who use tempo automation already do that. The only way this could be solved would be to have some kind of "start value" for each model, which would be remembered, and always set when starting playback... maybe it can be done later.

@tresf, can you test the latest updated branch?

[tresf]

I should have some time to test this afternoon.

[tresf]

Testing results:

So I had originally typed up what worked and what did not, but some strange things were happening, it was eventually getting back into sync around measure 115 making me think it was fixed.

So I started a new sample which wasn't as repetitive (beat changed slightly through the measure) and it was much more apparent that there was an issue.

Note that the symptoms are pretty much the same as before. At 800% zoom, the visual wave is drawn off by the same margin of error as the audible sound.

I'll fire it up on a second machine to doubly check.

[tresf]

Confirmed on another computer, same issues. Now I'm just doing a git pull from your branch to update it and then a make install to rebuild the changed files, correct?

IIRC, I shouldn't have to do make clean on changed source code files.

[diizy]

Ok. I tested it myself with a long sample track, and I got the same results. As far as I can tell, here's what's going on -

The calculations are perfectly accurate... they're just too slow. Since we need to take all tempo automation in account when calculating framepositions, this is solved by going from the beginning of the song to the position we want tick by tick and adding up the framecount.

So as far as I can tell, the graphical errors are coincidental and not really related to the audio errors. I've ruled out any possibility of there being any error in the frameposition calculations, I've went through a project, printing comparisons of real elapsed frames and calculated elapsed frames, and they match up perfectly. The error in audio probably gets introduced by missed ticks caused by too much CPU usage.

Mostly this should be solvable by optimizing and improving the sampletrack painting algorithm - I have a feeling that the repaints are most of what are bogging the system down.

As for increasing the efficiency of the timing calculations, that's more tricky and I'm not yet sure if it's necessary... a lot of this stuff would be a 1000 times easier and simpler if we had a singular tempo map, instead of allowing tempo automations willy-nilly all over the project, but what can you do...

If we had a single place for tempo automations, like a special tempo track or something, we could cache the values and update them with eg. signals, we could just go through the timemap and use multiplication to skip large parts of redundant calculations.

Anyway, long story short, this is starting to look like this feature will not make it to 1.1. This is just getting way too complex, there are way too many changes already, which makes me fear bitrot and merge conflicts in the future... I'm probably going to put this thing on hold and concentrate on other things for now. Tres, thanks for all your help and assistance in testing this thing. Hope we'll have better luck for 1.2...

[tresf]

I appreciate the many attempts.

I'm looking at the draw code versus the play code and they both seem to rely on similar TCO calculations and at 800% zoom I still experience the play offset and the drawing offset have the same margin of error.

My inexperience with CPP makes it quite challenging to trace the issue, but I'm still convinced these two issues are linked. If I figure something out, I'll post it here.

[tresf]

This request has been consolidated into a more encompasing Sample Track enhancement bug report and will be closed as duplicate for better tracking. #1471

Please feel free to reference this bug report or even amend or add comments if needed, but it will remain closed for better tracking purposes.

Edit: Closed via #3133.