[1.3.0] Song Export issues

[shuler323]

Hello!

I've collected some projects to work with problems while exporting song/stems and some fixes comes in new fw v1.3.1b3. For now, exporting song is much faster (e.g. up to 0.9x of real song time, ~5 min song tested). Exporting stems is still time consuming, but more reliable.

Work is still in progress, so I hope we'll deliver more efficient stems export soon. If there are still some problems, feel free to attach projects in comments with description below. Every new related issue will be redirected here.

Note that export will work smoother if you have quite lot free uSD card space.

Known issues that are still in development:

• no stereo separation on track stems
• slow stem export

Here are some projects which i've tested:

(EDIT:) final timing results in 1.3.1b3 are much better

@thebradclay sync test *tempo fx changes fixed

@jaap3 spoilt cabbage 223s song 199s took song export (0,89x song time) 6464s took song stems export (28,99x song time, yeah... slow but successful ;) )

@clickysteve old-fashioned amusement 195s song 176s song export (0,9x song time) 5056s stems export (25,93x song time)

@ancipital deep cows 71s song 61s song export (0,86x song time) 949s stems export (13,37x song time)

[thebradclay]

@shuler323 What does "quite lot free uSD card space" mean exactly? How much free space? Am I going to constantly need to remove samples/projects from the card to keep this unknown amount of space free to have better stem rendering? What size and read/write speed card should I be purchasing then?

Edit: When will the current fixes be available???

[bgdnrz]

Pardon, I don't want to derail this thread, but what factors would greatly improve export times? You mentioned a big memory card. As per @thebradclay - can you quantify how much free space? What is the point in mb at which the returns in export speed are negligible?

Beyond that, is the main bottleneck the CPU or the actual algorithm?

Generally I appreciate the strict adherence to doing everything on the Tracker. However, I wonder if there's any possibility of some kind of computer-based emulation that could be done for the purpose of quickly generating exports? What I mean is - you dump your project folder onto your computer, run "trackerexport.exe -d=ProjectFolder -stems" and it dumps out the WAV's utilizing your computer CPU in a non-realtime manner (meaning, you don't have to wait 5 minutes for a 5 minute song).

Totally impossible? Difficult, I understand because you have people on Mac, PC, Linux, but just brainstorming here...

Either way, thank you for your efforts on this, it's a pretty important issue, I reckon, so am grateful that it is being focused on. And great to see a 10% performance increase already!

[DJadolfo]

OK, Song Export in 1.3.1b2 now works, thank you. However the process is not heard during Export (I mean, you can't listen to the Song you export). Well, may be fine. May be confusing. If the process works - I'm satisfied. There are other improvements I would wish, and the main one is that I still do not get how correctly Midi Program Change (now simply MIDI CC F) works. I would wish this effect to cahnge timbre or drum pattern on an external device. What I get now is that it is always working on one MIDI channel (so limiting me with either sound module to change its timbers, or drum machine its patterns), I would wish them both (althrough obviously timbre module has Channel 1 input and drum machine Channel 10, as example).

[MetabaronCZ]

Its OK that song is not played during export. It means that more processing power goes to export so it will be faster. They could even turn display off to speed it up even more :).

[clickysteve]

Hey @shuler323

Thank you for the update.

I read in the lines forum from one of the Polyend developers that the stem export will still have tempo inconsistencies across the full song, but they will be in synch with one another. That means that the output will need to be warped/stretched in a DAW. (As in, tempo would be expected to be at 130bpm, but would come in at 129.77bpm, or something similar).

In the report here about this bug previously, before the full export engine rebuild, the stems were out of synch with each other, but not to the extent we now face: https://github.com/polyend/TrackerIssues/issues/281 - this could be rescued with DAW stretching (whereas the current exports can't easily). The exports at this stage were much faster, about 4x the song length.

If the core tempo drift is essentially going to be a wontfix, that raises a couple of questions for me:

1. Will the tempo drift be consistent per step (as in, not have significant jitter to different extents), or will some sections of the exported stems be faster/slower than others?
2. If we are going to have to use flex/warp time in our DAW for the exported stems anyway, then are we not as well just using the previous export mechanism, since it didn't take 5+ hours?

Number 2 would obviously not be ideal, but if I have to warp the tracks anyway to compensate for tempo drift, I think I'd rather be able to export them quickly.

[t8pt]

Is the core tempo drift considered as a wontfix on the programmers side? Hopefully not! Stretching/Warping in the DAW is only a provisory solution for me because every stretch algorithm has its tonal side effects.

[clickysteve]

Hopefully not! Stretching/Warping in the DAW is only a provisory solution for me because every stretch algorithm has its tonal side effects.

+1

[clickysteve]

Seems like I'm not going to get a reply, which is disappointing.

It appears that a new version of the firmware is out, which says there is 'drastically improved' tempo accuracy. Can somebody from Polyend please answer the above questions, and explain what 'drastically improved' means? Are the stems in synch? Is the tempo consistent across the stems (but the overall bpm different to what is expected)? If nothing else, these details are needed for efficient testing.

[ancipital]

It appears that a new version of the firmware is out, which says there is 'drastically improved' tempo accuracy. Can somebody from Polyend please answer the above questions, and explain what 'drastically improved' means? Are the stems in synch? Is the tempo consistent across the stems (but the overall bpm different to what is expected)? If nothing else, these details are needed for efficient testing.

Is it time for a sweepstake? Take a project that is noticably drifty and sounds like a bad DJ clanging it after 8 bars and choose a number until the new version trainwrecks? Bags me 16 if so...

It's a bit alarming that it's supposed to improve drift- doesn't claim to eradicate it. Something like tempo of exported stems should be completely deterministic and 1:1 in its correctness.

Oh well, have downloaded the new firmware, will give it a go this evening if I get time.

[thebradclay]

I'll build a quick project and test it right now. Fingers crossed.

[shuler323]

@clickysteve

The exports at this stage were much faster, about 4x the song length.

nope, previous export algorithm was based on real-time playing, stems were generated one by one, so it took much longer - (song time) x (number of stems)

which is disappointing

sorry for that

'drastically improved' tempo accuracy.

it means +-0.001BPM accurracy, tested on FL, is this ok? :)

Are the stems in synch?

they are now, just check new beta :)

@t8pt

Stretching/Warping in the DAW is only a provisory solution

yep, we know that, so we keep on improoving this functionality, for now there is no need for stretching, I think

@bgdnrz

I wonder if there's any possibility of some kind of computer-based emulation that could be done for the purpose of quickly generating exports?

we've not planned any emulation, now it runs much faster that it should not be necessary

@MetabaronCZ

They could even turn display off to speed it up even more :)

we've considered that but finally decided to keep display on :D

ok, so... remember to write something positive about export in new (1.3.1b3) beta below :)

[clickysteve]

Thanks @shuler323

Song Stem Export

I've just tested the export. Generally good news:

1. Exports in general are much faster. As in, they now take minutes, as opposed to hours. This is good news.
2. Stem exports appear to be in time with each other. There is still a gap when you zoom in, but this could simply be down to the particular samples used. I can't quite tell at this stage. Perhaps someone else can test with a simpler project.
3. Stem exports appear to be in time with the expected BPM.

There are a couple of issues remaining with song stem export:

1. The master export appears to be out of time with the stems export. See below screenshot.

2. The individual tracks appear to be 'late' on the bar. e.g the following:

Note that the master is even later (so further off the beat).

Pattern Stems I haven't been able to test pattern stems in any kind of detail, as it appears you can only export pattern stems one pattern at a time. Is that correct? ideally you could export all pattern stems at once.

Export levels

I can't tell if it's just me, but I am sure the export levels are now much hotter than they were previously - with some of my tracks clipping/maxxed out in places. I need to go back and compare the stems from previous tracks, or see if they can be compressed somehow.

Overall Impression

Generally this is 1000x better. The stem export now happens at a reasonable speed, and the stems appear to be in synch with each other across the song - with no significant drift that I can make out.

There are still some issues in terms of the export's relation to the beat, as well as them being out of time with the master export (so presumably also out of time with the reverb and delay stems, but I haven't been able to test this yet). I can't see a way to export all pattern stems at once, so haven't tested.

Overall, this update means that the song stem export is now much more usable, and the song can be manually corrected to be on the beat. Oh, and it looks like the issue with the generic export names from #512 has also been fixed, even though it wasn't mentioned specifically - so that is great.

This is a relief for me, thank you for working on this.

[doelman]

@shuler323 this sounds massive! Just before I check: are the stems in stereo now? That was another issue I opened, as of 1.3 the stems were mono (or should I say two channels with exactly the same information)

[clickysteve]

@shuler323 this sounds massive! Just before I check: are the stems in stereo now? That was another issue I opened, as of 1.3 the stems were mono (or should I say two channels with exactly the same information)

Yes, stems are in stereo.

[thebradclay]

I just made a quick two pattern song with a tempo change on step 1 and 17 of pattern two. I exported the song and exported pattern. Immediately after export, the patterns disappeared as in all the steps were gone. From the file page I could not press anything and when attempting to power off by holding the button it was stuck in a loop and I had to pull the plug.

Test "song" is roughly 16 seconds long. Song export was quick. Song wav seemed okay.

Stems came out as a "rawFile" on first try.

Put the sd card back in the tracker and exported stems again. Did not freeze this time and patterns were as they should be. Pattern export took about 33 seconds.

Master has a loud click present in the wav (looks like a square) between the first two kicks, added my test files if needed.

Beta131b3.zip

Stems exported fine, delay and verb are present, all in stereo. They are still not in sync though and there is a difference in the master and individual patterns. Upside is the tempo change is present in all stems but the drift is there.

@clickysteve I can confirm my master that was exported with the stem is as you described. Also, yes the stems do seem much hotter, very noticeable with the reverb and delay, I can live with this as I can just drop some gain but they are hot.

[MetabaronCZ]

@shuler323 Hi. Thank you for your work. I tested full song export.

SONG:

• duration: 4:12
• 13 patterns
• 8 instruments

EXPORT TIME:

• firmware 1.3.0: about 8:29
• firmware 1.3.1b3: about 2:39

Both exported files have same similar size about 43.4 MB and I would say they sound identically.

Interresting thing here is that during export I actually hear some high frequency parts of my song. The sound is very low - I would say the volume is similar to ground noise reported in other bugs.

If I cancel export I hear reverb tail of notes - should I report it as bug?

[clickysteve]

@shuler323 this sounds massive! Just before I check: are the stems in stereo now? That was another issue I opened, as of 1.3 the stems were mono (or should I say two channels with exactly the same information)

Yes, stems are in stereo.

Oh, actually. It seems the stems may not include pan data, suggesting they aren't true stereo. Will need to test this further.

[doelman]

Not including the pan data would make stems useless for me. But it wasn't mentioned as solved so I guess this bug is still open.

[clickysteve]

To carry out some more tests, I created a new project file with the latest beta firmware version. This involved:

1. Eight tracks which have a snare hit every four lines.
2. I trimmed the sample so it starts bang on the bar each time.
3. Each track contains the same sample, with no variation in placement, time, pitch, or whatever else.
4. Track 1 has panning effects - alternating between left and right hard pan. The rest are set to the centre.
5. I created a song with all tracks active which spanned 40 slots.
6. Tempo of 100bpm exactly.

This would allow an idea of:

1. Whether panning info is captured in the output stems.
2. Whether or not tracks really are in synch with each other, as each sample is the same.
3. Any drift over an extended period.

The results are as follows:

1. All of the individual tracks appear to be in synch with each other across the full export.
2. The master track starts out in synch with the rest, but then there is some kind of glitch sound, and it goes out of synch - falling behind. You can see this clearly in the waveform.
3. Generally, the tracks don't match up with the beat - they are about 12 'ticks' or points behind - even though they are in synch.
4. Panning effects are not exported. Meaning that the stems are double mono, not stereo.

At the start, all in line (though late on the beat)

and then by the end, with the master clearly out, but the rest in synch.

a zoom in showing that the other tracks are in time with each other:

Here is a clear image of the master track glitching out after the first pattern, after which it is no longer in time:

REVERB TEST

The next test I undertook was to repeat the above, except I:

1. Added in reverb with a pre-delay of 20 (the rest default settings)
2. Changed the length of the song export to 15, rather than 40 (so it would export a bit quicker)

The results were as follows:

1. The same as above, except that reverb appears to stay in time with the project over its length. After one pattern length, the master glitches and is out of time for the rest of the project.

In time, at the start...

Then after the glitch on the master:

One other thing I noticed, is that when loading up an old project, the output was noticeably louder on various tracks. This appears to happen with samples that previously were too quiet, and which I had to boost the volume on - which now are distorting. I guess that means the output volume or gain has been adjusted at some point. It would be good if that could be clarified, as while it's not a bad thing (it's probably a good one) it means going back and editing old projects.

In summary:

1. Individual tracks are out of time with the beat consistently.
2. Individual tracks appear to be in synch with each other, and are consistent throughout the length of the project (no visible/obvious drift).
3. Individual tracks match the BPM of the project.
4. The above appears to apply to the reverb export.
5. The master track is synched with the others, but then glitches after one pattern length and is out of synch consistently after that.
6. Tracks do not contain stereo information (e.g. panning, etc).
7. Gain of tracks are definitely hotter than before.

Note that I did not test the delay track, as I ran out of time.

I would share my projects, but they are large. I'll try upload the smaller project (reverb test) below in a separate comment.

p.s. when do I get a job as an official tester? ;)

[clickysteve]

I can't share my project file, as it's 500+mb. This is a very easy test to replicate though, so trust that my projects aren't required.

[thebradclay]

I just did another super simple test and got some really strange results.

betaTEST131B3_Song_S0005.zip

Exported the song, 4 slots, 4 patterns, only 2 samples and 2 tracks used, 128 steps, 100bpm.

Track one, starting at step 1 I placed a sample on every other step. Pan hard left Track two, starting at step 2 I placed a sample on every other step. Pan hard right(different sample than used for track one)

Same pattern for patterns one-four. Exported.

Ableton reads the imported files correctly, bpm is 100.

FIXED: Ableton Live trims the initial silence from wave files. Next to the warp settings make sure Start is at 0 Problem is the exported stems have the hits at exactly the same time starting on the one for track two even though the sample should start at step two. Stems are not panned.

The master.wav has the the track correct, panned as should be, however there is an audible and visible hiccup at almost the exact same place as I mentioned in the previously uploaded file. At 1.1.2

[thebradclay]

Exported stems from a 1.3.0 made song. Got all the stems except for Reverb and delay also got a RawFile.

EDIT: Exported a second time and all came out fine.

Export is much faster now. That is greatly appreciated.

[jaap3]

I exported my project (spoilt cabbage, one of the projects @shuler323 tested with). Stems came out perfectly in time. Pan effects are indeed missing in the stems, as others have noted. There was also a nasty surprise in the master waveform, a very loud noise burst:

I haven't tried exporting a second time, and the noise burst is absent in the stems. Might just be a one time glitch, but a very nasty one when you're wearing headphones.

[jaap3]

I exported song stems of another project. There's a noise burst is the master waveform as well, pretty much at the same spot as the previous project.

[thebradclay]

@jaap3 I had this issue with a couple of master files and haven't with others. I think the ones that I did have the issue were created on an earlier beta as I haven't had this issue with anything created in 1.3.3b

[jaap3]

I started that project just a couple of days ago, on the 1.3.1b3 firmware. The Tracker did flash itself to an older firmware in between (1.0.5, I'm unsure why/how it happened, I immediately flashed it back to 1.3.1.b3). So maybe?

[thebradclay]

No clue. As far as the tracker flashing itself (lol funny sentence) the manual only mentions a random ptf being chosen if you were to hit the hidden reset button, "if your device lost power during the upgrade, the SD file structure was damaged, or the firmware doesn’t boot up for any other reason, you can perform one of the two emergency flashing procedures as follows:

First method:

just simply press the hidden reset button on the back panel of the instrument and it will automatically flash with a randomly picked firmware .ptf file from the SD card Firmware/ folder."

That is strange though.....had you previously lost power during a session or during an upgrade? Just curious.

[jaap3]

You know what, I might've somehow triggered the hidden reset button. That makes sense

[thebradclay]

@jaap3 That reset button would be difficult to accidentally trigger as it is recessed and you would need to activate it with something small like a paperclip. Did you possibly lose power for a second?

[jaap3]

Someone else is reporting a random downgrade here: #801.

[thebradclay]

I think it has something to do with the the "randomly picked" part of this function. Something trigged that, I have no clue what but I bet this function is involved. "press the hidden reset button on the back panel of the instrument and it will automatically flash with a randomly picked firmware .ptf file from the SD card Firmware/ folder."

[ancipital]

I wonder if the newest beta has an intermittent startup crash or something that causes this, in very specific circumstances? I think it boots a random firmware when there's a boot failure too, doesn't it?

[thebradclay]

@ancipital you know that's a good guess. I have had a couple of issues powering down on this beta, where holding the button just makes it cycle right back to "on". No issues on power up though, which is just anecdotal.

[rgroot14]

Firmware version: 1.3.0 master.zip

I've got a hardware synth I'm sequencing with the Tracker. I have the audio running into the line in. If I just hit play, everything sounds great. When I export the song or pattern, the audio from the line-in is mangled badly. I've tried a couple of different hardware synths and I always get the same results. The only workaround I can find is to mute the incoming audio, but then I don't get the audio on the exported wave file. Is there a way for me to properly export the line-in audio, or is this a bug in the firmware?

I've attached the master.wav export file. The first pattern is just a sample instrument and it sounds fine. The second pattern is the sampled instrument and the line-in audio. It's mangled badly. The third pattern is just the line-in audio, also mangled. TURN DOWN THE AUDIO BEFORE YOU PLAY THIS - it's painful. :)

[doelman]

Any update about #580 ?is this expected to be fixed in 1.4?

[clickysteve]

Hey are the stems still double mono, or stereo at this point? Unsure of the current status @ambv . Thanks!

[DJadolfo]

Hey are the stems still double mono, or stereo at this point? Unsure of the current status @ambv . Thanks!

I did the stem export in 1.3.1.Final - and the Master is true stereo while all the tracks are "double mono" indeed. This is a problem as I use a lot of panning effects, so I have to record those tracks on DAW via audio out (muting the rest 7). The good news is that the tempo stays ok both for audio or stems. @ambv please let us know when this will be fixed (and I dislike the horizontal workflow idea, please keep the vertical as it is).

[clickysteve]

Hey @ambv @shuler323 any word on when the audio export will be fixed so it's stereo rather than double mono? That's the main issue left with audio exports at this point.

[clickysteve]

Looks like this has been fixed, using the latest beta.

[soulfish]

Looks like this has been fixed, using the latest beta.

Hey @clickysteve, I can confirm that 1.4.0b1 fixes the lost panning in stem export for me, which is fantastic news, the stems I export now reflect the stereo sound design I did in the Tracker, the exported stereo tracks do not have the the same data on each channel anymore. :partying_face: