BPM Not changing

[TomasMax22]

Im using the latest version in mono While playing rokuchounen to ichiya monogatari almost in the end the song slows a little bit but the BPM does not and it gets desynchronizated till the end of the song Rokuchounen to Ichiya Monogatari.zip .

[alexankitty]

I took some time to look into this. I found that the song starts off 200ms off, and then of course as you witnessed at the end, the BPM change kinda just doesn't happen and then the whole song is off by several beats. This appears to be a problem specifically with tja2bin which the project leverages to convert them into fumen format, and still occurred even when I converted directly instead of through tjaconvert. Also note that using a tjaplayer it's perfect all the way through.. My worry with this is that it's also potentially messing with a lot of other songs.

Though the only way to really fix is going to be reverse engineering how the fumen format is built. And it seems the internet is very scant with information regarding it. The most I know is that they're in gzip format, and it seems like each measure is padded by a block of FF bytes. Though according to other projects like Okku which is used for converting to fumen for 3DS, the format has a few limitations.

[vivaria]

Hi there! This issue is actually 2 issues in one:

1. 200ms offset: I believe this is due to TakoTako's processing of the .ogg file. (Incorrect, see comment below.)
2. BPM desync near the end: I took a few months to reverse engineer the fumen format, and I discovered that there is a specific adjustment that needs to be done whenever there is a #BPMCHANGE. I have tested this behavior extensively, and I created a new converter that doesn't have any of the #BPMCHANGE issues that the current tja2bin.exe converter has. Please try it out! https://github.com/vivaria/tja2fumen

So, we're halfway there, and I'm hoping to work together with @Fluto to fix the first issue, too. 😄

[vivaria]

I've discovered the cause of the full-song desync. tl;dr:

• Each TJA's OFFSET value (multiplied by 10) is being added to the fumenOffsetPos field in the data.json metadata file.
• Songs with a larger OFFSET value experience greater desync.
• Rokuchounen has a large OFFSET of -11.4s, so this results in a desync of 114ms, which is very noticeable.
• Setting fumenOffsetPos to 0 fixes the desync issues entirely.

Here is a more in-depth write-up:

Investigation

---- In talking to @Fluto in Discord DMs, he identified this line where `millisecondsAddedSilence` is set:

https://github.com/Fluto/TakoTako/blob/c269bcab60530877a16c2a473c84796b94d0a5ce/TJAConvert/Program.cs#L120 https://github.com/Fluto/TakoTako/blob/c269bcab60530877a16c2a473c84796b94d0a5ce/TJAConvert/Program.cs#L26-L27 * If `OFFSET` is greater than `-1`, then 2000ms of silence will be added * If `OFFSET` is less than `-1`, then 0ms will be added In Rokuchounen's case, the OFFSET is `-11.4302`, which is less than `-1`, so 0ms is added to the song (2:22 total song length). However, if I manually edit the line to add 2000ms to the song (2:24 total song length), then the offset is naturally _much much worse_! (e.g. ~200ms off before, ~2200ms off after.)

Given these details, I don't think `millisecondsAddedSilence` is the cause of the desync. ---- Separately, I *did* notice a specific field in the `data.json` metadata file called `fumenOffsetPos`:

- In my official fumen, `fumenOffsetPos` has a value of 0 - In the converted TJA, `fumenOffsetPos` has a value of -114 The different (114ms) seems awfully close to the ~200ms offset that was noticed. So, I tried updating the converted TJA's `data.json` to use a value of 0, and... it's synced up perfectly!!!

Aha! `fumenOffsetPos` is definitely the culprit here. ---- Just to confirm that `millsecondsAddedSilence` isn't causing any desync, I tried converting a song with OFFSET > `-1` that I know is synced up decently well (`Arc.tja`). Its offset is `-0.649`, so, as described previously, an extra 2000ms of silence is added to the audio file. And, accordingly, Arc's fumenOffsetPos value is `1994`. This means that `millsecondsAddedSilence` is compensated for, and isn't the cause of the problem. (In fact, I think the added silence is just to make sure that song with small offsets don't start too early, which would disorient the player. This is a good feature, @Fluto!) --- However, given the behavior with both Arc and Rokuchounen, I think what is happening is that the `fumenOffsetPos` value is computed by adding `millisecondsAddedSilence` + `OFFSET` (but... off by a factor of 10). - Rokuchounen: 0 + -11.4302s = -114 - Arc: 2000 + -0.649s = 1994 My theory is that the larger the OFFSET, the more desync is noticeable. This is why Rokuchounen (-11.4s) and Doppelgangers (-8.7s) result in a very noticeable desync of 114ms and 87ms respectively, but a song like Arc (-0.6s) results in a desync of 6ms, which is hardly noticeable. Given that the Ultramix songs use a fixed offset value of 0, I think the solution here is to _not_ add the OFFSET, and instead use a fixed value of 0, too. (Or, a value of 2000 if the silence is added.) I've tried using a value of 0 on other songs, too, and it fixes the offset entirely!!

[Fluto]

Issues addressed in f728d694b38da732d06a687b14ebbe676a6ff435. This fix will be available in the next update. Thank you everyone for their feedback! And thank you @vivaria for working on a fix.

[vivaria]

For anyone using an older version of TakoTako (v3.2.0 and below), I wrote a quick little script to fix the values of the fumenOffsetPos (see https://github.com/Fluto/TakoTako/issues/16#issuecomment-1636960820).

This way, all of your converted TJAs will be in sync, even when using an older version of TakoTako: https://gist.github.com/vivaria/663ac4ffaf37345f0abd55e1cc8dfe32