image-et-son / p600fw

GliGli based Prophet 600 firmware upgrade
10 stars 4 forks source link

Compatibility issue: Patches sound different on alpha 11 than they still did until alpha 10. #77

Closed el-folie closed 2 years ago

el-folie commented 2 years ago

I guess this is a matter of LFO speed rescaling, but seemingly also some filter cutoff depth rescaling or so?

I did 4 audio files for comparison, 2 sounds, each played on alpha 10 and alpha 11, the differences are hard to ignore.

Desktop.zip

I guess this will eventually be solved as a matter of rescaling/adapting paramaters. So just as a pointer here, if patch compatibility is still a must - of course I can reprogram all my patches if needed, the new functions are more precious to me...

image-et-son commented 2 years ago

Wow, good that you found this. In the first example it is simply the filter being a lot more closed in alpha 11. Is that a difference in the patch LFO speed in the second example or are you playing around with the LFO?

image-et-son commented 2 years ago

OK, now for the LFO speed the issue is this: the rescaling of the "old" patch depends on the storage version. I use the same storage version on all alphas (I even put a disclaimer on the release: "Note that "alpha" means that it is part of the development process. No guarantee for backward compatibility with prior alpha versions.").

But there shouldn't be anything on the bare cut-off. I fixed one thing that has always been wrong: the default patch had a storage version of 0 so it was going wrongly through the whole lot of compatibility transformations overwriting default values. It might affect for instance, the filter velocity!?

In any case, it is correct now (except for other as yet undiscovered bugs :-()

el-folie commented 2 years ago

I didn´t touch the LFO speed at all, it´s really different for the same patch on alpha 10 or 11.

The cutoff difference puzzled me too! The influence of filter velocity was on zero for both alphas, so no influence there.

I really don´t know what could cause the cutoff difference, as it´s also not on every patch, quite puzzling. I´ll send my patches as sysex again and see if anything changes for alpha 11.

image-et-son commented 2 years ago

The LFO difference is readily explainable by the changes I have made, I think. Also I would assume that the envelopes must sound different, because I have globally rescaled the times of attack, decay and release. I you load a patch that was saved in any of the prior alphas into alpha 11 they would be much shorter.

el-folie commented 2 years ago

Yes, LFO speed change is logical due to rescaling...

But cutoff change doesn´t make sense to me either. WRONG: Also on that unison bass patch the filter env amount seems to be on zero for alpha 11, if I flash back to alpha 10 filter env amount is back to normal. Weird. RIGHT: attack and decay were on zero, filter env amount was correct. See comment below:

el-folie commented 2 years ago

Ha! I think I found the reason:

The filter decay value isn´t read on alpha 11 for that unison bass patch in the audio file, I need to raise attack and decay value to match the sound of alpha 10. If I flash back to alpha 10 the filter decay value is back to normal without touching filter decay. So maybe, the new ordering of env shapes somehow influences the filter envelope values?

image-et-son commented 2 years ago

All envelope attack, decay and release times were strongly rescaled in alpha 11 to accommodate the new envelope shape. That is the reason, I think. Compatibility will be given if the patch was stored in v2.0 or V2.1RC3, but not prior alphas (these are not recognized as old patches to be rescaled).

Also: can you please check if you end up with a wrong envelope routing when loading the patch in alpha 11? I moved the parameter in the place where LFO shift was.

Wrong cutoff or filter envelope amounts would point to a bug, though.

el-folie commented 2 years ago

All my patches were stored in 2.1rc1, some were altered a bit in 2.1rc3 and the following alphas. So what do I need to do to retain my patches for alpha 11? Going back to 2.1rc3 (or just alpha 10 is enough?), storing each and every patch one time so that they then will be correctly "seen" in alpha 11?

Envelope routing is set to standard when flashing alpha 11. It´s really just so that all filter envelope times are on zero on my P600, on many/maybe all my sounds...

Cutoff/env amount was correct on alpha 11 - the cutoff just sounded lower because it was missing the filter decay phase going from higher cutoff down to sustain level.

image-et-son commented 2 years ago

Sorry, I should have been clearer on the patch topic and recommend to everybody to store patches in SysEx dumps before load an alpha version. I generally think that just leaving it in memory is not safe enough anyway. If you load a SysEx that was created in 2.1RC3 or earlier that should load properly. As long as you don't save patches in alpha 11 they are still there as they were.

To solve the issue: are the attack, decay and release times really zero exactly? If so, I will provide a hotfix, e.g. a fix just for alpha 11.

el-folie commented 2 years ago

Okay, I did the following:

  1. Going back to v2.1rc3 and sent a patch bulk dump to MIDIOX.
  2. Flashing alpha 11 and sent the bulk dump to the p600 Result: Still envelopes being off like before. When then going back to alpha 10 - envelopes are fine. Verified this procedure twice to be sure.

And I can now give a completely correct answer: due to your superb new parameter pick-up indicator(!) I found out that yes, on those presets with alpha 11, where the envelopes do sound completely wrong, all envelope times on amp and filter envelope are on zero. BUT sustain values were still up at their correct value.

My simple guess would be that the menue changes somehow influenced the envelope times. Maybe a reorder resets the correspondent target values somewhere in the code because the original target (in code) is not being found where it was before? (Just wildly guessing of course... ;-))

image-et-son commented 2 years ago

I have an idea and I'll send you a hotfix version 11.1. Do you use hex or syx?

image-et-son commented 2 years ago

alpha_11.1.zip

Hi, it could be that it is resolved with this hotfix. Can you please send me a couple of 2.1RC3 SysEx patches for ? I don't actually have any...

In this 11.1 you also benefit from yet another UI improvement: not more steppiness on Poly-Mod OSC B!

el-folie commented 2 years ago

Hurray - this 11.1 fixes all the wrong envelope times!

I could send you my personal patches as 2.1rc1 dump as a whole packet. I just don´t want all my patches out in the wild - do you have an email address that I could send them to? Maybe they can also be a little thank you for all the work you already did...

image-et-son commented 2 years ago

I knew it, it was a stupid variable type conversion error, the type of bug which is probably the most common cause of all crashes and exploits in the world...

For mail address: sent via PM on Gearspace. I'd be delighted to hear your patches and would not share them,

el-folie commented 2 years ago

Ha! It happens to the best obviously ;-) So cool, you were able to find the fix!

Will do...

el-folie commented 2 years ago

Bug in alpha 11.1:

The new "preset dial" (Speed) doesn´t have the full range from 00 to 99, instead always random values at the bottom, starting from 03, 02, or 01, up to 78,86 or 94, depending also on speed of dialing the knob. In alpha 11 it worked fine.

And a proposal: when changing the preset number by button presses the LCD scrolls every single time, which visually really makes me crazy. Could it be made so that the LCD just stays still, just displaying the number of button press without scrolling?

image-et-son commented 2 years ago

Hi el-fole, I put out an alpha 11.1 hotfix wih a post on gearspace. Note, however, that the 11.1 I put out is alpha 11 + only that one fix, so it's not the same as the I sent you. The bug (data dial) is not there and hopefully I should avoid it in alpha 12 :-)

el-folie commented 2 years ago

Great - thank you!

el-folie commented 2 years ago

One question about the alpha 10 and 11 difference and the rescaling process of LFO speeds and amounts:

Would it be possible to let the rescaling process actually measure the exact LFO frequency and LFO amount voltage in alpha 10 and then in a second step have these values approximated to the new unified and simplified LFO speed/amount parameters for alpha 11? That way the rescaling for alpha 11 would be so precise that all alpha 10 patches should be like 99% perfectly recreated.

But of course I understand if that procedure would simply add too many precious code lines or being too tedious just for backwards compatibility. Therefore, just an idea that came up while contemplating...

image-et-son commented 2 years ago

Actually: the rescaling transformation in the patch load works exactly as you describe. The reason this does not work for alpha 10 is that in the alpha series the patch version is all the same, so those patches don't appear "old" and are taken as is, e.g. not rescaled.

Proposal: to save you (and others supporting the alpha series test) the trouble of having to readjust patches stored under alpha <=10 I will raise the internal storage version from 8 to 9 in alpha 12. In that way, patches stored in alphas up to and including 10 (but NOT alpha 11, as I am rescaling the LFO amount again slightly because it is too soft) will appear as "old" and will enter the rescaling logic. Just don't store patches under alpha 11. But: if you store patches in alpha versions >=12 and the need for an additional rescaling arises, I will not be able to rescale those patches in future alpha versions. If you have any feedback on further rescaling (LFO speed?), then we should definitely do it in version 12!

Hope this was understandable :-\

el-folie commented 2 years ago

Edit: Ahhh - yes, now I understand the logic behind when to/when not to rescale. So yes, I think the rescaling process still needs some attention:

It´s obvious to me that the 2.1 patch dump I sent you sounds different in regard to LFO initial amount and LFO/VIB speeds on alpha 11. But as these patches were stored long ago and prior to the alphas I don´t understand why the speeds are not approximately 99 % the same on alpha 11, if the actual (old) frequencies are being taken as a precise basis for rescaling.

The two patch examples at the top of this thread show huge LFO/VIB speed differences but I can do new audio examples if needed.

image-et-son commented 2 years ago

Can you tell me what the original LFO speed, LFO speed range and LFO amount is and what the LFO speed and LFO amount under alpha 11 is for patches where it's strongly different? You can read values using pick-me-up mode, the orig. values for example read using alpha 10.

If that doesn't work the next step would be to examine the MIDI SesEx (e.g. dump those to patches under alpha 10 and then under alpha 11.1 and I descramble the data). But for now the display value may be all I need to find the error.

el-folie commented 2 years ago

Thank you so much! I will do that and post it here...

el-folie commented 2 years ago

Okay, here is the result and an idea:

patch no. 22 on alpha 11: LFO Speed to OSC B = value 82 - speed = very fast patch no. 22 on alpha 10: LFO Speed to OSC B = value 82 - BUT speed = almost 4 times as slow

Audio file: I recorded patch 22 (OSC B only) on alpha 10 and 11. On 10 I switched between LFO Speed LO and HI modes with the LO mode the original and correct one. On alpha 11 the same patch 22 has an LFO speed almost 4 times as high as on alpha 10.

Idea: At first I thought that maybe only the "LFO HI range" mode had been used for rescaling, but the alpha 11 LFO speed on patch 22 is still different in frequency than the "Hi range" speed in patch 22 on alpha 10. So it must still be a matter of how to take LO/HI range modes into account when rescaling for alpha 11.

patch 22 alpha 10 to 11 speed comparison.zip

el-folie commented 2 years ago

The same procedure and comparison for patch 23 of my dump. This comparison may be even better as VIB speed is also involved and different.

patch 23-alpha 10_11_comparison.zip

Patch 23 alpha 11 data: LFO speed = 71 (much faster than alpha 10) LFO depth = 79 (seems correctly rescaled) VIB speed = 76 ( also faster than alpha 10)

Patch 23 alpha 10 data: LFO speed = 71 LFO depth = 22 VIB speed = 76

image-et-son commented 2 years ago

Thank you. Interesting result. I now need to think to understand what's happening.

el-folie commented 2 years ago

Thank you for your perseverance!

image-et-son commented 2 years ago

I've got it. Like always: long search, quick fix. Your patch sounds nice now the way you load it into the alpha 11+fixes. I image that you intended it to sound like this. Particularly nice with a touch of mod wheel here an there. :-)

el-folie commented 2 years ago

So great that you were able to find the reason, must have been like the infamous needle in a haystack. Just awesome! Thank you :)

And yes, while my patches are mostly heavily 80s inspired bread and butter sounds as you might have heard ;-), sensitive use of envelopes, LFO speeds and modulation were absolutely deliberate choices...

el-folie commented 2 years ago

Alpha12: Patches still sound different in LFO, VIB speeds and Initial LFO amount. Attached two sounds of the patchdump as audio files:

alpha10_to_12_patch_comparison.zip

But really, if this is getting too tedious, I can just reprogram my patches when the release version is done. You are doing so much excellent work for this project and I don´t want you to feel bad about this one overcomplicated aspect that in the end can be overcome by reprogramming patches...

image-et-son commented 2 years ago

No, I think think that is absolutely crucial. You have a lot of users of 2.0 and 2.1 out there with their favourite patches. I want to reach them and therefore it s important that their patches work. In terms of vib and LFO I have done nothing that couldn't be precisely imported and re-interpreted by the new software version. If it doesn't work, it's a plain bug.

Curious, in one of your examples the frequency is lower in 12 and in the other it's higher!? Which one is vib and which one is LFO?

el-folie commented 2 years ago

Okay, I see - I only wanted to spare you too much work, but if it´s absolutely crucial to you, and I agree, then yeah, let´s go for it to make it perfectly fine...

The first patch in the audio files is patch 23: LFO to PWM by Initial amount - too less depth here. VIB to OSC freq by mod wheel - VIB speed too slow, the LCD actually shows VIB speed value of 00, which may indicate the bug, as the speed clearly is not zero.

Edit: Actually, when selecting the VIB parameters, no values are being shown on the LCD, only when I move the Speed knob. Was it always like this? Can´t remember.

el-folie commented 2 years ago

Okay, definitely on alpha 10 VIB values are shown on LCD, not so on alpha 12. Maybe that´s a hint re patches not sounding the same.

image-et-son commented 2 years ago

With your patch I found one bug: I forgot to rescale the vib frequency on load. But I could find nothing wrong with the LFO and vib amount. The vib amount is practically zero in the original patch 23. Very small: 48 out of 65535 units.

el-folie commented 2 years ago

You are right, the Initial PWM amount was set low. Maybe it did sound different/higher to me prior to that due to the oscillator detuning being different at the time.

And great that you found that bug!

image-et-son commented 2 years ago

OK, I think I managed to fix the whole lot of issue in 12.1. In that hotfix I also extended the low end of the LFO frequency because in the process of fixing the patch compatibility it was an easy thing to do. I wish I had two P600 to test. I am looking for a second one.

el-folie commented 2 years ago

Two P600s just for debugging would be quite dedicated to the task... ;-)))

Is the patch-fixed 12.1 a new release to come or is the fix already in the "hotfixed 12.1" from earlier today?

image-et-son commented 2 years ago

It's the one from earlier today.

el-folie commented 2 years ago

Okay, then patch22 of my dump still doesn´t sound right (LFO still too fast). Funnily though, some other patches seem to be ok now. (I´ve sent the old patch bulkdump again to make sure.)

I think I´d need to go through each and every patch to verify compatibility status of alpha 12.1 to alpha 10. Maybe I´ll do a simple run through recording for testing purposes with mod wheel tomorrow...

Here is the single patch dump (patch22) from 12.1: patch22_dumped from alpha12.1_LFO too fast.syx.zip

el-folie commented 2 years ago

The slooooooow LFO is great! :)

image-et-son commented 2 years ago

In your example: vib amount is zero and LFO amount is very small. LFO target is PW where you cannot really hear that small amount of LFO). I find it hard to hear a difference there. I recalculated all rescaled amounts from both patches, old and new, and I can't find a mistake..

el-folie commented 2 years ago

In my last example, patch22, the LFO speed is about 4 times faster on alpha12.1 than on alpha 10. Maybe I didn´t explain it well enough. And yes, Initial amount is only small, only the mod wheel introduces more PW modulation and that is when the much too fast LFO becomes obvious. Maybe this is just a case where the former "low/high" setting for the LFO was misinterpreted? And surprisingly only for this patch? It wouldn´t make any sense of course, when all patches are automatically being rescaled for alpha12.1 I can only say that the lfo speed really is about 4 times faster than on alpha10.

Hope this helps. I can record some audio of course...

el-folie commented 2 years ago

Alpha13: LFO speed is not the same on patch22 compared to alpha 10. Description in last comment is still valid.

image-et-son commented 2 years ago

Hi el-folie I did compare a few patches in alpha 10 and alpha 13.1, notably 16. I found that the frequencies of vibrato and LFO match but the amounts were wrong for both vibrato and LFO because I forgot to include a deadband in the value conversion. That patch (16) (which I quite like by the way) is now identical as far as I can tell (without two P600 next to each other). So I provided that version just now in 13.2

el-folie commented 2 years ago

Hi, I just wanted to say that the LFO speed on my patch22 (from the dump) is still way too fast on alpha13.4 compared to alpha10. It´s not hard to adjust the LFO speed of course! So, just in case you want to know this to further solve the puzzle about patch compatibility...

image-et-son commented 2 years ago

Hi el-folie. I get the same LFO speed in my test. I think it makes sense to compare the exact procedure.

I have tested two ways: 1) I install alpha 10 and then I load the patch bank from your V21.RC3 SysEx file. I do the firmware update by hex (super fast, more reliable), but it should not matter for this case as long as the patch bank is loaded from the V2.1RC3 file afterwards 2) I install the most recent alpha and load the patches from the V2.1RC3 SysEx file.

I am writing this because if you have stored the patch 22 on the instrument at any time during the alpha phase and you are only doing updates via SysEx (and in that way keep the patches on the instrument) then the storage version of the patch 22 remains version 8 and that version is not properly rescaled. I tried that and the LFO speed is then completely off (50 Hz or so).

My apologies if you are completely aware of this ...

el-folie commented 2 years ago

Thank you!!! I am aware but maybe my approach was flawed or maybe I simply made a mistake all the time, who knows. I will follow your steps exactly with the dump I sent you and report back here later. If it really already is solved, it´d be a dream!

el-folie commented 2 years ago

It´s done!

It was user error on my part, I did not correctly send the old dump by not entering the MIDI management mode correctly, so that the patches in the machine weren´t exchanged for the old dump. What a really idiotic mistake. And obviously I made it again and again without noticing, unbelievable. After now reading the PDF extract again I sent the old dump and all my old sounds were there with correct LFO speeds.

Sorry for my silly mistake and thank you for your perseverance!

Confirmed working - case closed, issue solved. :-)

image-et-son commented 2 years ago

Great!

But it makes me wonder: will users find it difficult to handle MIDI with the patch management mode? Should I program some kind of confirmation that something has been received?

el-folie commented 2 years ago

While generally I always prefer simplicity and minimalism over anything else, in the case of a patch dump, which is quite a big change for the whole synth, I´d think a short message would be good. Maybe something like DUMP RCVD or DUMP SAVD?