Feature Request: .scl / .tun Retuning

[bit-101010]

I would like to see alternate tuning options for oscillators, and perhaps everytihng that does key-tracking.

The most common standards for retuning are the AnaMark (.tun) and Scala (.scl / .kbm) formats. loading these should change the frequencies of each MIDI note.

There are some open tools for doing this already: There is some C++ source code provided by Mark Henning (creator of the AnaMark standard) for using both formats of tuning files, but it handles .scl files incorrectly. ZynAddSubFx handles .scl files as expected, but doesn't load TUN files. Microtonal.h/.cpp handles this function.

In addition (this is a stretch goal, and maybe beyond the scope of this initial request) being able to load and blend between multiple scales would open up a world of harmonic possibilities.

[baconpaul]

Oh that’s a really cool idea. The key to frequency code is fairly well isolated so it wouldn’t even be that tough

Question : surge modulates pitch by semitones like with an lfo or whatever. Wonder if other synths bind their modulation to the scale or use base note in scale space and frequency modulation in even temper space.

Obviously base note in scale space and frequency modulation in even temper space is way way easier!

[bit-101010]

The synths I've worked with keep modulations by semitones. Even pitch bend stays in semitones for some synths. It really comes down to implementation, which varies greatly for this stuff. The main exception I've seen to the pitch bend staying in semitones is AnaMark, which gives you the option of using the pitch bend wheel to morph between two scales.

[baconpaul]

One other question: do you think we should have tuning per osc, per scene, or per patch? I can make all the arguments but interested how other synths do this.

[bit-101010]

I'd go by scene or patch; loading somethings for each oscillator seems tedious.

[baconpaul]

So OK here's some notes on this issue

1. Surge uses an internal representation of key most of the way down as basically "midi note + bend". So if you play a middle c with the pitch bend wheel half way up you get a pitch of "60.5" (60 being the midi for middle c; .5 being the pitch bend). MPE works the same just with different bend granularity

2. The note is converted to frequency in the SurgeStorage function note_to_pitcn and note_to_pitch_inv which do linear interpolation of a static table which goes from integer notes to pitch in range -256 to +256. (note to self: Linear interpolating the inverse is not the same as the inverse of linear interpolation...)

3. That function reads the table "table_pitch" which is a float alignas(16)[512] but is also a DLL global. And that's a bummer because it means the easy way to do tuning (read an .scl file; assume a .kbm which maps note 0 to middle 6 and rotate the frequencies through the scale into the table) will reset it for all surge instances in your process space. Since it is a DLL global.

So to do this the work is basically

1. Make table_pitch and table_pitch_inv members of surge storage rather than globals; change all the callers of note_to_pitch and so on to have a storage reference they pass. (Or similarly squirrel it somewhere else that has the right alignment)

2. Read the .scl file and slap it onto the table for that synth

3. If the scl file has been loaded store it in the patch; if it has been read from the patch blat it onto the table. If there's no tuning, revert to standard tuning (that is even 100 cent tuning in scl speak)

4. Add a UI gesture to load a tuning for the current patch

The tricky part is the moving of the table. Sort of a moderately nasty code refactor. But I kinda want to do this since I would learn a lot... so let me keep pondering a bit when and how to do that refactor.

[bit-101010]

Makes sense, best of luck with the refactor. On the other hand, having a way to retune all instances at once could be nifty...

[baconpaul]

Thanks!

And yea it would! But doing it accidentally because memory is shared is not the way to do it :)

[baconpaul]

Huh that refactor wasn’t as bad as I thought. Just managed to make it work on a branch.

I’m interested in understanding this so may add this to my list for Sooner rather than later

Thanks for raising it

[bit-101010]

By "this" do you mean the refactor/that part of the synth's inner workings? or the possibilities of retuning? Either way, thanks for humoring me and looking into this.

[baconpaul]

The thing that was easy was the refactor

The thing I am interested - and have been interested in for a while - is the inner working of alternate tuning systems

So your issue has the wonderful combination of being code which isn’t a huge pita while also letting me learn about one of the things I want to know more about

So basically: it sounds fun to add retuning to surge. And it also sounds tractable. So I will do it this spring. But probably not this afternoon :)

[baconpaul]

So @bit-101010 I could use a favor.

I worked on this tonight some with SCL files. The headless app can actually play one of the goldberg variations to a wav file no problem so I don't have to deal with UI and streaming patch stuff. And I got it working with alternate scales. (I only tried 12 note scales)

Anyway this .zip file contains a standard midi file, a .scl tuning file, and the wav file that surge renders if it plays that midi file back with that tuning attached in my current branch.

tuning.zip

Is it possible for you to take a synth that you know to be a good renderer of midi with scl files and render out an audio file of it to see if we are doing the same tunings? I'm not quite sure how to test this but have been doing things like "printing velocities at the retune moment" and have confirmed that an .scl file with uniform tuning (so 100.0 200.0 etc...) doesn't change pitches and so on.

Would very much appreciate anything you can do to confirm that other synths with this .scl file render properly though

The branch is baconpaul/tuning-828 if you are interested but like I said the code isn't in the UI to actually use it yet so you have to be able to build and play around with headless to try stuff. Not ready for general use.

[baconpaul]

Hmmm one other thing you made me think. One of my rack modules is a quantizer. Now I know how this works I can make a VCV Rack module that quantizes CV to a .scl file step set also.

[bit-101010]

I've had some ideas about that myself, but I've had trouble getting my Rack plugins to build. There is a whole world of tuning stuff that would make VCV rack even more powerful in my opinion. The main thing I've had in mind is something like your HarMoNee that can do just intonation intervals. Sometimes I feel like I'm on a crusade for the subminor third.

[baconpaul]

Ha! Well I linked the BaconPlugs issue to this one. Happy to chat on that issue tracker about ideas.

It would be very very easy though to do something like my polygnome where you have a CV input and you output that CV plus A/B. So if you want like the dodacahedral scale I used above you could just dial in the 19/12, 1732/1874 and so on. Basically an in-rack .scl editor for N tones.

But lets not spam up surge issue tracker with ideas for rack. Like I said happy to beat around ideas for those plugins.

[baconpaul]

Just a note to self

the table_pitch is calibrated as follows

table_pitch[0] = 1 table_pitch[12] = 2 table_pitch[24] = 4 table_pitch[36] = 8

you get the idea. These are the distances in multiplier from 0 which is why my tuning worked.

But which is the 'center' note. Well in a couple of places it is assumed the frequency is 16.3519783 * the table pitch.

That means we are tuned such that 1 = C0.

So if we want to tune to a 440 A constant and work our scales from there we need to make sure that note_to_pitch(57) = 26.9087 and then calibrate from there. Similarly if we want C3 constant then we need to make sure that well you get the idea.

This explains why my tuning works but my offsets are off. Not fixing it now just writing down my research for myself.

[baconpaul]

Your comments on slack were super handy; I've now fixed up my tuning and have it working.

My current choice is to keep C3 = 261.626hz a constant and use that as note 0. How do other synths do it in absence of a KBM file? Is it an option (so "hold A3=440; hold C4; hold C3")?

It gets weird when you chose a non-12-note scale too. Fun stuff.

[bit-101010]

Probably not the best option given that multiple places assume a 16.3519783hz reference, but if you could make that value a variable, then you could just set it equal to the reference specified by the .kbm file. Then the fractions in a .scl file would translate directly to the values in the table. Start from the reference note and multiply up / divide down by the scale's formal octave (last degree in the scale) to fill it out. The cents-to-decimal calculations wouldn't be too bad to implement either. Unfortunately, that means touching other parts of the code that could be avoided.

Without a .kbm file, you could just assume a reference note and pitch, but the implementations that I like let the user define those. For example, Zyn has parts in the GUI where you can set a reference even if you don't load a .kbm file.

[baconpaul]

Great Surge assumes that pitch value 1 has frequency 16.3 and that note 0 has pitch value 1. Since we have a table we can break the later. So what I’m doing is keeping note 48 constant at 261.5 etc (or really note 48 constant at pitch 16). Then if I use a 6 note scale or something then note 0 ends up with a way lower pitch value.

Seems that 48 constant selection is what I need to let people pick absent a Kbm file

But what I really need to do is make it so you can try this and let me know if my implementation is ok! Maybe next week!

[bit-101010]

Groovy! If there's a way to load .kbm, that's all you need in terms of letting the user select a reference pitch. GUI for that is just an extra bonus.

[baconpaul]

Yeah I will look at kbm files next now I have scl working. Fun!

[SeanArchibald]

I'm now subscribed to this thread and happy to help test the .scl/.kbm implementation when it's available. @baconpaul do you still need someone to test that tuning.zip above?

[bit-101010]

I think he has Tuning.zip testing under control (that conversation happened on the Surge Slack Channel). The BaconTuning branch has retuning somewhat implemented, but it might be a bit before that gets finished and merged in. Right now there's a lot of other v1.6 business, and recently these folks have started porting Surge to VCV Rack modules.

Whenever it does happen, it would be nice to have a tester with a good deal of retuning experience. Nice to have you on board, @SeanArchibald !

[baconpaul]

Yeah I ran out of time last week for other reasons the. Spent the weekend screwing around with vcv - this branch is mostly done. I’ll bring it back up the stack and try and get a beta into the nightlies

[baconpaul]

Just a reminder to myself from slack: http://sevish.com/scaleworkshop/

[baconpaul]

Alright @bit-101010 here's what I have done.

I have pushed a completed version of tuning but none of the associated infrastructure to the nightlies.

This means if you grab a nightly you have a new menu item "Tuning (experimental)" which contains 3 menu items; set to standard tuning; apply an .scl file; and apply a .kbm file

Apply a .kbm file shows an error. set to standard tuning sets to standard tuning.

apply an .scl file lets you load a .scl file and surge then retunes to C4/261hz constant application of that tuning. (That center note is not editable). You can play it and test your scales but - and this is important - right now the scale is not saved anywhere. So if you make a patch with an alternate tuning you are out of luck; if you run a daw session and save it when you come back you will have standard tuning if you restart your daw. That sort of stuff.

So obviously I need to fix and finish all that code before this is a usable feature but with the current setup it is definitely good enough for a user knowledgable in .scl files to download the nightly and test it and report bugs or so on. I tested it all with the sin oscillator and a few others; I notice some oscillators seem to be "off by 2 octaves" so a super small (5 notes to a scale) .scl file works properly on sin but is really wacky with square. But really I have pushed this in so folks can bug hunt it while I"m on vaca a bit, and so I commit to shipping it in 1.6.2 along with a collection of other new features.

The nightly will have just kicked off. I'll pop a note on slack when it's done but basically: any nightly an hour from now or later should have the feature.

Lemme know if it works at all! Thanks

[bit-101010]

It works, somewhat. Thankfully, loading 12edo as a .scl behaves as expected. that's an important box to check off in my book.

It crashes hard if you load a bad file. Haven't looked to far into this but loading a different file type for the heck of it did not go over well.

The different oscillator types have different reference points, i guess, because different types will sound different pitches from the same MIDI note. This seems to be an issue primarily with the size of the scale. a 12 note, 2/1 octave scale sounds the same pitch per MIDI note on every oscillator except Window, which on the 12-22_Dorian scale i've included was a half step lower than the others.

Any non-octave scale isn't handled gracefully, but you acknowledged that on the Slack Channel. I've attached a couple scales and audio demos to test with. 12-22_Dorian is 12 notes to an octave, but not evenly spaced. 19edo has a 2/1 octave across 19 tones 17edX has an "octave" of 7/3 split into 17 equal parts Q4 splits a perfect fifth (3/2) into four parts (i inculded it because it has large step sizes and covers a big frequency range. surgescales.zip

Thanks for the work you've put into Surge, and enjoy the vacation.

[baconpaul]

Excellent thanks. As I mentioned in slack know exactly why the non-2/1 doesn’t work. I’ll fix that first. Hadn’t tried opening random file! Yeah I am not very defensive in my parser at all.

Appreciate the info! I’ll peek a bit more before I leave mid week

[baconpaul]

This was super useful. I was just incorrectly transferring ratios to cents (duh), and also assuming a 2.0. So with my push if you use the sin oscillator I think it is tuned correctly. And if you choose a non-sci file you don't get a core.

About 45 minutes until there's a new nightly. If you want to give a whirl next week that would be great. When I'm back I will look at the other oscillators and persisting the scale in the patch.

[baconpaul]

Oh another feature we will need, when we save .scl in patches, is a menu item to "show current tuning file"

[baconpaul]

Alright I don't have a fix but I have a diagnosis on the mistuned classic oscillator

In the ::convolute method there's this code

   const float s = 0.99952f;
   float sync = min((float)l_sync.v, (12 + 72 + 72) - pitch);
   float t;
   if (oscdata->p[5].absolute)
      t = storage->note_to_pitch_inv(detune * pitchmult_inv * (1.f / 440.f) + sync);
   else
      t = storage->note_to_pitch_inv(detune + sync);

where t is used to set the rate of the voice. In normal operation when this is called, detune and sync are 0; so this is reading the "note 0" which is the "pitch 1" point in standard tuning. Basically this allows the entire tuning to shift by a bit and tune across the keyboard.

But this works remarkably poorly when you have a tiny scale like Q4.scl; at that point the "0" point (since the "48" point is held firm at 16) - then you read the super duper low frequency of the 0 point and apply that to your phase.

I can fix this by assuming the '0' point is 48 and the 'pitch' value there is 16 as follows

   float sync = min((float)l_sync.v, (12 + 72 + 72) - pitch);
   float t;
   if (oscdata->p[5].absolute)
      t = storage->note_to_pitch_inv(detune * pitchmult_inv * (1.f / 440.f) + sync);
   else
      t = storage->note_to_pitch_inv(detune + sync + 48) * 16;

You can see that re-centers the time to 48 as 16 rather than 0 as 1. And then the square wave oscillator on Q4 works just like the sin.

but I am totally not ready to commit that right now. There's so much testing I would have to do that it's not prudent this morning. So I'll leave this comment here so I don't forget.

[baconpaul]

Another “note to self” style comment as I get my part time summer music hacking organized.

OK I think the actual tuning in the nightly is correct so here’s a note to myself on what needs to happen to have this be finished

1. ~Add an internal state variable on whether you are retuned with a .scl file (which init_tables sets to false and retuneWithScale sets to true) and an accessor on storage.~
2. ~Retain the raw text of the .scl file on the storage object~
3. Add a menu item to “show current tuning” which at least works on mac and windows to show the .scl file (and on linux dumps it to /tmp and pops up a message saying where to find it if url open to a file:// doesn’t work). Commit at the point that 1-3 are done and build a nightly.
4. Stream the .scl file into and out of the patch (in association with the rest of #915 things)
5. Make sure that loading an old patch resets the tuning to standard (that is, if there is no tuning section, call init_tables)
6. Make sure that loading a new patch without tuning does the same
7. Make sure that loading a new patch with tuning applies it properly

If that all works then I will call “.scl file tuning done”. At that point I would open two more issues

1. Support for .tun files (https://www.mark-henning.de/files/am/Tuning_File_V2_Doc.pdf)
2. Support for .kbm files (http://www.huygens-fokker.org/scala/help.htm#mappings)
3. An issue for the fascinating suggestion from @bit-101010 at the start of this thread to implement scale blending. That may be out of scope for surge ... but worth thinking about

Those 3 may get tagged 1.6.n not 1.6.2. I’ll see!

[baconpaul]

OK with the push I did tonight .scl support works, is stored in the DAW, is optionally stored in a patch, has reasonable override semantics, shows its status in the UI, and tunes the keyboard properly, which was sort of the minimal viable tuning implementation. So to keep the 1.6.2 milestone chugging along, I added 4 issues with a “tuning” tag which are the work I would add in 1.6.3 probably, and am closing this one, since I think it’s now done!

What a cool thing to add to surge. Thanks for suggesting it and thanks for all the help testing and designing the feature.