Read functional scale data, reference MIDI note, & reference frequency?

[vsicurella]

I've been using this library in a MIDI tuning plugin I've been working on for SCL and TUN import, and I'm having trouble figuring out how I can interpret a functional tuning from TUN files under the CSingleScale class. For reference, I've been using TUN's from Sevish's Scale Packs (generated by Scala) and ones exported by ScaleWorkshop.

Is reading in the functional scale data beyond the scope of using this library for reading TUN files? Or am I missing something? I can write my own parser, but I would like to use the official library if possible. I prefer to use the functional scale because I have a framework that supports multichannel MIDI input, so I don't want to go off of the built-in tables if I don't need to. While I do have an experimental function that attempts to extract a periodic scale from a frequency table, it seems silly to use that on TUNs that contain functional scale data.

But even then I have some issues getting the reference note & frequency defined in the file. If I open these TUN files in a text editor, I can see the MIDI reference note and reference frequency at the very end of the file. They seem to correlate with the preceding tuning tables as well. However, when I use CSingleScale.GetBaseNote() and CSingleScale.GetBaseFrequency(), I'm only getting the default initialized values, note 69 and 440hz, and there doesn't seem to be a method of getting the reference values.

I'm a bit confused. Any help is appreciated, even if it's just acknowledgment that I'm not using this library in the intended way!

[zardini123]

Hi @vsicurella! Thank you for your issue!

Is reading in the functional scale data beyond the scope of using this library for reading TUN files? Or am I missing something?

As per TUN spec, the functional scale data is read in as "formulas." The list that contains all the formulas after reading the TUN file is in m_lformulas, but for some reason its private. There only a public method to add formulas. Formula management is part of the TODO of the all-encompassing tuning library that I've been working on that is currently hosted in branch api_redesign.

Whenever a formula is read into a CSingleScale, the formula itself is applied to the frequency table (see the CFormula::Apply function here). I'm having troubles exactly understanding your question. Though if your question if its possible to get frequency data from the [Functional Tuning] section, yes. If you are asking if it's possible to get the formulas, currently no, but you can if you pop open m_lformulas by making a public function for it.

While I do have an experimental function that attempts to extract a periodic scale from a frequency table, it seems silly to use that on TUNs that contain functional scale data.

Would you possibly like to contribute your ideas/code for solving periodic scale from frequency conversion? Mark Henning and I were looking into a method of solving that problem months ago. Mark did propose a solution that was really interesting, but we haven't had time to implement it. We'd love to hear what you are trying!

But even then I have some issues getting the reference note & frequency defined in the file.

May you attach an example file and point out what exact part of the file sets this reference note/frequency? Reference has a couple different connotations in TUN making me confused, so I'd love some examples to get my mind back on track.

Curious to hear what you think!

[vsicurella]

Thanks a lot for your speedy reply! The clarification on the spec helps, and I'm excited to hear it's being considered in the API redesign. To clarify my intentions, my goal is to get a cents representation of the scale, as CSCL_Import::GetLineInCents delivers, and also get the MIDI note on which the TUN scale is rooted. I did consider modifying the library to achieve this but I didn't want to heavily depend on that, and I was concerned it would compromise the integrity of claiming the app complies with the TUN spec. But if accessing that information is planned for future, I feel a little better making a workaround and then reimplementing it later with the API redesign.

May you attach an example file and point out what exact part of the file sets this reference note/frequency? Reference has a couple different connotations in TUN making me confused, so I'd love some examples to get my mind back on track.

So, here's an example file I generated with ScaleWorkshop. If you scroll to the bottom under the [Functional Tuning] section, you can see the lines: ; Set reference key to absolute frequency (not scale note but midi key) note 60="! 261.625565"

When I use either CSingleScale::GetNoteFrequencies or CSingleScale::GetMIDINoteFreqHz, I do get the expected values. However, when I use CSingleScale::GetBaseNote and CSingleScale::GetBaseFreqHz I am expecting to get 60 and 261.625565 respectively, but I've found they return the default values of 69 and 440.0, and I can't really tell what the real root note and frequency are.

Perhaps, the real issue is ScaleWorkshop doesn't include an InitEqual(note, frequency) line to start the [Functional Tuning] section? Even still, I find it a bit odd that CSingleScale::GetBaseNote and CSingleScale::GetBaseFreqHz will give values that are irrelevant to the frequency table. This partially complicates the scale extraction approach since I might be using the wrong root note/frequency.

Would you possibly like to contribute your ideas/code for solving periodic scale from frequency conversion? Mark Henning and I were looking into a method of solving that problem months ago.

Sure, I'm more than happy to offer my idea!

My method assumes the target scale is periodic. If there is no given root, you can start from the first frequency or look for the first whole number frequency.

• Starting from the root index, redefine the remainder (ascending) of the frequency table in terms how many cents one frequency is from the previous one. Basically the first derivative of the table in cents.
• Loop through the cents steps in incrementing-sized chunks. I'll refer to the first n-sized chunk as the pattern.
• Compare the n cents-steps in the pattern with those in the rest of the chunks.
• When a pattern and chunk have a cents-step that does not match, restart with a chunk size of n+1.
• When every chunk matches each cents-step of the pattern, consider that pattern being the first derivative of the target scale.
• Convert the pattern into an absolute cents scale, and commit that as the result.

In consideration of rounding errors, it's probably sufficient to use three - four decimal places of precision.

I suspect there's a more savvy way to execute this concept, possibly using an FFT based approach, but I don't know how to engineer that right now.

[vsicurella]

Another note, I noticed TUNs exported by Scala don't include any explicit mention of a reference note & frequency. It's just embedded in the frequency table, so I guess there's not really an ideal way to determine a "real reference" from that.

In my app, I think eventually I'll provide a basic set of controls that allows a user to manually select the beginning and end frequency of a scale since it'd probably be much easier to find exactly what you're looking for that way.

[zardini123]

However, when I use CSingleScale::GetBaseNote and CSingleScale::GetBaseFreqHz I am expecting to get 60 and 261.625565 respectively

Oh wow, this problem shows naming of API is really everything 😆. Like you alluded, the values of GetBaseNote() and GetBaseFreqHz() are only ever set when calling InitEqual() or by embedding a function call of InitEqual = (69,440) in your TUN file itself. That function is optional as per spec, so no, Sevish Workshop should not include it. It is a problem on this library's API.

My method assumes the target scale is periodic. If there is no given root, you can start from the first frequency or look for the first whole number frequency.

Wow, yeah if I remember correctly Mark came up a really similar solution! I'll come back here to see your idea again once I get into implementing the conversion for the library redesign.

Note that our most recent problem we have been tackling is not just converting table to periodic statically, but also dealing with conversion dynamically. Dynamic scale formats like MTS and MTS-ESP are forefront in the redesign of this library. Let me know if you also have any ideas for that!

I did consider modifying the library to achieve this but I didn't want to heavily depend on that, and I was concerned it would compromise the integrity of claiming the app complies with the TUN spec.

So funny thing about that. baconpaul of Surge synthesizer asked if there was a reference implementation of TUN, and I was unable to have an answer. Even though Mark himself wrote this (which is really awesome), we as outside observers have no idea if this really works perfectly to the spec. We can prove that with a testing suite, but I have not had the time to make that. Just from me using the library and reading the code alongside the spec, so far I've found no problems (other than what we found now). 😄

So there are changes that should be done to satisfy these issues:

1. Make a read-only function called GetFormulas() that just returns a const reference of the CFormula list for people to screw around with.
2. Preferably, make public functions IsEnsureHzFormula(), GetEnsureHz(), and GetEffectingNoteIndex() inside CFormula to find that reference frequency and note.
3. Change GetBaseNote() -> GetInitEqualBaseNote() and GetBaseFreqHz() -> GetInitEqualFreqHz(), and add documentation that ensures understanding that this is for InitEqual only.

This is not set in stone, so please let me know if you have any ideas. If you have time and are willing to implement it, I'd appreciate your contribution with a pull request!

[zardini123]

Actually to append onto the list, I think that there should be a helper function in CSingleScale that then takes the 3 things mentioned above and do:

1. Loop through each CFormula and find the first one (is that per spec?) that is a ensure hz formula.
2. If one exists, output its note index and frequency.
3. If one does not exist, output standard 69 440.

Though I don't think this takes into account InitEqual properly. I'd have to hear what you think first tomorrow before I look more into it.

[vsicurella]

I'm so glad this issue is proving to be productive! 😁

Preferably, make public functions IsEnsureHzFormula(), GetEnsureHz(), and GetEffectingNoteIndex() inside CFormula to find that reference frequency and note.

This would be nice. One minor note is I think you'd want to use GetAffectingNoteIndex() for silly English reasons 😛

Change GetBaseNote() -> GetInitEqualBaseNote() and GetBaseFreqHz() -> GetInitEqualFreqHz(), and add documentation that ensures understanding that this is for InitEqual only.

Yes, in my opinion that would immediately clarify things!

If one does not exist, output standard 69 440.

Just to double-check my assumption here - if there is no "ensure hz" line, then scale note 69 will always be mapped to the frequency 440? If so, that sounds perfect. If not, I wonder if it might be more useful to use note 69 by default, however return the associated frequency from the [Exact Tuning] table rather than a default 440hz.

Though I don't think this takes into account InitEqual properly.

Hmm that's a good point. I can't find an explicit direction in the spec on what should happen if both an InitEqual line and an "ensure note hz" line are provided, and particularly conflict with each other. But the code makes this clear with this line in CFormula. I did test this with a TUN you can view here, but of course this does not affect the output of GetBaseNote() and GetBaseFreqHz(). I personally find it confusing to have two separate commands that define a specific note's frequency, unless I'm missing a nuance between them. But I suppose for compatibility purposes there's not much we can do other than providing alternate functions and making it clear what they do in the documentation.

There's one more thing I would find convenient, and would argue it could provide better consistency across TUN implementations, which would be some way to parse a CFormula into a relative cents scale. I'll have to better digest the CFormula class to provide a more detailed suggestion, since it seems that the same CFormula can provide various results given different frequency vectors in CFormula::Apply due to referential and frequency shifting tokens. It's pretty neat that they can generate a wider variety of frequency tables that way, but having spent more time with the SCL format, I'd ultimately like to get an array of cents that can be applied to an arbitrary base frequency.

[vsicurella]

Dynamic scale formats like MTS and MTS-ESP are forefront in the redesign of this library.

Yes, this is definitely something I'm concerned with supporting too! I've laid down some boilerplate stuff in my app with the intention of supporting dynamic tunings, but I haven't went in depth with them yet. I don't yet have any immediate thoughts, but will be happy to share them when I do.

If you have time and are willing to implement it, I'd appreciate your contribution with a pull request!

For sure, I would be interested in contributing! I do have a separate project I need to prioritize, but I expect I'll be able to invest some time in this within a few weeks.

Also, I did get my extraction method working as described, and you can view the code here and feel free to borrow from it.

[zardini123]

One minor note is I think you'd want to use GetAffectingNoteIndex() for silly English reasons

So I spent like a minute trying to think of the difference. Does GetHasAnEffectOnNoteIndex make sense? Because I think that's technically what GetAffectingNoteIndex() would translate to. Haha English sucks.

if there is no "ensure hz" line, then scale note 69 will always be mapped to the frequency 440?

No, thats the problem.

If not, I wonder if it might be more useful to use note 69 by default, however return the associated frequency from the [Exact Tuning] table rather than a default 440hz.

Thats an idea! The idea I was thinking is to use the InitEqual note/frequency, but the huge problem is the fact that the following formulas could totally overwrite the original values set by the InitEqual call.

And another problem would be, does saying the reference note by default is 69 mean anything in regards to the scale the author created? Its best to air on the side of caution and say no. Instead, the API should just say "nope, no reference here :)"

But the code makes this clear with this line in CFormula

Careful, I think that comment is stating it "overwrites everything in the formula." The Reset call there does not effect InitEqual. If I recall correctly, InitEqual can be placed anywhere in the section, so if a reference formula is before, it's ignored!

A test I'd like to have done is read a TUN file with an InitEqual after a reference formula, then save the file, using this library. I'm honestly curious if we found an edge case where the input file != output file.

I personally find it confusing to have two separate commands that define a specific note's frequency, unless I'm missing a nuance between them. But I suppose for compatibility purposes there's not much we can do other than providing alternate functions and making it clear what they do in the documentation.

Yeah thats why I think the separation of concerns of slapping InitEqual in the beginning of the function names is the best we can do without dropping functionality. If the previous edge case I mentioned exists, then whoops, InitEqual becomes more annoying than we thought.

which would be some way to parse a CFormula into a relative cents scale.

May you give me an example of what you mean by "relative cents scale"? I'm not familiar. I see you mention a SCL example, may you send that to me?

For sure, I would be interested in contributing!

Wow, thank you! Yeah as we know, no time pressure! I may have time this next week so I'll see.

For sure, I would be interested in contributing! I do have a separate project I need to prioritize, but I expect I'll be able to invest some time in this within a few weeks.

Also, I did get my extraction method working as described, and you can view the code here and feel free to borrow from it.

Hey @vsicurella, are you familiar with my project Xen MIDI Retuner? I see your Everyone Tuner has a pretty identical goal of trying to solve tuning issues via pitchbend and other methods. I've been working on it since 2019 so its really impressive how fast you've been working on Everyone Tuner! I'd love for you to give Xen MIDI Retuner a look! The main goal of the project is to make tuning as approachable as possible, technically and artistically, so trying to find ways to do polyphony with single channel midi has been tough.

[vsicurella]

@zardini123 Hey, I'm really sorry about my super late reply! Rounding off the next release of my other project happened to take longer than expected, and I forgot to circle back here. I wasn't turned off by your comment or anything!!

Yeah, I have seen Xen MIDI Retuner a little over a year ago but haven't kept up with the changes, so I will check it out again! It might seem like I did Everytone Tuner really fast, but I actually started that as a different project in 2019 and have only revived it recently. I always wanted to make something like it, but the main motivation was I needed octave transposition by MIDI channel to support larger pitch sets for my Lumatone, which is something the current premium tuning plugins don't have.

To be honest, I'll need a little time to refresh my thinking about this stuff, but I'm planning spending time with my project again so I'll try to get back to you again soon.

As a side note, there's a great community on the Xenharmonic Alliance discord server if you are interested in chatting with more folks!

[zardini123]

I wasn't turned off by your comment or anything!!

Haha no problem @vsicurella! As we understand, being this is open source there is no requirement on timing! We have our outside priorities (I certainly do haha)!

I needed octave transposition by MIDI channel to support larger pitch sets for my Lumatone, which is something the current premium tuning plugins don't have.

I honestly have no idea what this means. May you elaborate? Curious to learn!

As a side note, there's a great community on [the Xenharmonic Alliance discord server (https://discord.com/invite/FSF5JFT) if you are interested in chatting with more folks!

Thank you so much! I've been looking for a Xen community! Cool to see there are more of us!