cardonabits / haxo-rs

Software for the haxophone
MIT License
41 stars 10 forks source link

Notemap - format to allow MIDI Pitch Bend and multiphonics for alternate fingerings #46

Open Ekolide opened 2 weeks ago

Ekolide commented 2 weeks ago

Hi!

Given that one of goals of the Haxophone is to offer fingerings that are the same as an acoustic saxophone, supporting non-standard fingerings could provide an increased capacity for learning the saxophone, increase potential for expressiveness, and decrease the number of "illegal" fingerings -- that are currently silent -- on the Haxophone. This might increase the avenues of learning, expression, and fun using the Haxophone.

Currently, a notemap entry looks like:

"5055632": 60,

Where the first value is the hardware scan code and the second value is the resulting MIDI Note value.

I suggest the following notemap format in CSV:

5055632, 60, 2

Where the first column is the hardware scan code, the second column is the resulting MIDI Note value, and the third column -- in this case 2 -- is the change in cents relative to the base MIDI Note. The format could also follow a range of -1 to +1, meaning that 2 cents = 0.02 and 20 cents = 0.2 and -20 cents = -0.2, possibly resulting in fine details like 16.5 cents = 0.165.

The point of this format being that the MIDI Note values are in their own column as a preparation for multiphonic fingerings on the Haxophone.

The purpose of this is that many alternate fingerings on the saxophone result in microtones. On the Haxophone, they simply don't make a sound, instead resulting in silence. These alternate fingerings are not only useful for expression, but they offer a learning experience when hitting alternate keys than those you were intending. Instead of the Haxophone being silent, literally limiting your sound in a way that the same fingering wouldn't on an acoustic saxophone, it gives you an experience of "what happened?", which can result in an exploration phase of "what happens if I do ...?", engaging players with experience, information, and sounds they can use on their Haxophone as well as their acoustic saxophone.

Now, onto multiphonics...

Some fingerings on the saxophone produce more than one sounding pitch at once, called multiphonics. To support these, polyphony would need to be managed by the Haxophone software, but also be configurable in the notemap.

Here's what a fingering that produces more than one MIDI Note value as well as Pitch Bend might look like in the notemap:

5055632, 13 60 82, 0

Where the first column is the hardware scan code, the second column are three different MIDI Notes, and the third column are their respective changes in cents to their base MIDI Note.

Apart from the notemap format, the Haxophone would need to manage its polyphony. Either setting polyphony settings in real-time, or setting them at a sensible max during startup. I don't think I've heard or seen a multiphonic fingering resulting in more than, say four or five, distinct notes.

Of course the complexity increases with multiphonics, where each of the tones produced can be pitched in different amounts and directions relative to each other. For example, one tone may be sufficiently close to a typical fingered pitch, but one of the multiphonic tones is a quarter-step flat and one is a bit sharp. Since MIDI only supports one Pitch Bend command without proper MPE -- where the different notes are being sent on different MIDI channels -- that is to my knowledge not possible without implementing MPE.

Here's what a fingering with multiple MIDI Note values as well as a corresponding Pitch Bend value per Note might look like:

5055632, 13 60 82, 0 13 -16

Unfortunately for us, the multiphonic tones aren't often equal in volume. Oops. Here's what a notemap entry for a fingering with multiple MIDI Note values, with corresponding Pitch Bend messages, followed by relative velocity information per Note:

5055632, 13 60 82, 0 13 -16, -14 -8 +14

Assuming that MPE is the way to go for this solution, it would offer a way to send this message.

Though this creates some problems. For example, should a note be sounding at Velocity 14 simply because you fingered the note without putting any air pressure on the sensor? That doesn't sound right. And what about when you max out the pressure sensor, then the relative volumes of the tones will be skewed as you approach the top 14 values -- given that the note set at relative +14 velocity will max out even when there's pressure left to give the sensor for the highest dynamics.

Therefore, how about we limit the relative velocity information to only negative values? Or at least the stock fingerings? This way we avoid most of the problems described. The previous example could look like:

5055632, 13 60 82, 0 13 -16, -24 -22 0

This would mean that, at velocity 7, only the third pitch will be sounding and it will be sounding at velocity 7. At velocity 23, the second pitch is sounding at velocity 1 while the third pitch is sounding as velocity 23. At velocity 72, the first pitch is sounding at velocity 48, etc. Making sure that fingerings are centered around your breath, to have an even volume across different fingerings.

Let me know if there are any thoughts, questions, or ideas!

jcard0na commented 2 weeks ago

Hi @Ekolide,

Thanks for sharing your ideas on multiphonics. Some thought below:

  1. Given that this is a large scope feature request and not a problem with the existing code, I would close this issue and move this discussion to Ideas
  2. Also, given that this would require breaking changes to notemap.json, I would only consider merging this feature onto the main branch (when/if implemented) if there was good support from the community of haxophonists (for instance, like we did with transposition)
  3. Moving from json to CSV would be like a step back in time. I don't really see why you could not implement this using json, which is better at handling structured data and easier to parse.
  4. If you want to go ahead, my recommendation would be to implement this in a forked repo. That repo should also include detailed testing instructions, as this would be an entire new instrument, quite different from a regular saxophone.

Thanks!

Ekolide commented 3 days ago

Hi and sorry for a late reply.

The suggestion is primarily aimed at getting microtones playable, in a format that is later ready for multiphonics.

  1. Is the haxo-rs discussion page the right space? Considering it's the least active of the discussion pages, and that you are posting announcements and updates on the haxo-hw discussion pages, there might be more suitable? At the same time, it really isn't hardware related.
  2. What type of support are you looking for, and what type of support are you expecting?
  3. I mostly wanted to make an example that was correctly formatted since I don't know JSON.
  4. My point is that this would bring the haxophone closer to the actual function of a saxophone. By implementing more of the fingerings that make any sound on a saxophone, we can get closer, not further away.

Thanks!