fare / projects

Various projects you might work on
5 stars 0 forks source link

Guitar Anti-Hero, a.k.a. Guitar Autotune #1

Open fare opened 5 years ago

fare commented 5 years ago

Guitar Anti-Hero a.k.a. Guitar Autotune is a Phone app (still to be written) that will empower anyone with a sense of groove to play the music they like: tap or strum on your phone, or shake it, and thereby have it correctly play the next note for your song, pitch-perfect, with the nuances you express.

By contrast, Guitar Hero, the opposite, is where you do your best to fake virtuosity on a simplified simulation of playing the correct pitch (the left hand of the guitar) while the machine plays notes without nuance (the right hand of the guitar), and judges how much you stray from the regularity of a soulless metronome (or the exact sense of rhythm of whichever performer recorded the template, if any).

Guitar Hero is no doubt great fun as a game; but it is the exact opposite of what you want as a tool to actually perform music: it reverses the roles of humans and machines with respect to what they are respectively good at. Machines are good at exactly following the score (the left hand of the guitar), whereas humans are good at expressing their feelings via nuances that complement or alter the score (the right hand of the guitar).

Guitar Anti-Hero is meant as a tool for musicians: singers who don't want to spend ten years perfecting their guitar skills just to accompany themselves while singing; performers who want to focus fully on nuance and emotion rather than virtuosity at following a score. You set the rhythm. You set the nuances. The computer plays the correct note.

Of course, this supposes that there is a correct note, a score to follow. Therefore, this software isn't suitable for improvisation or free-form performances. If you are not completely satisfied with the score, you'll have to edit it before the day of the performance, or somehow override it using functionality beyond what Guitar Anti-Hero offers as such. Thus, future variations might make it possible to "branch" on multiple choices or otherwise allow dynamic alterations and reconfiguration—but that's for after Guitar Anti-Hero is completed.

As input, Guitar Anti-Hero accepts a program in a suitable Domain Specific Language (DSL). That program might itself be generated from a midi file or a score file, possibly annotated with additional information to map one or many user strokes to one or many notes. Indeed, at times the user may want to precisely control when each note is played; at times, he may prefer to only give the beat, with the computer following, sometimes with virtuoso fast notes in a single beat, sometimes with notes or pauses spanning many beats.

As output, the software plays the notes of the song and/or of its accompaniment. The software also records the parameters from the user input, so that the performance can be easily replayed identically, and/or with a future enhanced version of the score.

Note that one user strum or tap may correspond to more or fewer than one notes play of accompaniment: the tempo may have to be inferred from the user notes, and the computer notes would follow that tempo as well as other indications by the user (such as velocity or modulation, etc.). The notes played by the computer may or may therefore may or may not fall at the same time as the notes tapped by the user.

I've been talking about this project for many years, but haven't done more than explore a DSL for writing down the music in Clojure. If I make a lot of money, and/or lose my job, I'll fund this project. In the meantime, I'm looking for someone who'd write it out of love.

Potential implementation languages:

JBetz commented 11 months ago

How could the touchscreen of a phone possibly enable nuanced expression without velocity sensitivity let alone MPE? Not to mention the fact that it's impossible to navigate without actually looking at the screen. At least with real guitars you have the physical strings for tangible navigation.

Artiphon's Instrument 1 lets you map chords to a single key press, which is good, and also has physical strumming bridge, which is also good. But the bridge isn't even velocity sensitive, which makes it less expressive than a MIDI keyboard. Which makes it useless.

Open tunings are a better solution for accompaniment. Only one shape for any diatonic chord, so the skill floor is much lower than the abomination that is standard tuning.

I'd be much more interested in a guitar that decouples strings from their tunings, since the worst part about open tuning is having to retune to optimize for different keys.

fare commented 11 months ago

I was thinking the accelerometer might be more useful than the touchscreen, actually. But I have no practical knowledge of programming mobile devices, what are the specs of their sensors, and how to achieve real time feedback.

JBetz commented 11 months ago

Maybe. Though that ends up looking like air guitar and totally loses the cool factor, taking the name anti-hero a bit too far. And electric guitar is very much about looking cool.

image

I think a better use for the accelerometer in the case of accompaniment is to adjust the tempo of the backing track. It wouldn't even require direct contact with the phone if the performer is tapping their foot or otherwise expressing the beat with their body.

fare commented 11 months ago

I was thinking you could set not just the tempo, but also the velocity and maybe also the attack.

deleolajide commented 10 months ago

I am on a similiar journey. Check out https://github.com/jus-Be/orin-ayo