SortAnon / ControllableTalkNet

A web app that lets you play around with TalkNet models
GNU Affero General Public License v3.0
121 stars 48 forks source link

MIDI Support #12

Open RAYTRAC3R opened 2 years ago

RAYTRAC3R commented 2 years ago

Would it be possible to make the script able to directly read MIDI files to get durations and pitch? It'd be very helpful in cases where you don't have clean vocals, but you have a MIDI based on the vocals.

I've been looking into the code, and it looks like it might be possible if you make it able to read the note durations and pitches in a MIDI and convert it to the proper format, but I'm not a skilled enough coder to do it.

Alternatively, I use a concatenative singing synthesizer called UTAU, and the .ust files made with it seem fairly simple in terms of structure, so it might even be possible to import durations and pitch from it instead. UST files even contain lyrics for each note, so a transcript could be extracted.