search

ongyx / putao

Poor man's UTAU
MIT License
11 stars 0 forks source link

Feature request: FoxDot compatibility/tutorial #1

Closed ElonSatoshi closed 3 years ago

ElonSatoshi commented 3 years ago

https://github.com/Qirky/FoxDot https://foxdot.org/

Real time singing synthesis has been done with FoxDot using Espeak and Sinsy: https://github.com/mathigatti/RealTimeSingingSynthesizer

It would be neat to use UTAU with this live coding language.

Also this project is underrated!

ongyx commented 3 years ago

It would be nice to integrate FoxDot so UTAU songs can directly be written in it. putao currently uses a special subset of MML which is much less intuitive. However, putao's song model has to be bridged with foxdot somehow, maybe by rendering directly to a player. I'll definitely look into integrating foxdot :)

ElonSatoshi commented 3 years ago

Come to think of it... Is it even possible to stream UTAU rendering?

And, doesn't putao compile MML into json and then render from the json? Could it be possible to use FoxDot to generate json and feed it into putao?

ongyx commented 3 years ago

It actually is possible to pipe putao's output to FoxDot. Currently, putao renders a note like this:

  1. Load the wav sample from the voicebank
  2. Trim using pydub and apply tune/effects using a Resampler class
  3. Append the rendered note to the final render

So we just need a custom Renderer class that handles the rendered note in step 3. (As long as WAV files can be directly added to the player).

ongyx commented 3 years ago

As for compiling MML to the JSON representation putao uses, the bulk of the work is actually done by the backend module. Since FoxDot scripts are just Python code, no explicit loading needs to be done. We just need to subclass FoxDot's Player so we can add putao's custom commands (such as adding syllables to the notes).

Alternatively, putao could be implemented as an instrument in FoxDot's object model.

ongyx commented 3 years ago

After poking around the FoxDot documentation, it doesn't seem that there is a way to load arbitrary WAV data into a player. There is not enough documentation on how FoxDot works internally, so maybe somebody else with more experience with SuperCollider can help. Otherwise, it would be hard to implement compatibility.

ongyx commented 3 years ago

However, I do plan to overhaul the MML syntax putao uses to be more user-friendly so it is easier to write songs.