📚 [Want to contribute?] ebook2audiobookxtts roadmap

[DrewThomasson]

All Features open to public Contributions ⭐

• [ ] -h -help parameter info in different languages
• [x] Notebooks Folder Talked about here
• [ ] Make Chinese text splitting not split words and improve pause timing Talked about here
• [ ] Get Kaggel Notebook working (Needs to be giving dropdown menus to interact with the headless version) Talked about here
• [x] Get Working Google Colab Notebook Talked about here

Wanted Extra Parameters

• [ ] Make ebook input parameter accept a list of files for multiple files.
• [ ] Make ebook input parameter accept a folder containing ebook files to auto-run through.
• [ ] OCR for PDF files (as a Parameter) Talked about here
• [ ] Add a force use device (cpu or GPU) (This will force set the device at the top of the script) Talked about here(currently being added by @ROBERT-MCDOWELL) ref here
• [ ] Use Deepfilternet2 to de-noise any reference audio for voice cloning, demo huggingfacespace using it, Talked about here
• [ ] Custom model dir input for pointing to a folder containing all of the custom model files if available instead of having to point to each model file individually
• [ ] Change voices per chapter parameter Talked about here

My Other Repos I Want to Integrate into the App for Extra Options :)

• [ ] Add app parameter that launches the ebook2audiobookpiper-tts GUI.(Piper tts appears to have issues working in ARM (apple silicon)MAC But runs fine in the docker on ARM)
• [ ] Add app parameter that launches the ebook2audiobookstyletts2 GUI.
• [ ] Add app parameter that launches the ebook2audiobook-espeak GUI.
• [ ] Add app parameter that launches the FineTune XTTS GUI.
• [ ] Add app parameter for using Barktts documentation from coqui tts

Create a standard function for load_model() and inference_model() for:

• [ ] ⓍXTTSv2
• [ ] Styletts2
• [ ] 🪈 Piper-tts
• [ ] 🐶 Bark tts

  # Standard functions should be:
  def load_model() - Will load model and download model to load if not available locally
  def inference_model() - Will inference the pre-loaded model

Create Readme in these languages

• [x] English (en)
• [ ] Spanish (es)
• [ ] French (fr)
• [ ] German (de)
• [ ] Italian (it)
• [ ] Portuguese (pt)
• [ ] Polish (pl)
• [ ] Turkish (tr)
• [ ] Russian (ru)
• [ ] Dutch (nl)
• [ ] Czech (cs)
• [ ] Arabic (ar)
• [x] Chinese (zh-cn)
• [ ] Japanese (ja)
• [ ] Hungarian (hu)
• [ ] Korean (ko)

Binary builds Working pyinstaller script for:

• [ ] 🍎 Mac Intel x86
• [ ] 🪟 Windows x86
• [ ] 🐧 Linux x86
• [ ] 🖥️🍏 Apple Silicon Mac
• [ ] 🪟💪 ARM Windows
• [ ] 🐧💪 ARM Linux

🐍 Pip install that works for:

• being overseen by @ROBERT-MCDOWELL
• [ ] 🍎 Mac Intel x86
• [ ] 🪟 Windows x86
• [ ] 🐧 Linux x86
• [ ] 🖥️🍏 Apple Silicon Mac
• [ ] 🪟💪 ARM Windows
• [ ] 🐧💪 ARM Linux

Extra Overkill for training models and such (All supported Coqio tts models and piper-tts in one easy command)

• For info about this @DrewThomasson, he is currently working on the development of this, work-in-progress-repo here
• [ ] Make a easy to use training gui for all coqio tts models in th ljspeech format training recipes here from coqui tts
• More info:
• Bark TTS training repo
• Coqui Training docs
• Coqui training Tutorial for beginners
• Formatting your dataset
• ljspeech dataset generator(Uses whisper for transcription?)
• DeepFilterNet2 for high quality denoising training data The GUI/headless should be as simple as:
• On training piper-tts
• Good news! They all appear to use LJSpeech FORMAT! Even the standard Piper-tts training code!
• Remember tho that the dataset generator thing repo you linked makes the names of the files in the dataset in the metadata.csvfile like this "wavs/1272-128104-0000.wav" But the LJspeech trainers all appear to want it like this "1272-128104-0000", So you should probs make the dataset generating function your making take that into account and have the final metatdata.csv file that it'll use to pass into the many trainer functions.
• I plan on giving this a gradio gui potentially,

  Step 1:

• Select model to fine-tune
• Give input audio file/files
• Press process and create dataset

  Step 2: (Once Process dataset has been completed)

• Change any training parameters/config from standard training config for that selected model
• Press Train model

  Step 3: (once training is complete)

• Load trained model
• Test influencing the trained/fine-tuned model
• Export model & zipped dataset OR Export condensed model & zipped dataset

  That's it! Planned to be simple as that! 🎉

Wanted Auto-testing scripts for development

• [ ] Standard model headless run through every language sample Samples located here

@DrewThomasson if you want to help out at all! 😃

[ROBERT-MCDOWELL]

Another interesting option would be to change voices between chapters. for i.e.: --voice_mapping {"chapters": {1:"john.wav",2:"stella.wav",3:"child.wav",4:"random"} } so the selected chapters will see their voice mapped, others will keep the main --voice intact.

[DrewThomasson]

Another interesting option would be to change voices between chapters. for i.e.: --voice_mapping {"chapters": {1:"john.wav",2:"stella.wav",3:"child.wav",4:"random"} } so the selected chapters will see their voice mapped, others will keep the main --voice intact.

@ROBERT-MCDOWELL Added to roadmap checklist