Project is currently on hold as of 12/20/2023, but this is one I'll be coming back to in a little bit.
This repo utilizes SOTA AI voice generation tools such as Tortoise and RVC to generate the audio required to make an audiobook. To my knowledge, Tortoise and RVC combined replicates speech in a way that is currently unparalleled to anything else that exists out there that is open-sourced and able to be ran locally. Eleven labs is absolutely fantastic... one of the best IMO, but it's not "free" and it's not open-source
There are two ways to install this, via Package or Manually. If you don't have any
:heavy_check_mark: Sentence generation using Tortoise -> RVC
:heavy_check_mark: RVC AI Voice model compatibility (V1 & V2 as well as 40k & 48k trained models)
:heavy_check_mark: Generation of an entire text file with some basic sentence parsers and sorters
:heavy_check_mark: Selectively playback sentences by clicking and choosing them
:heavy_check_mark: Selectively regenerate audio for sentences by clicking and choosing them
:heavy_check_mark: Progress saving and continuing for audiobook generation in case of a crash or want to continue later
:heavy_check_mark: Audiobook loading from previous generations
:heavy_check_mark: Export of Audiobooks to a single wave file
:heavy_check_mark: Audiobook can be updated with a new text file in case sentences need to be changed or order adjusted
Make sure you have Tortoise installed and working as stated in prerequisites
Head over to the releases tab and download the audiobook maker 7zip folder from my HuggingFace Repo:
Unzip using 7zip (or your preferred 7zip unpacker)
Inside of the audiobook maker folder, right click and edit tort.yaml with a voice that is working in Tortoise TTS (more details in video)
Run the start_package.bat
Before generating an audio, make sure that Tortoise TTS is also running in the background so that it can generate audio.
Make sure you have Tortoise installed and working as stated in prerequisites
Open a powershell/cmd window, clone, and then cd into the repo:
git clone https://github.com/JarodMica/audiobook_maker.git
cd audiobook_maker
Download and extract rvc to the audiobook_maker folder:
rvc_lightweight
, and then copy the rvc
folder into the audiobook_maker
folder audiobook_maker/rvc
and NOT like audiobook_maker\rvc_lightweight
Set-up and activate virtual environment
python -m venv venv
venv\Scripts\activate
Install pytorch using command below (recommended) or get from https://pytorch.org/get-started/locally/:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
pip install -r rvc/requirements.txt
(if you get an error here, check to make sure you copied the rvc folder correctly)
pip install https://huggingface.co/Jmica/rvc/resolve/main/fairseq-0.12.2-cp310-cp310-win_amd64.whl
pip install git+https://github.com/JarodMica/rvc-tts-pipeline.git@lightweight#egg=rvc_tts_pipe
pip install git+https://github.com/JarodMica/tortoise_api.git
Download and install ffmpeg: https://ffmpeg.org/download.html
Place whatever RVC AI voices (.pth) you want into the voice_models
directory and indexes into voice_indexes
I am able to build these tools thanks to all of the fantastic open source repos out there, borrowing from different projects to get this all frankensteined and hashed together. Without these, it wouldn't be possible for me to have gotten the functionality needed to create such a fantastic tool: