search

henk717 / KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!
http://koboldai.com
GNU Affero General Public License v3.0
359 stars 130 forks source link

Fix for message about pad_token_id #467

Closed ebolam closed 11 months ago

ebolam commented 11 months ago

Minor change to remove the warning about "Setting pad_token_id to eos_token_id:{eos_token_id} for open-end generation."

ebolam commented 11 months ago

Also added in fixes for tortoise TTS which in theory speeds up generation by x15.

Requires users running the following to install tortoise and dependencies without messing up transformers:

pip install git+https://github.com/neonbjb/tortoise-tts OmegaConf deepspeed
pip install torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt --no-dependencies

Need experimental features turned on in the UI and turn on generate audio. Should background process the text into ogg files for playback in the UI (was there before, just faster now with better chunking)

henk717 commented 11 months ago

We should not override the PAD id at all times just to get rid of a message, I agree with the idea of suppressing the error by replicating what HF is doing, but we should only do so if the pad token is not present.

ebolam commented 11 months ago

The code should use the tokenizer's pad id always. Is there another source of pad ids?