Mozer / wav2lip_extension

21 stars 5 forks source link

can't get it to work #5

Closed Wuzzooy closed 5 months ago

Wuzzooy commented 5 months ago

Hello, i tried everything that i could think of to make it work but didn't manage to and can't find any clue. The module initialize without error but when i try to generate a video, i don't see any api request and i don't hear the audio from xtss even though the audio has been generated.

When i try the live stream option, i get one api request which show an endless video without audio and when i try again i see no request and no video anymore. The audio output from tts is in the sillytavern extra folder as said in the tutorial and when i check the out.wav file it confirm the audio has been generated. I've tried with cuda option and cpu. I don't see any error or any verbose output from wav2lip in the terminal. I've tried on firefox/chrome/edge.

I've tried wav2lip as a standalone and it works.

I'm on windows 10

Mozer commented 5 months ago

Check browser console in Chrome. F12 - Console. it will show you if there are any js errors. You can paste it here. The generation of wav2lip is triggered when TTS finishes it's generation. Is xttsv2 working for you without wav2lip? Maybe Silly tavern got an update and broke my code. What version are you using?

upd: ST got a recent updatе to 1.11.2, I will check if it is working there.

Wuzzooy commented 5 months ago

Yes the TTS works and i can hear it when i disable the module and when i enable it, i can't hear the audio but it does generate it. I've tried on the last staging version and the last main version 1.11.2 I don't see error in the console.

ll

Mozer commented 5 months ago

I am investigating. My patcher is not patching one line in this new version of ST for some reason. As a workaround just manually update lines 459-462 function completeTtsJob() in \SillyTavern\public\scripts\extensions\tts\index.js into this:

function completeTtsJob() {
    console.info(`Current TTS job for ${currentTtsJob?.name} completed.`);
    currentTtsJob = null;
    if (extension_settings.wav2lip !== undefined && extension_settings.wav2lip.enabled && wav2lipIsGeneratingNow) wav2lipMain("text", 0, extension_settings.wav2lip.char_folder, extension_settings.wav2lip.device);
}

I will update the patcher as soon as i understand what's broken.

Mozer commented 5 months ago

I updated the patcher file patch_silly_tavern.py In a new version 1.11.2 of ST they changed new line character from \n to \r\n so it couldn't find one line to patch. Redownload file patch_silly_tavern.py and then double click first restore_silly_tavern.py and then patch_silly_tavern.py it will automatically patch needed files (\tts\index.js)

Wuzzooy commented 5 months ago

Thank you very much, it works now. I'm closing the issue.