Mozer / talk-llama-fast

Port of OpenAI's Whisper model in C/C++ with xtts and wav2lip
MIT License
708 stars 64 forks source link

the video window may close unexpectedly in the middle of a sentence #32

Open HEYHEYHEEEEY opened 1 month ago

HEYHEYHEEEEY commented 1 month ago

Hello.

Sometimes the video window may close unexpectedly in the middle of a sentence when the assistant is saying something. If you start talking yourself, so that the assistant begins to regenerate the text and voice it, the window will appear again.

I found out that this is a bug in silly_extras.bat. It wants to open a wav file, but can't. Any ideas why?

cv2: missing video frame 
Exception in thread Thread-4 (wav2lip_server_play_init): Traceback (most recent call last): 
    File "D:\miniconda\envs\extras\Lib\threading.py", line 1045, in _bootstrap_inner 
        self.run() 
    File "D:\miniconda\envs\extras\Lib\threading.py", line 982, in run 
        self._target(*self._args, *self._kwargs) 
    File "D:\talk-llama-fast\xtts\SillyTavern-Extras\modules/wav2lip\server_wav2lip.py", line 342, in wav2lip_server_play_init 
    next_video_chunk_global = play_video_with_audio(video_file_path, audio_file_path, True, next_video_chunk_global, rand_caption) 

    File "D:\talk-llama-fast\xtts\SillyTavern-Extras\modules/wav2lip\server_wav2lip.py", line 475, in play_video_with_audio 
    wf = wave.open(audio_file, 'rb') # Open the next audio file 

    File "D:\miniconda\envs\extras\Lib\wave.py", line 631, in open
        return Wave_read(f)
    File "D:\miniconda\envs\extras\Lib\wave.py", line 283, in __init__ 
        self.initfp(f) 
    File "D:\miniconda\envs\extras\lib\wave.py", line 274, in initfp 
    raise Error('fmt chunk and/or data chunk missing') wave.Error: fmt chunk and/or data chunk missing 1718026936.074945 in wav2lip gen server chunk:23_2
in wav2lip_server_generate: is busy: 0, face_detect_running: 0, chunk: 23, chunk needed: 23, reply: 2