search

KoljaB / RealtimeTTS

Converts text to speech in realtime
1.78k stars 159 forks source link

KeyError: 'speaker_embedding' and TypeError: Log._log() got an unexpected keyword argument 'exc_info' #7

Open yipy0005 opened 10 months ago

yipy0005 commented 10 months ago

Hi, I encountered the following error, could someone help me?

Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] on linux Type "help", "copyright", "credits" or "license" for more information.

from RealtimeTTS import TextToAudioStream, CoquiEngine engine = CoquiEngine() tts_models/multilingual/multi-dataset/xtts_v2 is already downloaded. /home/yipyewmun/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2 Using model: xtts

Process Process-1: Traceback (most recent call last): File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/site-packages/RealtimeTTS/engines/coqui_engine.py", line 149, in _synthesize_worker gpt_cond_latent, speaker_embedding = get_conditioning_latents(cloning_reference_wav) File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/site-packages/RealtimeTTS/engines/coqui_engine.py", line 92, in get_conditioning_latents speaker_embedding = (torch.tensor(latents["speaker_embedding"]).unsqueeze(0).unsqueeze(-1)) KeyError: 'speaker_embedding'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, self._kwargs) File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/site-packages/RealtimeTTS/engines/coqui_engine.py", line 152, in _synthesize_worker logging.exception(f"Error initializing main faster_whisper transcription model: {e}") File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/logging/init.py", line 2113, in exception error(msg, *args, exc_info=exc_info, *kwargs) File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/logging/init.py", line 2105, in error root.error(msg, args, kwargs) File "/home/yipyewmun/miniforge3/envs/realtimeTTS/lib/python3.10/logging/init.py", line 1506, in error self._log(ERROR, msg, args, **kwargs) TypeError: Log._log() got an unexpected keyword argument 'exc_info'

KoljaB commented 10 months ago

Did you use a cloning_reference_wav in the CoquiEngine constructor?

The error is kind of strange. It happens in these lines of the code:

with open(filename_json, "r") as new_file:
    latents = json.load(new_file)

speaker_embedding = (torch.tensor(latents["speaker_embedding"]).unsqueeze(0).unsqueeze(-1))

No rocket science here, we only load a dictionary from a defined file (the generated json) and then access a member. So currently it looks like a corrupt json file (not containing the "speaker_embedding" key, where it clearly should). Can you open the .json file and look if the "speaker_embedding" key is present there? What happens if you copy the female.json from the test directory and use the cloning_reference_wav = "female.wav"?