search

TensorSpeech / TensorFlowTTS

:stuck_out_tongue_closed_eyes: TensorFlowTTS: Real-Time State-of-the-art Speech Synthesis for Tensorflow 2 (supported including English, French, Korean, Chinese, German and Easy to adapt for other languages)
https://tensorspeech.github.io/TensorFlowTTS/
Apache License 2.0
3.8k stars 810 forks source link

Inference error - Tacotron2 + MB MelGAN #617

Closed CracKCatZ closed 3 years ago

CracKCatZ commented 3 years ago

Hey guy's, when trying to run End-to-End Examples my code errors out like this:

2021-07-10 01:19:57.150145: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-07-10 01:20:03.939021: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-07-10 01:20:03.941066: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2021-07-10 01:20:03.987728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.8GHz coreCount: 68 deviceMemorySize: 10.00GiB deviceMemoryBandwidth: 707.88GiB/s
2021-07-10 01:20:03.988020: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-07-10 01:20:04.511797: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-07-10 01:20:04.511921: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-07-10 01:20:04.588899: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-07-10 01:20:04.624910: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-07-10 01:20:04.911415: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-07-10 01:20:05.147846: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-07-10 01:20:05.152237: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-07-10 01:20:05.152640: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-07-10 01:20:05.154233: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-07-10 01:20:05.156576: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:07:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.8GHz coreCount: 68 deviceMemorySize: 10.00GiB deviceMemoryBandwidth: 707.88GiB/s
2021-07-10 01:20:05.156918: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-07-10 01:20:05.157032: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-07-10 01:20:05.157128: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-07-10 01:20:05.157258: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-07-10 01:20:05.157379: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-07-10 01:20:05.157503: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-07-10 01:20:05.157612: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-07-10 01:20:05.157701: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-07-10 01:20:05.157836: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-07-10 01:20:05.948896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-10 01:20:05.949122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2021-07-10 01:20:05.949374: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2021-07-10 01:20:05.950285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8444 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, 
pci bus id: 0000:07:00.0, compute capability: 8.6)
2021-07-10 01:20:05.951655: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-07-10 01:20:15.976811: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-07-10 01:20:16.684127: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-07-10 01:20:17.641441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-07-10 01:20:17.653311: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-07-10 01:20:20.397962: W tensorflow/core/framework/op_kernel.cc:1763] OP_REQUIRES failed at conv_ops.cc:1106 : Not found: No algorithm worked!
Traceback (most recent call last):
  File "first-method-to-inference-taco-and-mb_melgan.py", line 13, in <module>
    tacotron2 = TFAutoModel.from_pretrained("C://Users//supre//Documents//TensorflowTTS//main_trained_models//trained_tacatron_models//taco.v3//checkpoints//model-2630.h5",
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow_tts\inference\auto_model.py", line 104, in from_pretrained
    model._build()
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow_tts\inference\savable_models.py", line 43, in _build
    self([input_ids, input_lengths, speaker_ids])
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1012, in __call__
    outputs = call_fn(inputs, *args, **kwargs)
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow_tts\inference\savable_models.py", line 37, in call
    return super().inference(input_ids, input_lengths, speaker_ids)
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\def_function.py", line 828, in __call__
    result = self._call(*args, **kwds)
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\def_function.py", line 888, in _call
    return self._stateless_fn(*args, **kwds)
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\function.py", line 2942, in __call__
    return graph_function._call_flat(
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\function.py", line 1918, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\function.py", line 555, in call
    outputs = execute.execute(
  File "C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found:  No algorithm worked!
         [[node encoder/conv_batch_norm/tf_tacotron_conv_batch_norm/conv_._0/conv1d (defined at C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow_tts\models\tacotron2.py:110) ]]
         [[decoder/while/LoopCond/_65/_34]]
  (1) Not found:  No algorithm worked!
         [[node encoder/conv_batch_norm/tf_tacotron_conv_batch_norm/conv_._0/conv1d (defined at C:\Users\supre\anaconda3\envs\tts3-tensorflow\lib\site-packages\tensorflow_tts\models\tacotron2.py:110) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_inference_9464]

Function call stack:
inference -> inference

This is my code:

import numpy as np
import soundfile as sf
import yaml

import tensorflow as tf

from tensorflow_tts.inference import TFAutoModel
from tensorflow_tts.inference import AutoProcessor
from tensorflow_tts.inference import AutoConfig

# initialize tacotron2 model.
tac2_config = AutoConfig.from_pretrained("C://Users//supre//Documents//TensorflowTTS//TensorflowTTS-masters//TensorflowTTS-masters-tacotron2//TensorFlowTTS//examples//tacotron2//conf//tacotron2.v1.yaml")
tacotron2 = TFAutoModel.from_pretrained("C://Users//supre//Documents//TensorflowTTS//main_trained_models//trained_tacatron_models//taco.v3//checkpoints//model-2630.h5",
    config=tac2_config, name="tacotron2")

# initialize mb_melgan model
mb_config = AutoConfig.from_pretrained("./examples/multiband_melgan/conf/multiband_melgan.v1.yaml")
mb_melgan = TFAutoModel.from_pretrained("C://Users//supre//Documents//TensorflowTTS//main_trained_models//trained_multiband_melgan//mb_melgan_v1//checkpoints//generator-20000.h5",
   config=mb_config)

# inference
processor = AutoProcessor.from_pretrained("./tensorflow_tts/processor/pretrained/ljspeech_mapper.json")
ids = processor.text_to_sequence("Voici le texte que j'avais envie d'écrire pour tester le système de synthèse.")
# tacotron2 inference
decoder_output, mel_outputs, stop_token_prediction, alignment_history = tacotron2.inference(
    input_ids=tf.expand_dims(tf.convert_to_tensor(ids, dtype=tf.int32), 0),
    input_lengths=tf.expand_dims(tf.convert_to_tensor(len(ids), dtype=tf.int32), 0),
    speaker_ids=tf.convert_to_tensor([0], dtype=tf.int32),
)
mel_outputs = tf.reshape(mel_outputs, [-1, 80]).numpy()
# melgan inference
audio = mb_melgan.inference(mel_outputs[None, ...])

# save to file
sf.write("./audio_test_fr.wav", audio[0, :, 0], 22050)

This is my setup:

gpu = rtx3080
cuda = 11.0
cudnn = 8.0
tensorflow = 2.4.0
tensorflow-gpu = 2.5.0

Please let me know if there is something wrong with the code or my setup:) Thank's for every help and suggestion in advance :)

neso613 commented 3 years ago

Hi @CracKCatZ try this _, mel_outputs, stop_token_prediction, alignment_history = tacotron2.inference( tf.expand_dims(tf.convert_to_tensor(input_ids, dtype=tf.int32), 0), tf.convert_to_tensor([len(input_ids)], tf.int32), tf.convert_to_tensor([0], dtype=tf.int32) )

CracKCatZ commented 3 years ago

Hey @neso613, I was already able to fix the error by myself. But thank's for your fast response and help:)