NOT AN ISSUE: Fixed colab notebook to work as of October 14th, 2021

qwertyuu commented 3 years ago

Hello!

Thanks a whole for implementing this, it works wonders.

There has been some time since the notebook was written, and it did not work out of the box anymore. So I went ahead and made this one: https://colab.research.google.com/drive/1EPoNkTHIyYeLq_fZZY5HS1ecjCUQEGvg?usp=sharing

It is still running on my end but it did successfully generate the first utterance. It's very very nice. Thanks again!

vaibhawkhemka commented 3 years ago

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath' WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

loaded model at logs-Tacotron/pretrained/model.ckpt-189500 Hyperparameters: allow_clipping_in_normalization: True attention_dim: 128 attention_filters: 32 attention_kernel: (31,) cleaners: english_cleaners cumulative_weights: True decoder_layers: 2 decoder_lstm_units: 1024 embedding_dim: 512 enc_conv_channels: 512 enc_conv_kernel_size: (5,) enc_conv_num_layers: 3 encoder_lstm_units: 256 fft_size: 1024 fmax: 7600 fmin: 125 frame_shift_ms: None griffin_lim_iters: 60 hop_size: 256 impute_finished: False input_type: raw log_scale_min: -32.23619130191664 mask_encoder: False mask_finished: False max_abs_value: 4.0 max_iters: 2500 min_level_db: -100 num_freq: 513 num_mels: 80 outputs_per_step: 1 postnet_channels: 512 postnet_kernel_size: (5,) postnet_num_layers: 5 power: 1.1 predict_linear: False prenet_layers: [256, 256] quantize_channels: 65536 ref_level_db: 20 rescale: True rescaling_max: 0.999 sample_rate: 22050 signal_normalization: True silence_threshold: 2 smoothing: False stop_at_any: True symmetric_mels: False tacotron_adam_beta1: 0.9 tacotron_adam_beta2: 0.999 tacotron_adam_epsilon: 1e-06 tacotron_batch_size: 16 tacotron_decay_learning_rate: True tacotron_decay_rate: 0.4 tacotron_decay_steps: 50000 tacotron_dropout_rate: 0.5 tacotron_final_learning_rate: 1e-05 tacotron_initial_learning_rate: 0.001 tacotron_reg_weight: 1e-06 tacotron_scale_regularization: True tacotron_start_decay: 50000 tacotron_teacher_forcing_ratio: 1.0 tacotron_zoneout_rate: 0.1 trim_silence: True use_lws: True Constructing model: Tacotron WARNING:tensorflow:From /root/Tacotron-2/tacotron/synthesizer.py:16: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/synthesizer.py:19: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/tacotron.py:52: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/zoneout_LSTM.py:88: The name tf.nn.rnn_cell.LSTMStateTuple is deprecated. Please use tf.compat.v1.nn.rnn_cell.LSTMStateTuple instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:16: conv1d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Use tf.keras.layers.Conv1D instead. WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/layers/convolutional.py:218: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:17: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.BatchNormalization instead. In particular, tf.control_dependencies(tf.GraphKeys.UPDATE_OPS) should not be used (consult the tf.keras.layers.batch_normalization documentation). WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:20: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.dropout instead. WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:82: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version. Instructions for updating: Please use keras.layers.Bidirectional(keras.layers.RNN(cell)), which is equivalent to this API WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/rnn.py:464: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version. Instructions for updating: Please use keras.layers.RNN(cell), which is equivalent to this API WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/zoneout_LSTM.py:145: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/rnn.py:244: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/attention.py:161: The name tf.layers.Conv1D is deprecated. Please use tf.compat.v1.layers.Conv1D instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/attention.py:164: The name tf.layers.Dense is deprecated. Please use tf.compat.v1.layers.Dense instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:146: MultiRNNCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version. Instructions for updating: This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0. Traceback (most recent call last): File "synthesize.py", line 34, in main() File "synthesize.py", line 28, in main tacotron_synthesize(args) File "/root/Tacotron-2/tacotron/synthesize.py", line 81, in tacotron_synthesize run_eval(args, checkpoint_path, output_dir, sentences) File "/root/Tacotron-2/tacotron/synthesize.py", line 14, in run_eval synth.load(checkpoint_path) File "/root/Tacotron-2/tacotron/synthesizer.py", line 24, in load self.model.initialize(inputs, input_lengths) File "/root/Tacotron-2/tacotron/models/tacotron.py", line 112, in initialize CustomDecoder(decoder_cell, self.helper, decoder_init_state), File "/root/Tacotron-2/tacotron/models/custom_decoder.py", line 47, in init if not rnn_cell_impl._like_rnncell(cell): # pylint: disable=protected-access AttributeError: module 'tensorflow.python.ops.rnn_cell_impl' has no attribute '_like_rnncell'

@qwertyuu still giving the above issue after running the notebook. Above issue produced after executing this cell. Waiting for your help...

qwertyuu commented 3 years ago

On it

qwertyuu commented 3 years ago

It is now generating on my end. Can you validate it also works on yours @vaibhawkhemka ?

Thanks for testing :)

vaibhawkhemka commented 3 years ago

@qwertyuu It works!! Great Job!! Btw Curious to know what changes u did??

qwertyuu commented 3 years ago

@vaibhawkhemka I changed 2 things:

The moment I switch the environment from TF 2.x to TF 1.x (seems like there is a timing issue with the dependencies, very weird, did not go in the details as of why)
I forked the Tacotron-2 project and removed the check that used _like_rnncell (the error that you had). Luckily this was just an assertion that could be removed.

Nothing else seemed to cause compatibility problems so it worked. Have fun!

LottieSebes commented 2 years ago

Hi @qwertyuu I was trying to replicate this book on my colab account but at the stage of installing the dependencies I got the following error. It it let me proceed to the next cells so I did. Here is the first error:

ERROR: Could not find a version that satisfies the requirement tensorflow<=1.9.0 (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0, 1.15.2, 1.15.3, 1.15.4, 1.15.5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0, 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0rc0, 2.4.0rc1, 2.4.0rc2, 2.4.0rc3, 2.4.0rc4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.8.0rc0, 2.8.0rc1, 2.8.0) ERROR: No matching distribution found for tensorflow<=1.9.0 |████████████████████████████████| 312 kB 33.8 MB/s |████████████████████████████████| 50 kB 8.2 MB/s ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed. tensorflow 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.2.4 which is incompatible. |████████████████████████████████| 13.8 MB 30.3 MB/s ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed. yellowbrick 1.4 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible. xarray 0.18.2 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible. tensorflow 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.2.4 which is incompatible. tensorflow 2.8.0 requires numpy>=1.20, but you have numpy 1.15.4 which is incompatible. tables 3.7.0 requires numpy>=1.19.0, but you have numpy 1.15.4 which is incompatible. scikit-image 0.18.3 requires numpy>=1.16.5, but you have numpy 1.15.4 which is incompatible. pywavelets 1.2.0 requires numpy>=1.17.3, but you have numpy 1.15.4 which is incompatible. pyerfa 2.0.0.1 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible. pyarrow 6.0.1 requires numpy>=1.16.6, but you have numpy 1.15.4 which is incompatible. plotnine 0.6.0 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible. pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.15.4 which is incompatible. kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.15.4 which is incompatible. jaxlib 0.3.0+cuda11.cudnn805 requires numpy>=1.19, but you have numpy 1.15.4 which is incompatible. jax 0.3.1 requires numpy>=1.19, but you have numpy 1.15.4 which is incompatible. datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible. cupy-cuda111 9.4.0 requires numpy<1.24,>=1.17, but you have numpy 1.15.4 which is incompatible. astropy 4.3.1 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible. albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible. |████████████████████████████████| 421 kB 33.0 MB/s Building wheel for pysptk (setup.py) ... done |████████████████████████████████| 131 kB 30.3 MB/s |████████████████████████████████| 58 kB 7.2 MB/s |████████████████████████████████| 1.5 MB 37.3 MB/s |████████████████████████████████| 53.9 MB 228 kB/s |████████████████████████████████| 228 kB 53.6 MB/s Building wheel for librosa (setup.py) ... done Building wheel for matplotlib (setup.py) ... done ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. yellowbrick 1.4 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible. seaborn 0.11.2 requires matplotlib>=2.2, but you have matplotlib 2.0.2 which is incompatible. scikit-image 0.18.3 requires numpy>=1.16.5, but you have numpy 1.15.4 which is incompatible. pycocotools 2.0.4 requires matplotlib>=2.1.0, but you have matplotlib 2.0.2 which is incompatible. plotnine 0.6.0 requires matplotlib>=3.1.1, but you have matplotlib 2.0.2 which is incompatible. plotnine 0.6.0 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible. mizani 0.6.0 requires matplotlib>=3.1.1, but you have matplotlib 2.0.2 which is incompatible. kapre 0.3.7 requires librosa>=0.7.2, but you have librosa 0.5.1 which is incompatible. kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.15.4 which is incompatible. arviz 0.11.4 requires matplotlib>=3.0, but you have matplotlib 2.0.2 which is incompatible. albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible. |████████████████████████████████| 125 kB 28.0 MB/s |████████████████████████████████| 2.0 MB 68.2 MB/s Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done |████████████████████████████████| 150 kB 66.0 MB/s |████████████████████████████████| 410 kB 67.6 MB/s Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Building wheel for nnmnkwii (PEP 517) ... done Building wheel for pysptk (PEP 517) ... done Building wheel for lws (setup.py) ... done

The next error happened at the 'Mel-spectrogram prediction by Tacoron2' at which point I got the following stacktrace but it still let me continue:

Traceback (most recent call last): File "synthesize.py", line 2, in from tacotron.synthesize import tacotron_synthesize File "/root/Tacotron-2/tacotron/synthesize.py", line 4, in from hparams import hparams, hparams_debug_string File "/root/Tacotron-2/hparams.py", line 1, in import tensorflow as tf File "/tensorflow-1.15.2/python3.7/tensorflow/init.py", line 99, in from tensorflow_core import * File "/tensorflow-1.15.2/python3.7/tensorflow_core/init.py", line 36, in from tensorflow._api.v1 import compat File "/tensorflow-1.15.2/python3.7/tensorflow_core/_api/v1/compat/init.py", line 23, in from tensorflow._api.v1.compat import v1 File "/tensorflow-1.15.2/python3.7/tensorflow_core/_api/v1/compat/v1/init.py", line 672, in from tensorflow_estimator.python.estimator.api._v1 import estimator File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/init.py", line 10, in from tensorflow_estimator._api.v1 import estimator File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/_api/v1/estimator/init.py", line 12, in from tensorflow_estimator._api.v1.estimator import inputs File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/_api/v1/estimator/inputs/init.py", line 10, in from tensorflow_estimator.python.estimator.inputs.numpy_io import numpy_input_fn File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/python/estimator/inputs/numpy_io.py", line 26, in from tensorflow_estimator.python.estimator.inputs.queues import feeding_functions File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/python/estimator/inputs/queues/feeding_functions.py", line 40, in import pandas as pd File "/usr/local/lib/python3.7/dist-packages/pandas/init.py", line 22, in from pandas.compat import ( File "/usr/local/lib/python3.7/dist-packages/pandas/compat/init.py", line 15, in from pandas.compat.numpy import ( File "/usr/local/lib/python3.7/dist-packages/pandas/compat/numpy/init.py", line 7, in from pandas.util.version import Version File "/usr/local/lib/python3.7/dist-packages/pandas/util/init.py", line 1, in from pandas.util._decorators import ( # noqa File "/usr/local/lib/python3.7/dist-packages/pandas/util/_decorators.py", line 14, in from pandas._libs.properties import cache_readonly # noqa File "/usr/local/lib/python3.7/dist-packages/pandas/_libs/init.py", line 13, in from pandas._libs.interval import Interval File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

Then the Waveform synthesis by Wavenet stage failed in the third cell because tacotron_output folder did not exist:

`FileNotFoundError Traceback (most recent call last) in () 2 from tqdm import tqdm 3 ----> 4 with open("../Tacotron-2/tacotron_output/eval/map.txt") as f: 5 maps = f.readlines() 6 maps = list(map(lambda x:x[:-1].split("|"), maps))

FileNotFoundError: [Errno 2] No such file or directory: '../Tacotron-2/tacotron_output/eval/map.txt'`

I suppose the problem lies with the dependencies. Do you have any ideas about how to resolve this? Thanks!

r9y9 / wavenet_vocoder

NOT AN ISSUE: Fixed colab notebook to work as of October 14th, 2021 #216