search

Rayhane-mamah / Tacotron-2

DeepMind's Tacotron-2 Tensorflow implementation
MIT License
2.27k stars 905 forks source link

What's the proper version of tensorflow for this project? #434

Open mindmapper15 opened 5 years ago

mindmapper15 commented 5 years ago

I saw the Dockerfile and it automatically downloads the newest version of Tensorflow. Recently, Tensorflow revealed 2.0 version and it's quiet different from the older version. Could you let me know the proper version of tensorflow to run this project?

kennethnakasone commented 5 years ago

I can confirm that tensorflow-gpu 1.13.1, 1.14, and 1.15rc2 work (at least for training Tacotron). I haven't gotten around to doing wavenet training yet. : \

JasonWei512 commented 5 years ago

Use Tensorflow 1.10.

I trained Wavenet on Tensorflow 1.14, and I got an error when I tried to do inference from checkpoint file. It seemed that some variables were missing from checkpoint file. (#421) I retrained WaveNet with Tensorflow 1.10 and the problem is gone.

kennethnakasone commented 5 years ago

I can't do Wavenet inference with tensorflow 1.14. There's a bug with checkpoint loading. Tensorflow 1.10 works fine for me.

Good to know. Thanks. (Does this mean I have to re-train my tacotron model that was trained on 1.13?)

JasonWei512 commented 5 years ago

I can't do Wavenet inference with tensorflow 1.14. There's a bug with checkpoint loading. Tensorflow 1.10 works fine for me.

Good to know. Thanks. (Does this mean I have to re-train my tacotron model that was trained on 1.13?)

You don't have to. My tacotron model was trained with TF 1.14, and now I'm training WaveNet with it on TF 1.10. I see no problem so far.

kennethnakasone commented 5 years ago

You don't have to. My tacotron model was trained with TF 1.14, and now I'm training WaveNet with it on TF 1.10. I see no problem so far.

Awesome! (I was worried I was going to have to re-do a lot of it). Thanks for the info

mindmapper15 commented 5 years ago

I can't do Wavenet inference with tensorflow 1.14. There's a bug with checkpoint loading. Tensorflow 1.10 works fine for me.

Thank's for the info! I'll have to check the newest as possible version of Tensorflow for WaveNet...

orascheg commented 4 years ago

Can it be that at saving of the checkpoints at WaveNet training the eval model (5,3M) of Wavenet is used instead the synth model (3,2M)? I am also experiencing this problem (tensorflow 1.14 GPU) and have put the following line into the function create_shadow_saver() in wavenet_vocoder/train.py log('Shadow variables {}'.format(shadow_variables)) When comparing the output of the shadow variables, I find that they are different if I start the training and the synthesis. Is it making sense to initialize the sh_saver directly after creating the training model (model, stats = model_train_mode(args, feeder, hparams, global_step)) and before the creation of the eval model (eval_model = model_test_mode(args, feeder, hparams, global_step))? I am not fully into the wirings of tensorflow, however I could imagine that the saver uses the latest model created. Perhaps this was different in tf 1.10

orascheg commented 4 years ago

Actually I have tried it. In the original check out from git (12th December 2019 head master ab5cb08a931fc842d3892ebeb27c8b8734ddd4b8) the output of the shadow variables with command line "python3 train.py --model='WaveNet'" was: 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/input_convolution/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/input_convolution/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_causal_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_causal_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_skip_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_skip_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_out_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_out_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_causal_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_causal_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_cin_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_cin_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_skip_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_skip_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_out_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_out_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_causal_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_causal_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_cin_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_cin_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_skip_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_skip_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_out_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_out_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_causal_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_causal_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_cin_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_cin_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_skip_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_skip_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_out_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_out_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_causal_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_causal_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_cin_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_cin_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_skip_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_skip_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_out_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_out_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_causal_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_causal_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_cin_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_cin_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_skip_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_skip_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_out_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_out_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_causal_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_causal_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_cin_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_cin_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_skip_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_skip_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_out_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_out_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_causal_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_causal_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_cin_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_cin_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_skip_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_skip_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_out_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_out_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_causal_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_causal_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_cin_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_cin_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_skip_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_skip_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_out_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_out_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_causal_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_causal_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_cin_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_cin_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_skip_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_skip_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_out_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_out_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_causal_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_causal_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_cin_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_cin_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_skip_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_skip_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_out_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_out_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_causal_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_causal_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_cin_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_cin_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_skip_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_skip_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_out_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_out_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_causal_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_causal_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_cin_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_cin_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_skip_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_skip_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_out_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_out_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_causal_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_causal_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_cin_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_cin_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_skip_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_skip_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_out_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_out_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_causal_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_causal_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_cin_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_cin_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_skip_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_skip_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_out_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_out_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_causal_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_causal_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_cin_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_cin_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_skip_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_skip_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_out_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_out_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_causal_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_causal_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_cin_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_cin_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_skip_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_skip_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_out_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_out_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_causal_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_causal_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_cin_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_cin_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_skip_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_skip_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_out_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_out_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_causal_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_causal_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_cin_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_cin_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_skip_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_skip_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_out_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_out_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_causal_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_causal_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_cin_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_cin_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_skip_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_skip_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_out_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_out_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_2/bias/ExponentialMovingAverage'

When starting synthsis with the command line "python3 synthesize.py --model='WaveNet' --wavenet_name='WaveNet'" the output was: 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/SubPixelConvolution_layer_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/input_convolution/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/input_convolution/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_skip_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_skip_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_out_conv_ResidualConv1DGLU_0/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_out_conv_ResidualConv1DGLU_0/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_cin_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_cin_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_skip_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_skip_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_out_conv_ResidualConv1DGLU_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_1/residual_block_out_conv_ResidualConv1DGLU_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_cin_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_cin_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_skip_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_skip_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_out_conv_ResidualConv1DGLU_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_2/residual_block_out_conv_ResidualConv1DGLU_2/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_cin_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_cin_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_skip_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_skip_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_out_conv_ResidualConv1DGLU_3/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_3/residual_block_out_conv_ResidualConv1DGLU_3/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_cin_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_cin_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_skip_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_skip_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_out_conv_ResidualConv1DGLU_4/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_4/residual_block_out_conv_ResidualConv1DGLU_4/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_cin_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_cin_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_skip_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_skip_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_out_conv_ResidualConv1DGLU_5/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_5/residual_block_out_conv_ResidualConv1DGLU_5/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_cin_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_cin_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_skip_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_skip_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_out_conv_ResidualConv1DGLU_6/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_6/residual_block_out_conv_ResidualConv1DGLU_6/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_cin_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_cin_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_skip_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_skip_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_out_conv_ResidualConv1DGLU_7/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_7/residual_block_out_conv_ResidualConv1DGLU_7/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_cin_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_cin_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_skip_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_skip_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_out_conv_ResidualConv1DGLU_8/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_8/residual_block_out_conv_ResidualConv1DGLU_8/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_cin_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_cin_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_skip_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_skip_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_out_conv_ResidualConv1DGLU_9/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_9/residual_block_out_conv_ResidualConv1DGLU_9/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_cin_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_cin_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_skip_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_skip_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_out_conv_ResidualConv1DGLU_10/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_10/residual_block_out_conv_ResidualConv1DGLU_10/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_cin_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_cin_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_skip_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_skip_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_out_conv_ResidualConv1DGLU_11/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_11/residual_block_out_conv_ResidualConv1DGLU_11/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_cin_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_cin_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_skip_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_skip_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_out_conv_ResidualConv1DGLU_12/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_12/residual_block_out_conv_ResidualConv1DGLU_12/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_cin_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_cin_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_skip_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_skip_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_out_conv_ResidualConv1DGLU_13/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_13/residual_block_out_conv_ResidualConv1DGLU_13/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_cin_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_cin_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_skip_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_skip_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_out_conv_ResidualConv1DGLU_14/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_14/residual_block_out_conv_ResidualConv1DGLU_14/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_cin_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_cin_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_skip_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_skip_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_out_conv_ResidualConv1DGLU_15/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_15/residual_block_out_conv_ResidualConv1DGLU_15/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_cin_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_cin_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_skip_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_skip_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_out_conv_ResidualConv1DGLU_16/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_16/residual_block_out_conv_ResidualConv1DGLU_16/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_cin_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_cin_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_skip_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_skip_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_out_conv_ResidualConv1DGLU_17/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_17/residual_block_out_conv_ResidualConv1DGLU_17/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_cin_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_cin_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_skip_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_skip_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_out_conv_ResidualConv1DGLU_18/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_18/residual_block_out_conv_ResidualConv1DGLU_18/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_cin_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_cin_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_skip_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_skip_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_out_conv_ResidualConv1DGLU_19/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_19/residual_block_out_conv_ResidualConv1DGLU_19/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_1/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_1/bias/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_2/kernel/ExponentialMovingAverage', 'WaveNet_model/WaveNet_model/inference/final_convolution_2/bias/ExponentialMovingAverage'

After moving the creation of the writer the output is the same as in the synthesize:

#Set up model
global_step = tf.Variable(0, name='global_step', trainable=False)
model, stats = model_train_mode(args, feeder, hparams, global_step)
#GO 020120 moved so that it is created before the eval model
sh_saver = create_shadow_saver(model, global_step)
eval_model = model_test_mode(args, feeder, hparams, global_step)

I am just doing additional training and will then try if it loads without the mentioned error. Strange though because the model is given as parameter to create the shadow saver, thus I assume it is some side effect of something in tensorflow 1.14.

orascheg commented 4 years ago

Further investigation shows that the difference is: Synthesize: WaveNet_model/WaveNet_model/inference/ResidualConv1DGLU_0/residual_block_causal_conv_ResidualConv1DGLU_0 Train: WaveNet_model/WaveNet_model/inference/residual_block_causal_conv_ResidualConv1DGLU_0

Somehow the names differ. I am still debugging where this comes from... Must be somewhere in models/wavenet.py constructor:

    #Residual Blocks
    self.residual_layers = []
    for layer in range(hparams.layers):
        self.residual_layers.append(ResidualConv1DGLU(
        hparams.residual_channels, hparams.gate_channels,
        kernel_size=hparams.kernel_size,
        skip_out_channels=hparams.skip_out_channels,
        use_bias=hparams.use_bias,
        dilation_rate=2**(layer % layers_per_stack),
        dropout=hparams.wavenet_dropout,
        cin_channels=hparams.cin_channels,
        gin_channels=hparams.gin_channels,
        weight_normalization=hparams.wavenet_weight_normalization, 
        init=init, 
        init_scale=hparams.wavenet_init_scale,
        residual_legacy=hparams.residual_legacy,
        name='ResidualConv1DGLU_{}'.format(layer)))
Arafat4341 commented 4 years ago

Hello everyone! Can someone please tell me why am I getting this IndexError?

Traceback (most recent call last): File "train.py", line 138, in main() File "train.py", line 132, in main train(args, log_dir, hparams) File "train.py", line 52, in train checkpoint = tacotron_train(args, log_dir, hparams) File "/content/drive/My Drive/Tacotron-2/tacotron/train.py", line 399, in tacotron_train return train(log_dir, args, hparams) File "/content/drive/My Drive/Tacotron-2/tacotron/train.py", line 152, in train feeder = Feeder(coord, input_path, hparams) File "/content/drive/My Drive/Tacotron-2/tacotron/feeder.py", line 33, in init hours = sum([int(x[4]) for x in self._metadata]) frame_shift_ms / (3600) File "/content/drive/My Drive/Tacotron-2/tacotron/feeder.py", line 33, in hours = sum([int(x[4]) for x in self._metadata]) frame_shift_ms / (3600) IndexError: list index out of range

Why list index would be out of range I don't understand! I checked the size of self._metadata. I don't know what's going on! Kindly help if you find the reason! Thanks in advance!