synthesize.py fails with dimension error

[indam23]

Tacotron-2 training completed and logs look fine, but synthesis with python3 syntehsize.py --model="Tacotron-2" --mode="eval" --GTA="False" fails with ValueError: Dimension size must be evenly divisible by 4 but is 1 Number of ways to split should evenly divide the split dimension for 'model/split' (op: 'Split') with input shapes: [], [1] and with computed input tensors: input[0] = <0>. This is the same error seen during Wavenet training as described in issue #1. Full traceback:

Using TensorFlow backend.
Running End-to-End TTS Evaluation. Model: Tacotron-2
Synthesizing mel-spectrograms from text..
loaded model at logs-Tacotron-2/taco_pretrained/tacotron_model.ckpt-120000
Hyperparameters:
  allow_clipping_in_normalization: True
  attention_dim: 128
  attention_filters: 32
  attention_kernel: (31,)
  cin_channels: 80
  cleaners: english_cleaners
  clip_mels_length: False
  cross_entropy_pos_weight: 20
  cumulative_weights: True
  decoder_layers: 2
  decoder_lstm_units: 1024
  embedding_dim: 512
  enc_conv_channels: 512
  enc_conv_kernel_size: (5,)
  enc_conv_num_layers: 3
  encoder_lstm_units: 256
  fmax: 7600
  fmin: 25
  frame_shift_ms: None
  freq_axis_kernel_size: 3
  gate_channels: 512
  gin_channels: -1
  gpu_start_idx: 0
  griffin_lim_iters: 60
  hop_size: 256
  input_type: raw
  kernel_size: 3
  layers: 24
  log_scale_min: -32.23619130191664
  mask_decoder: True
  mask_encoder: True
  max_abs_value: 4.0
  max_iters: 2500
  max_mel_frames: 900
  max_time_sec: None
  max_time_steps: 5000
  min_level_db: -100
  n_fft: 1024
  natural_eval: False
  num_freq: 513
  num_gpus: 4
  num_mels: 80
  out_channels: 30
  outputs_per_step: 2
  postnet_channels: 512
  postnet_kernel_size: (5,)
  postnet_num_layers: 5
  power: 1.2
  predict_linear: False
  prenet_layers: [256, 256]
  quantize_channels: 65536
  ref_level_db: 20
  rescale: True
  rescaling_max: 0.999
  residual_channels: 512
  sample_rate: 22050
  signal_normalization: True
  silence_threshold: 2
  skip_out_channels: 256
  smoothing: False
  stacks: 4
  stop_at_any: True
  symmetric_mels: True
  tacotron_adam_beta1: 0.9
  tacotron_adam_beta2: 0.999
  tacotron_adam_epsilon: 1e-06
  tacotron_batch_size: 48
  tacotron_data_random_state: 1234
  tacotron_decay_learning_rate: True
  tacotron_decay_rate: 0.2
  tacotron_decay_steps: 40000
  tacotron_dropout_rate: 0.5
  tacotron_final_learning_rate: 1e-05
  tacotron_initial_learning_rate: 0.001
  tacotron_random_seed: 5339
  tacotron_reg_weight: 1e-06
  tacotron_scale_regularization: True
  tacotron_start_decay: 50000
  tacotron_swap_with_cpu: False
  tacotron_teacher_forcing_decay_alpha: 0.0
  tacotron_teacher_forcing_decay_steps: 280000
  tacotron_teacher_forcing_final_ratio: 0.0
  tacotron_teacher_forcing_init_ratio: 1.0
  tacotron_teacher_forcing_mode: constant
  tacotron_teacher_forcing_ratio: 1.0
  tacotron_teacher_forcing_start_decay: 10000
  tacotron_test_batches: 96
  tacotron_test_size: None
  tacotron_zoneout_rate: 0.1
  train_with_GTA: False
  trim_fft_size: 512
  trim_hop_size: 128
  trim_silence: True
  trim_top_db: 60
  upsample_conditional_features: True
  upsample_scales: [16, 16]
  use_all_gpus: True
  use_bias: True
  use_lws: True
  wavenet_adam_beta1: 0.9
  wavenet_adam_beta2: 0.999
  wavenet_adam_epsilon: 1e-06
  wavenet_batch_size: 4
  wavenet_data_random_state: 1234
  wavenet_dropout: 0.05
  wavenet_ema_decay: 0.9999
  wavenet_learning_rate: 0.0001
  wavenet_random_seed: 5339
  wavenet_swap_with_cpu: False
  wavenet_test_batches: None
  wavenet_test_size: 0.0441
  win_size: None
Constructing model: Tacotron
Tensor("inputs_1:0", shape=(1, ?), dtype=int32)
Tensor("input_lengths:0", shape=(1,), dtype=int32)
DEBUG:  Tensor("input_lengths_1:0", shape=(1,), dtype=int32) 4
Traceback (most recent call last):
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1626, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension size must be evenly divisible by 4 but is 1
    Number of ways to split should evenly divide the split dimension for 'model/split' (op: 'Split') with input shapes: [], [1] and with computed input tensors: input[0] = <0>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "synthesize.py", line 91, in <module>
    main()
  File "synthesize.py", line 85, in main
    synthesize(args, hparams, taco_checkpoint, wave_checkpoint, sentences)
  File "synthesize.py", line 32, in synthesize
    wavenet_in_dir = tacotron_synthesize(args, hparams, taco_checkpoint, sentences)
  File "/home/ctext/Desktop/tacotron2-multiGPU/Tacotron-2/tacotron/synthesize.py", line 150, in tacotron_synthesize
    return run_eval(args, checkpoint_path, output_dir, hparams, sentences)
  File "/home/ctext/Desktop/tacotron2-multiGPU/Tacotron-2/tacotron/synthesize.py", line 57, in run_eval
    synth.load(checkpoint_path, hparams)
  File "/home/ctext/Desktop/tacotron2-multiGPU/Tacotron-2/tacotron/synthesizer.py", line 30, in load
    self.model.initialize(inputs, input_lengths)
  File "/home/ctext/Desktop/tacotron2-multiGPU/Tacotron-2/tacotron/models/tacotron.py", line 62, in initialize
    tower_input_lengths = tf.split(input_lengths, num_or_size_splits=hp.num_gpus, axis=0)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1326, in split
    axis=axis, num_split=num_or_size_splits, value=value, name=name)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 7897, in split
    "Split", split_dim=axis, value=value, num_split=num_split, name=name)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3272, in create_op
    op_def=op_def)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1790, in __init__
    control_input_ops)
  File "/home/ctext/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1629, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimension size must be evenly divisible by 4 but is 1
    Number of ways to split should evenly divide the split dimension for 'model/split' (op: 'Split') with input shapes: [], [1] and with computed input tensors: input[0] = <0>.

_Originally posted by @melindaloubser1 in https://github.com/MlWoo/Tacotron-2/issue_comments#issuecomment-425809232_

[carlfm01]

Hello @melindaloubser1 did you managed to fix the issue?

[indam23]

Hey @carlfm01 - I haven't worked on that project in over a year, to be honest I can't remember what the latest was on this issue.