Problem while running train2.py

[vivek-sethia]

After running python train2.py default default I ran into an error. I have changed the path of dataset in hparams.py under class Train2 like this data_path = '{}/arctic/bdl/*.wav'.format(data_path_base)

`Model loaded. mode: train2, model_name: epoch_10_step_10 Traceback (most recent call last):
File "train2.py", line 98, in train(logdir1=logdir1, logdir2=logdir2,queue=False) File "train2.py", line 59, in train summ, gs = sess.run([summ_op, global_step]) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run run_metadata_ptr) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1128, in _run feed_dict_tensor, options, run_metadata) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run options, run_metadata) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [32,?,40] [[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[32,?,40], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op u'Placeholder', defined at: File "train2.py", line 98, in train(logdir1=logdir1, logdir2=logdir2,queue=False) File "train2.py", line 18, in train model = Model(mode="train2", batch_size=hp.Train2.batch_size, queue=queue) File "/home/ideabay/../deep-voice-conversion/models.py", line 22, in init self.x_mfcc, self.y_ppgs, self.y_spec, self.y_mel, self.num_batch = self.get_input(mode, batch_size, queue) File "/home/ideabay../deep-voice-conversion/models.py", line 43, in get_input x_mfcc = tf.placeholder(tf.float32, shape=(batch_size, None, hp_default.n_mfcc)) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1680, in placeholder return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3141, in _placeholder "Placeholder", dtype=dtype, shape=shape, name=name) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3160, in create_op op_def=op_def) File "/home/ideabay/anaconda3/envs/deepvoice/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1625, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [32,?,40] [[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[32,?,40], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]] `

[vivek-sethia]

What could be possibly my mistake? Any help would be appreciated.

[VictoriaBentell]

Oh, I ran into this error. In hparams.py under class Train2, set your batch size to 16 instead of 32.

[vivek-sethia]

@VictoriaBentell I did that now but I have the same issue ?

[vivek-sethia]

@VictoriaBentell Is there anything else that needs to be changed?

[VictoriaBentell]

How many gigabytes of memory do you have on your graphics card? Also can you post your hparams.py?

[vivek-sethia]

I am working on CPU with less epochs as of now. Once my code works I will move this to GPU.

Here is my hparams.py

# -*- coding: utf-8 -*-
#/usr/bin/python2

# path
## local
data_path_base = './datasets'
logdir_path = './logdir'

## remote
# data_path_base = '/data/private/vc/datasets'
# logdir_path = '/data/private/vc/logdir'

class Default:
    # signal processing
    sr = 16000 # Sampling rate.
    frame_shift = 0.005  # seconds
    frame_length = 0.025  # seconds
    n_fft = 512

    hop_length = int(sr*frame_shift)  # 80 samples.  This is dependent on the frame_shift.
    win_length = int(sr*frame_length)  # 400 samples. This is dependent on the frame_length.

    preemphasis = 0.97
    n_mfcc = 40
    n_iter = 60 # Number of inversion iterations
    n_mels = 80
    duration = 2

    # mean_log_spec = -4.25
    # std_log_spec = 2.15
    # min_log_spec = -21.25
    # max_log_spec = 3.0

    # model
    hidden_units = 256  # alias = E
    num_banks = 16
    num_highway_blocks = 4
    norm_type = 'ins'  # a normalizer function. value: bn, ln, ins, or None
    t = 1.0  # temperature
    dropout_rate = 0.2

    # train
    batch_size = 32

class Train1:
    # path
    data_path = '{}/timit/TIMIT/TRAIN/*/*/*.WAV'.format(data_path_base)

    # model
    hidden_units = 256  # alias = E
    num_banks = 16
    num_highway_blocks = 4
    norm_type = 'ins'  # a normalizer function. value: bn, ln, ins, or None
    t = 1.0  # temperature
    dropout_rate = 0.2

    # train
    batch_size = 32
    lr = 0.0003
    num_epochs = 10
    save_per_epoch = 2

class Train2:
    # path
    #data_path = '{}/kate/sense_and_sensibility_split/*.wav'.format(data_path_base)
    data_path = '{}/arctic/slt/*.wav'.format(data_path_base)

    # model
    hidden_units = 512  # alias = E
    num_banks = 16
    num_highway_blocks = 8
    norm_type = 'ins'  # a normalizer function. value: bn, ln, ins, or None
    t = 1.0  # temperature
    dropout_rate = 0.2

    # train
    batch_size = 16
    lr = 0.0005
    num_epochs = 10
    save_per_epoch = 2

class Test1:
    # path
    data_path = '{}/timit/TIMIT/TEST/*/*/*.WAV'.format(data_path_base)

    # test
    batch_size = 32

class Test2:
    # test
    batch_size = 32

class Convert:
    # path
    data_path = '{}/arctic/bdl/*.wav'.format(data_path_base)

    # convert
    batch_size = 2
    emphasis_magnitude = 1.2

[VictoriaBentell]

Hm, that looks fine to me. Try testing it out on the GPU for a few minutes just to see if it works with that. There may be some extra parameters that you need to change for CPU training, but I'm not sure.

[VictoriaBentell]

Oh! Or maybe in class Test2, you might want to lower batch_size down to 8, and same with Train2 just to be safe. Edit: Just turn the batch size for Train2 down to 8; I forgot that Test2 is called separately. Are you getting the error as soon as you run the program?

[emmacirl]

@andabi @VictoriaBentell I met the same problem, and it doesn't work after change batch size. And I found that this problem is caused by one placeholder is not assigned. However I don't know how to fix it. Do you?

[VictoriaBentell]

Can you give some more details on this placeholder you speak of?

[emmacirl]

@VictoriaBentell the error is the same as vivek-sethia's. @vivek-sethia Do you resolve it ?

[emmacirl]

I set queue=True in 'train2.py' and it works.

[vivek-sethia]

@emmacirl i did that earlier only and it didnt work for me. Anything else did you change? If you have repo somewhere , can I fork it?

[vivek-sethia]

@emmacirl Sorry for ealier comment, queue=True in 'train2.py' actually worked.