monikkinom / ner-lstm

Named Entity Recognition using multilayered bidirectional LSTM
540 stars 183 forks source link

Error: ValueError: Cannot feed value of shape (4, 1000, 122) for Tensor u'Placeholder:0', which has shape '(?, 1000, 111)' #14

Closed nitinmeharia closed 7 years ago

nitinmeharia commented 7 years ago

I'm a newbie in this particular field. Any help would be much appreciated. I'm getting the following error.

Traceback (most recent call last):
  File "model.py", line 124, in <module>
    train(parser.parse_args())
  File "model.py", line 96, in train
    model.output_data: train_out[ptr:ptr + args.batch_size]})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 767, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 944, in _run
    % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (4, 1000, 122) for Tensor u'Placeholder:0', which has shape '(?, 1000, 111)'

The args I've set in are as follows:

python wordvec_model.py --corpus 'all_data.txt' --dimension 111 --vocab_size 5000000 --sentence_length 1000 --workers 4

python get_conll_embeddings.py --train 'eng.train_50' --test_a 'eng.test_a_50' --test_b 'eng.test_b_50' --sentence_length 1000 --use_model 'wordvec_model_111.pkl' --model_dim 111

python model.py --word_dim 111 --sentence_length 1000 --class_size 20 --batch_size 256

I'm getting the error in this part. Before execution our objs look like this:

model.input_data: <tf.Tensor 'Placeholder:0' shape=(?, 1000, 111) dtype=float32> model.output_data: <tf.Tensor 'Placeholder_1:0' shape=(?, 1000, 20) dtype=float32>

sess.run(model.train_op, {model.input_data: train_inp[ptr:ptr + args.batch_size],
                                          model.output_data: train_out[ptr:ptr + args.batch_size]})

Also, I ported to tensorflow 1.0, but that shouldn't be a problem here I guess.

exsperazo commented 7 years ago

The word dimensions for the model.py (--word_dim) is actually your word dimensions from the word embeddings plus the length of your features Lets say your dimensions in the word2vec model are 200 and you have 5 chunk tags + 5 pos tags then for model.py those would be 210 dimensions

nitinmeharia commented 7 years ago

So as I understand, in the above implementation word_dim should be 122(111 from word2vec and additional 11 as mentioned in README) . Is it correct?

monikkinom commented 7 years ago

Yes!