Error on running run_blstm.sh

[divija96]

Using TensorFlow backend. [INFO] [2017-10-11 07:58:30,363] Writing to path: TEST_RESULTS/blstm/results-blstm_10-1000-10-32-True [INFO] [2017-10-11 07:58:30,364] Loading VSM [INFO] [2017-10-11 07:58:39,627] Reading training data [INFO] [2017-10-11 07:58:39,628] Training a model [INFO] [2017-10-11 07:58:39,628] Config: [10, 1000, 10, 32, True] 2017-10-11 07:58:39.636500: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-10-11 07:58:39.636600: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-10-11 07:58:39.636667: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. Traceback (most recent call last): File "../code/blstm.py", line 189, in classifier = build_lstm(len(y_values)+1, embedding_matrix) File "../code/blstm.py", line 49, in build_lstm merged_cnn = merge([cnn(embedded) for cnn in cnn], mode="concat") File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1528, in merge name=name) File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1188, in init self.add_inbound_node(layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 572, in add_inbound_node Node.create_node(self, inbound_layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 154, in create_node output_tensors = to_list(outbound_layer.call(input_tensors, mask=input_masks)) File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 1275, in call return K.concatenate(inputs, axis=self.concat_axis) File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 716, in concatenate return tf.concat(axis, [to_dense(x) for x in tensors]) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1061, in concat dtype=dtypes.int32).get_shape( File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 611, in convert_to_tensor as_ref=False) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 676, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 121, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 102, in constant tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 376, in make_tensor_proto _AssertCompatible(values, dtype) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible (dtype.name, repr(mismatch), type(mismatch).name)) TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

Could you please look into this? It would be of great help. Thanks in advance.

[dodinh]

We did use Theano as a backend, but judging by your error message, this seems to be a problem with your Tensorflow version, see also:

[1] https://github.com/tensorflow/tensorflow/issues/7031 [2] https://stackoverflow.com/questions/41813665/tensorflow-slim-typeerror-expected-int32-got-list-containing-tensors-of-type [3] https://github.com/carpedm20/DCGAN-tensorflow/issues/99 [4] https://stackoverflow.com/questions/45266707/keras-typeerror-expected-int32-got-list-containing-tensors-of-type-message

[divija96]

In the requirements.txt the Tensorflow version is specified as 0.10.0rc0. However, it seems to have been commented out. Could you please tell me which version of tensorflow is required or update the requirements file accordingly.

[dodinh]

The requirements file states that we used Theano==0.8.2 (i.e. not Tensorflow). I cannot tell you about Tensorflow other than that the error message you posted seems to match the errors in the links I gave you.

If you can get it running with a newer Tensorflow version, we'd be happy to include that version in the Readme.

[divija96]

Sorry, I had to change the backend to Theano in keras.json But inspite of doing that I'm getting another error:

[INFO] [2017-10-13 16:30:12,819] Writing to path: TEST_RESULTS/blstm/results-blstm_5-None-10-32-True [INFO] [2017-10-13 16:30:12,820] Loading VSM [INFO] [2017-10-13 16:30:24,124] Reading training data [INFO] [2017-10-13 16:30:24,124] Training a model [INFO] [2017-10-13 16:30:24,124] Config: [5, None, 10, 32, True] [INFO] [2017-10-13 16:30:24,909] Summary: Traceback (most recent call last): File "../code/blstm.py", line 192, in classifier.fit(X_train, to_categorical(y_train, len(y_values)+1), batch_size=BATCH_SIZE, class_weight=class_weight, verbose=1, nb_epoch=EPOCHS) File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1034, in fit batch_size=batch_size) File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 961, in _standardize_user_data exception_prefix='model input') File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 108, in standardize_input_data str(array.shape)) Exception: Error when checking model input: expected input_1 to have shape (None, 5) but got array with shape (0, 1)

[dodinh]

I just cloned and tested, everything running. However, at first I forgot to add the test data, and got a similar error to yours (further below in the code). Thus what you are probably seeing here is that an empty input list is supplied to classifier.fit().

Did you download and copy the data (training and labeled test) into the correct folders (see README.md)?

[divija96]

The issue is solved, but how to test this on other unannotated data?

[SteffenEger]

Bring it in the same format as the original test data or else adapt the code to handle your own data format. Please be aware that our code is only for classifying already extracted keyphrases. We do not perform keyphrase extraction.

[divija96]

Okay, thanks for all your help :)