search

guillaume-chevalier / GloVe-as-a-TensorFlow-Embedding-Layer

Taking a pretrained GloVe model, and using it as a TensorFlow embedding weight layer **inside the GPU**. Therefore, you only need to send the index of the words through the GPU data transfer bus, reducing data transfer overhead.
https://www.neuraxio.com/
MIT License
90 stars 19 forks source link

Shapes of both tensors are mismatching #1

Open Dhawaleswrar opened 6 years ago

Dhawaleswrar commented 6 years ago

2018-07-31 19:50:48.846720: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA word_to_index dict restored from 'embeddings/glove.840B.300d.json'. 2018-07-31 19:50:52.224149: W tensorflow/core/framework/allocator.cc:108] Allocation of 2635221600 exceeds 10% of system memory. Traceback (most recent call last): File "2_word_similarities_tf.py", line 78, in tf_embedding = load_embedding_tf(word_to_index, TF_EMBEDDINGS_FILE_PATH, word_representations_dimensions) File "2_word_similarities_tf.py", line 49, in load_embedding_tf embedding_saver.restore(sess, save_path=tf_embeddings_file_path) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1752, in restore {self.saver_def.filename_tensor_name: save_path}) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 900, in run run_metadata_ptr) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _run feed_dict_tensor, options, run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [2196015,300] rhs shape= [2196018,300] [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@Embedding"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Embedding, save/RestoreV2)]]

Caused by op u'save/Assign', defined at: File "2_word_similarities_tf.py", line 78, in tf_embedding = load_embedding_tf(word_to_index, TF_EMBEDDINGS_FILE_PATH, word_representations_dimensions) File "2_word_similarities_tf.py", line 48, in load_embedding_tf embedding_saver = tf.train.Saver(variables_to_restore) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1284, in init self.build() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1296, in build self._build(self._filename, build_save=True, build_restore=True) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1333, in _build build_save=build_save, build_restore=build_restore) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 781, in _build_internal restore_sequentially, reshape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 422, in _AddRestoreOps assign_ops.append(saveable.restore(saveable_tensors, shapes)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 113, in restore self.op.get_shape().is_fully_defined()) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/state_ops.py", line 219, in assign validate_shape=validate_shape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 60, in assign use_locking=use_locking, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3414, in create_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1740, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [2196015,300] rhs shape= [2196018,300] [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@Embedding"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Embedding, save/RestoreV2)]]

guillaume-chevalier commented 5 years ago

Strange, it looks like 3 rows were added to the tensor. I'd be intrigued to see your "2_word_similarities_tf.py" file as it may differ from the current project.