search

tlatkowski / multihead-siamese-nets

Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.
MIT License
182 stars 43 forks source link

do you process the unknown words? #9

Closed xy-always closed 5 years ago

xy-always commented 6 years ago

I found that if the sentence have the unknown word, this program will go wrong.

tlatkowski commented 6 years ago

Hi, what do you mean by go wrong? Could you provide more details?

I looked into DatasetVectorizer class and it seems that i haven't implemented out-of-vocabulary token case.

xy-always commented 6 years ago

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call return fn(*args) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn status, run_metadata) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = 2427 is not in [0, 2426) [[Node: embeddings/Gather_1 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](embeddings/word_embeddings/read, _arg_Placeholder_1_0_1)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "run.py", line 217, in main() File "run.py", line 213, in main predict(main_config, model_config, args.model, args.dataset) File "run.py", line 177, in predict prediction = session.run([model.temp_sim], feed_dict=feed_dict) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run run_metadata_ptr) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run feed_dict_tensor, options, run_metadata) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run options, run_metadata) File "/usr/local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = 2427 is not in [0, 2426) [[Node: embeddings/Gather_1 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](embeddings/word_embeddings/read, _arg_Placeholder_1_0_1)]]

tlatkowski commented 6 years ago

that's right, it looks like a bug related to OOV token - current version of code does not support OOV token.

xy-always commented 6 years ago

ok, thanks.

tlatkowski commented 6 years ago

one additional question, have you encountered this issue during the training time?

xy-always commented 6 years ago

yes, when transform a sentence use def vectorize(self, sentence): a = np.array(list(self.vocabulary.transform([sentence]))) the OOV token index is not zero, i use a = np.array(list(self.vocabulary.fit_transform([sentence]))) and it works.