Guide how to encode a simple sentence

[djmv]

Hello, I'm new and I trying to use this tool. I have downloaded the google word2vec pre-trained file (GoogleNews-vectors-negative300.bin.gz). Then, I use the example code

import tensorflow as tf
from gensim.models import KeyedVectors

from skip_thoughts import SkipThoughts

# Initialize the word2vec and skip-thoughts models only once:
word2vec_model = KeyedVectors.load('GoogleNews-vectors-negative300.bin.gz', mmap='r')
graph = tf.Graph()
with graph.as_default():
  # Refer to the constructor docstring for more information on the arguments.
  model = SkipThoughts(word2vec_model, **kwargs) # What params have to put ? 

with tf.Session(graph=graph):
  # Restore the model only once.
  # Here, `save_dir` is the directory where the .ckpt files live. Typically
  # this would be "output/mymodel" where --model_name=mymodel in train.py.
  model.restore(save_dir)

  # Run the model like this as many times as desired.
  print(model.encode(sentence_strings))

Put in keyVectorload the file downloaded, but then when I run, got errors. I need some basic tutorial for test the example. please. I'm trying to enconde simple sentences and then I want to use that for train a classifier. And I read that with skip-thoughts are obtained good results and I want to test.

My question are: Where I put sentences? I need train some model or pre-trained file is enough ?

Please, I need help. Thanks !

[danielwatson6]

@djmv could you please dump your error log?

I can see that the word2vec loading line will crash because of the mmap keyword argument. You should use KeyedVectors.load_word2vec_format("GoogleNews-vectors-negative300.bin.gz", binary=True).

This will load the word2vec more slowly. You can load them way faster with the mmap option as above, but sacrifice disk space by converting the unzipped word2vec.bin into two word2vec and word2vec.syn0.npy files with this (you only need to run it once):

path = "GoogleNews-vectors-negative300.bin.gz"
w2v = KeyedVectors.load_word2vec_format(path, binary=True)
w2v.init_sims(replace=True)
w2v.save("word2vec")

[djmv]

@djmv could you please dump your error log?

I can see that the word2vec loading line will crash because of the mmap keyword argument. You should use KeyedVectors.load_word2vec_format("GoogleNews-vectors-negative300.bin.gz", binary=True).

This will load the word2vec more slowly. You can load them way faster with the mmap option as above, but sacrifice disk space by converting the unzipped word2vec.bin into two word2vec and word2vec.syn0.npy files with this (you only need to run it once):

path = "GoogleNews-vectors-negative300.bin.gz"
w2v = KeyedVectors.load_word2vec_format(path, binary=True)
w2v.init_sims(replace=True)
w2v.save("word2vec")

Thanks! @danielwatson6 , now I have two files word2vec and word2vec.vectors.npy then I re-run the script with the changes. The model load, but I got errors in model.restore()

import tensorflow as tf
from gensim.models import KeyedVectors

from skip_thoughts import SkipThoughts

# Initialize the word2vec and skip-thoughts models only once:
word2vec_model = KeyedVectors.load('word2vec', mmap='r')
graph = tf.Graph()
with graph.as_default():
  # Refer to the constructor docstring for more information on the arguments.
  model = SkipThoughts(word2vec_model) # <-- What params have to put ? or for simplicity left default ? 

with tf.Session(graph=graph):
  # Restore the model only once.
  # Here, `save_dir` is the directory where the .ckpt files live. Typically
  # this would be "output/mymodel" where --model_name=mymodel in train.py.
  model.restore(save_dir)  # <-- What model I should restore ?  

  # Run the model like this as many times as desired.
  print(model.encode(sentence_strings))

what is the next step ? Also, some additional info

Intel core i5-8250U 
ubuntu 18.04 
python 3.6.9
tensorboard (1.14.0)
tensorflow (1.14.0)
tensorflow-estimator (1.14.0)
gensim (3.8.0)

Thanks! for reply. The errors I got:

2020-05-06 08:20:21.095301: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-05-06 08:20:21.118819: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1800000000 Hz
2020-05-06 08:20:21.119188: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6c7f4f0 executing computations on platform Host. Devices:
2020-05-06 08:20:21.119235: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/Documents/practices/python/skip-thoughts/skip_thoughts.py:250: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.

2020-05-06 08:20:21.172960: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Traceback (most recent call last):
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
    return fn(*args)
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias
     [[{{node bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "encoder_example.py", line 26, in <module>
    print(model.encode("sentence_strings"))
  File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 254, in encode
    return sess.run(self._get_thought, feed_dict={self._inputs: sequences})
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 950, in run
    run_metadata_ptr)
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1173, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
    run_metadata)
  File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias
     [[node bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read (defined at /home/Documents/practices/python/skip-thoughts/skip_thoughts.py:141) ]]

Original stack trace for 'bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read':
  File "encoder_example.py", line 17, in <module>
    model = SkipThoughts(word2vec_model)
  File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 117, in __init__
    self._get_thought = self._thought(self._inputs)
  File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 141, in _thought
    dtype=tf.float32, time_major=self._time_major)[1]
  File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 503, in bidirectional_dynamic_rnn
    scope=bw_scope)
  File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 707, in dynamic_rnn
    dtype=dtype)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 916, in _dynamic_rnn_loop
    swap_memory=swap_memory)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3501, in while_loop
    return_same_structure)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3012, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2937, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3456, in <lambda>
    body = lambda i, lv: (i + 1, orig_body(*lv))
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 882, in _time_step
    skip_conditionals=True)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 283, in _rnn_step
    new_output, new_state = call_cell()
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 870, in <lambda>
    call_cell = lambda: cell(input_t, state)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 385, in __call__
    self, inputs, state, scope=scope, *args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 537, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 591, in __call__
    self._maybe_build(inputs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1881, in _maybe_build
    self.build(input_shapes)
  File "/home/lib/python3.6/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py", line 154, in build
    init_ops.zeros_initializer(dtype=self.dtype)))
  File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1484, in add_variable
    return self.add_weight(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 450, in add_weight
    **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 384, in add_weight
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 663, in _add_variable_with_custom_getter
    **kwargs_for_getter)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1496, in get_variable
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1239, in get_variable
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 562, in get_variable
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 514, in _true_getter
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _get_single_variable
    aggregation=aggregation)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 259, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 220, in _variable_v1_call
    shape=shape)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 198, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2511, in default_variable_creator
    shape=shape)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 263, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1568, in __init__
    shape=shape)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1752, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/home/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 86, in identity
    ret = gen_array_ops.identity(input, name=name)
  File "/home/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4253, in identity
    "Identity", input=input, name=name)
  File "/home/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
    op_def=op_def)
  File "/home/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2005, in __init__
    self._traceback = tf_stack.extract_stack()