Open djmv opened 4 years ago
@djmv could you please dump your error log?
I can see that the word2vec loading line will crash because of the mmap
keyword argument. You should use KeyedVectors.load_word2vec_format("GoogleNews-vectors-negative300.bin.gz", binary=True)
.
This will load the word2vec more slowly. You can load them way faster with the mmap
option as above, but sacrifice disk space by converting the unzipped word2vec.bin into two word2vec
and word2vec.syn0.npy
files with this (you only need to run it once):
path = "GoogleNews-vectors-negative300.bin.gz"
w2v = KeyedVectors.load_word2vec_format(path, binary=True)
w2v.init_sims(replace=True)
w2v.save("word2vec")
@djmv could you please dump your error log?
I can see that the word2vec loading line will crash because of the
mmap
keyword argument. You should useKeyedVectors.load_word2vec_format("GoogleNews-vectors-negative300.bin.gz", binary=True)
.This will load the word2vec more slowly. You can load them way faster with the
mmap
option as above, but sacrifice disk space by converting the unzipped word2vec.bin into twoword2vec
andword2vec.syn0.npy
files with this (you only need to run it once):path = "GoogleNews-vectors-negative300.bin.gz" w2v = KeyedVectors.load_word2vec_format(path, binary=True) w2v.init_sims(replace=True) w2v.save("word2vec")
Thanks! @danielwatson6 , now I have two files word2vec
and word2vec.vectors.npy
then I re-run the script with the changes. The model load, but I got errors in model.restore()
import tensorflow as tf
from gensim.models import KeyedVectors
from skip_thoughts import SkipThoughts
# Initialize the word2vec and skip-thoughts models only once:
word2vec_model = KeyedVectors.load('word2vec', mmap='r')
graph = tf.Graph()
with graph.as_default():
# Refer to the constructor docstring for more information on the arguments.
model = SkipThoughts(word2vec_model) # <-- What params have to put ? or for simplicity left default ?
with tf.Session(graph=graph):
# Restore the model only once.
# Here, `save_dir` is the directory where the .ckpt files live. Typically
# this would be "output/mymodel" where --model_name=mymodel in train.py.
model.restore(save_dir) # <-- What model I should restore ?
# Run the model like this as many times as desired.
print(model.encode(sentence_strings))
what is the next step ? Also, some additional info
Intel core i5-8250U
ubuntu 18.04
python 3.6.9
tensorboard (1.14.0)
tensorflow (1.14.0)
tensorflow-estimator (1.14.0)
gensim (3.8.0)
Thanks! for reply. The errors I got:
2020-05-06 08:20:21.095301: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-05-06 08:20:21.118819: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1800000000 Hz
2020-05-06 08:20:21.119188: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6c7f4f0 executing computations on platform Host. Devices:
2020-05-06 08:20:21.119235: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/Documents/practices/python/skip-thoughts/skip_thoughts.py:250: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
2020-05-06 08:20:21.172960: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Traceback (most recent call last):
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias
[[{{node bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "encoder_example.py", line 26, in <module>
print(model.encode("sentence_strings"))
File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 254, in encode
return sess.run(self._get_thought, feed_dict={self._inputs: sequences})
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias
[[node bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read (defined at /home/Documents/practices/python/skip-thoughts/skip_thoughts.py:141) ]]
Original stack trace for 'bidirectional_rnn/bw/cudnn_compatible_gru_cell/candidate/input_projection/bias/read':
File "encoder_example.py", line 17, in <module>
model = SkipThoughts(word2vec_model)
File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 117, in __init__
self._get_thought = self._thought(self._inputs)
File "/home/Documents/practices/python/skip-thoughts/skip_thoughts.py", line 141, in _thought
dtype=tf.float32, time_major=self._time_major)[1]
File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 503, in bidirectional_dynamic_rnn
scope=bw_scope)
File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 707, in dynamic_rnn
dtype=dtype)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 916, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3501, in while_loop
return_same_structure)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3012, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2937, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3456, in <lambda>
body = lambda i, lv: (i + 1, orig_body(*lv))
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 882, in _time_step
skip_conditionals=True)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 283, in _rnn_step
new_output, new_state = call_cell()
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 870, in <lambda>
call_cell = lambda: cell(input_t, state)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 385, in __call__
self, inputs, state, scope=scope, *args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 537, in __call__
outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 591, in __call__
self._maybe_build(inputs)
File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1881, in _maybe_build
self.build(input_shapes)
File "/home/lib/python3.6/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py", line 154, in build
init_ops.zeros_initializer(dtype=self.dtype)))
File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1484, in add_variable
return self.add_weight(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 450, in add_weight
**kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 384, in add_weight
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 663, in _add_variable_with_custom_getter
**kwargs_for_getter)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1496, in get_variable
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1239, in get_variable
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 562, in get_variable
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 514, in _true_getter
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _get_single_variable
aggregation=aggregation)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 259, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 220, in _variable_v1_call
shape=shape)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 198, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2511, in default_variable_creator
shape=shape)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 263, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1568, in __init__
shape=shape)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1752, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/home/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 86, in identity
ret = gen_array_ops.identity(input, name=name)
File "/home/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4253, in identity
"Identity", input=input, name=name)
File "/home/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2005, in __init__
self._traceback = tf_stack.extract_stack()
Hello, I'm new and I trying to use this tool. I have downloaded the google word2vec pre-trained file (GoogleNews-vectors-negative300.bin.gz). Then, I use the example code
Put in keyVectorload the file downloaded, but then when I run, got errors. I need some basic tutorial for test the example. please. I'm trying to enconde simple sentences and then I want to use that for train a classifier. And I read that with skip-thoughts are obtained good results and I want to test.
My question are: Where I put sentences? I need train some model or pre-trained file is enough ?
Please, I need help. Thanks !