suriyadeepan / practical_seq2seq

A simple, minimal wrapper for tensorflow's seq2seq module, for experimenting with datasets rapidly
http://suriyadeepan.github.io/2016-12-31-practical-seq2seq/
GNU General Public License v3.0
569 stars 270 forks source link

can't pickle _thread.lock objects #31

Open Jatapiaro opened 7 years ago

Jatapiaro commented 7 years ago

I'm running the twitter chatbot code but I'm getting the next error: "can't pickle _thread.lock objects"

What does that means?

dexterchan commented 7 years ago

I got the same issue with Python 3.6 with tensor 1.1.0

Building Graph --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () 5 ckpt_path='ckpt/cmudict/', 6 emb_dim=emb_dim, ----> 7 num_layers=3 8 ) /Users/dexter/Downloads/TensorFlowTutorial/practical_seq2seq-master/seq2seq_wrapper.py in __init__(self, xseq_len, yseq_len, xvocab_size, yvocab_size, emb_dim, num_layers, ckpt_path, lr, epochs, model_name) 77 sys.stdout.write(' Building Graph ') 78 # build comput graph ---> 79 __graph__() 80 sys.stdout.write('') 81 /Users/dexter/Downloads/TensorFlowTutorial/practical_seq2seq-master/seq2seq_wrapper.py in __graph__() 56 # inputs : encoder, decoder inputs, LSTM cell type, vocabulary sizes, embedding dimensions 57 self.decode_outputs, self.decode_states = tf.contrib.legacy_seq2seq.embedding_rnn_seq2seq(self.enc_ip,self.dec_ip, stacked_lstm, ---> 58 xvocab_size, yvocab_size, emb_dim) 59 # share parameters 60 scope.reuse_variables() /Users/dexter/anaconda/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in embedding_rnn_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, output_projection, feed_previous, dtype, scope) 356 357 # Encoder. --> 358 encoder_cell = copy.deepcopy(cell) 359 encoder_cell = core_rnn_cell.EmbeddingWrapper( 360 encoder_cell, /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 178 y = x 179 else: --> 180 y = _reconstruct(x, memo, *rv) 181 182 # If is its own copy, don't memoize. /Users/dexter/anaconda/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 278 if state is not None: 279 if deep: --> 280 state = deepcopy(state, memo) 281 if hasattr(y, '__setstate__'): 282 y.__setstate__(state) /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy) 238 memo[id(x)] = y 239 for key, value in x.items(): --> 240 y[deepcopy(key, memo)] = deepcopy(value, memo) 241 return y 242 d[dict] = _deepcopy_dict /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_list(x, memo, deepcopy) 213 append = y.append 214 for a in x: --> 215 append(deepcopy(a, memo)) 216 return y 217 d[list] = _deepcopy_list /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 178 y = x 179 else: --> 180 y = _reconstruct(x, memo, *rv) 181 182 # If is its own copy, don't memoize. /Users/dexter/anaconda/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 278 if state is not None: 279 if deep: --> 280 state = deepcopy(state, memo) 281 if hasattr(y, '__setstate__'): 282 y.__setstate__(state) /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy) 238 memo[id(x)] = y 239 for key, value in x.items(): --> 240 y[deepcopy(key, memo)] = deepcopy(value, memo) 241 return y 242 d[dict] = _deepcopy_dict /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 178 y = x 179 else: --> 180 y = _reconstruct(x, memo, *rv) 181 182 # If is its own copy, don't memoize. /Users/dexter/anaconda/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 278 if state is not None: 279 if deep: --> 280 state = deepcopy(state, memo) 281 if hasattr(y, '__setstate__'): 282 y.__setstate__(state) /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy) 238 memo[id(x)] = y 239 for key, value in x.items(): --> 240 y[deepcopy(key, memo)] = deepcopy(value, memo) 241 return y 242 d[dict] = _deepcopy_dict /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 178 y = x 179 else: --> 180 y = _reconstruct(x, memo, *rv) 181 182 # If is its own copy, don't memoize. /Users/dexter/anaconda/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 278 if state is not None: 279 if deep: --> 280 state = deepcopy(state, memo) 281 if hasattr(y, '__setstate__'): 282 y.__setstate__(state) /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy) 238 memo[id(x)] = y 239 for key, value in x.items(): --> 240 y[deepcopy(key, memo)] = deepcopy(value, memo) 241 return y 242 d[dict] = _deepcopy_dict /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 178 y = x 179 else: --> 180 y = _reconstruct(x, memo, *rv) 181 182 # If is its own copy, don't memoize. /Users/dexter/anaconda/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy) 278 if state is not None: 279 if deep: --> 280 state = deepcopy(state, memo) 281 if hasattr(y, '__setstate__'): 282 y.__setstate__(state) /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 148 copier = _deepcopy_dispatch.get(cls) 149 if copier: --> 150 y = copier(x, memo) 151 else: 152 try: /Users/dexter/anaconda/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy) 238 memo[id(x)] = y 239 for key, value in x.items(): --> 240 y[deepcopy(key, memo)] = deepcopy(value, memo) 241 return y 242 d[dict] = _deepcopy_dict /Users/dexter/anaconda/lib/python3.6/copy.py in deepcopy(x, memo, _nil) 167 reductor = getattr(x, "__reduce_ex__", None) 168 if reductor: --> 169 rv = reductor(4) 170 else: 171 reductor = getattr(x, "__reduce__", None) TypeError: can't pickle _thread.lock objects
Jatapiaro commented 7 years ago

I found a solution for it. You should check my code https://github.com/Jatapiaro/TwinkerBot_Revival Twitter traning starts the bot training, that's where I got the error. Copy paste my seq2seq_wrapper code, use python 3 and please use tensorflow 1.0.0

dexterchan commented 7 years ago

then I receive this... thank you

Building Graph --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in () 5 ckpt_path='ckpt/twitter/', 6 emb_dim=emb_dim, ----> 7 num_layers=3 8 ) /Users/dexter/Downloads/TensorFlowTutorial/TwinkerBot_Revival-master/seq2seq_wrapper.py in __init__(self, xseq_len, yseq_len, xvocab_size, yvocab_size, emb_dim, num_layers, ckpt_path, lr, epochs, model_name) 76 sys.stdout.write(' Building Graph ') 77 # build comput graph ---> 78 __graph__() 79 sys.stdout.write('') 80 /Users/dexter/Downloads/TensorFlowTutorial/TwinkerBot_Revival-master/seq2seq_wrapper.py in __graph__() 39 self.keep_prob = tf.placeholder(tf.float32) 40 # define the basic cell ---> 41 basic_cell = tf.contrib.rnn.core_rnn_cell.DropoutWrapper( 42 tf.contrib.rnn.core_rnn_cell.BasicLSTMCell(emb_dim, state_is_tuple=True), 43 output_keep_prob=self.keep_prob) AttributeError: module 'tensorflow.contrib.rnn' has no attribute 'core_rnn_cell'
Jatapiaro commented 7 years ago

@dexterchan Are you shure that you're using Tensorflow 1.0.0 and python3? You can check your tensorflow version using the command below: python3 -c "import tensorflow as tf; print(tf.version)"

dexterchan commented 7 years ago

Sorry, forget to upgrade my tensorflow.... it is running training now... thank you for your support.... :)

Jatapiaro commented 7 years ago

@dexterchan You're welcome :) And please be careful with this https://github.com/suriyadeepan/practical_seq2seq/issues/2 I recommend you to left your training running as most as you can, I've lost one day trying to restore sessions but it doesn't work, for some reason begins again.

say4n commented 7 years ago

To anyone trying to run python3 -c "import tensorflow as tf; print(tf.version)", you'll get a AttributeError, version has been renamed to VERSION, so, now you've to run python3 -c "import tensorflow as tf; print(tf.VERSION)"

anthdm commented 7 years ago

Using Tensorflow 1.0.0 fix this issue. To downgrade the gpu version pip3 install tensorflow-gpu==1.0.0 To downgrade the cpu version pip3 install tensorflow==1.0.0

Macintoshxz commented 7 years ago

@anthdm Thank you ! That works for me! Downgrade the Tensorflow to 1.0.0.

LilyDreamZhao commented 6 years ago

Is there any other way ?I wouldn't downgrade the tensorflow to 1.0.0