search

ottokart / punctuator2

A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text
http://bark.phon.ioc.ee/punctuator
MIT License
657 stars 195 forks source link

module '_pickle' has no attribute 'HIGHEST_PROTOCOL' #53

Closed helga-lvl closed 4 years ago

helga-lvl commented 4 years ago

Hey!

I was running the model (on Python 3.7.3) with around 200 megabytes of data. After 11 hours and 10 epochs of training, I got the following message:

  rval = inputs[0].__getitem__(inputs[1:])
Total number of training labels: 9985024
Total number of validation labels: 1248128
Validation perplexity is 1.1226
Traceback (most recent call last):
  File "main.py", line 210, in <module>
    net.save(model_file_name, gsums=gsums, learning_rate=learning_rate, validation_ppl_history=validation_ppl_history, best_validation_ppl=best_ppl, epoch=epoch, random_state=rng.get_state())
  File "/home/Projects/punctuator2-master/models.py", line 252, in save
    cPickle.dump(state, f, protocol=cPickle.HIGHEST_PROTOCOL)
AttributeError: module '_pickle' has no attribute 'HIGHEST_PROTOCOL'

Then I decided to see what the next step would give me and I got:

cat terrafoldernormallt/unurlnorm.dev.txt | python punctuator.py Model_models.py_h256_lr0.02.pcl model_output.pcl


Loading model parameters...
Traceback (most recent call last):
  File "punctuator.py", line 149, in <module>
    net, _ = models.load(model_file, 1, x)
  File "/home/Projects/punctuator2-master/models.py", line 63, in load
    state = cPickle.load(f, **cpickle_options)
EOFError: Ran out of input

Any help would be greatly appreciated, thanks!

AASHISHAG commented 4 years ago

@helga-lvl : How did you solve this issue? I'm also getting the same issue?

AttributeError: module '_pickle' has no attribute 'HIGHEST_PROTOCOL'

vjosapreniqi commented 4 years ago

I have encountered the same exact problem, I was also running the model on Python 3.7. @ottokart Do you have any clue about this error? @helga-lvl if you have already solved this problem, can you guide us on how you did it?

ottokart commented 4 years ago

Punctuator2 was originally implemented in Python 2.7. Python3 no longer has the attribute HIGHEST_PROTOCOL so I removed it in the latest pushed commit: https://github.com/ottokart/punctuator2/commit/8edec709146bf55f6833334ae8d6fbf9dd6a329b