rikeda71 / Char-BLSTM-CRF-for-Japanese

An Inplementation of Character based Bidirectional LSTM CRF for Japanese
MIT License
5 stars 1 forks source link

pickle error when running the predict_sample.py code #5

Closed KavyaGujjala closed 5 years ago

KavyaGujjala commented 5 years ago

code:

from deepjapaner.modelapi import ModelAPI

api = ModelAPI(model_path=r'D:\CRF_Jap\model_1_jap.pth', train_path=r'D:\japanese_ner_datasets\IOB2Corpus-master\hironsan.txt',
               wordemb_path='D:\CRF_Jap\ja_tar\ja.txt', charemb_path=r'D:\CRF_Jap\ja_tar\ja.txt',
               hidden_size=300)

label = api.predict('私は六花亭の白い恋人を食べました')
print(label)

ERROR looks like:

Traceback (most recent call last):
  File "D:/CRF_Jap/Char-BLSTM-CRF-for-Japanese-master/predict_sample.py", line 5, in <module>
    hidden_size=300)
  File "D:\CRF_Jap\Char-BLSTM-CRF-for-Japanese-master\deepjapaner\modelapi.py", line 35, in __init__
    self.model.load(model_path)
  File "D:\CRF_Jap\Char-BLSTM-CRF-for-Japanese-master\deepjapaner\model.py", line 62, in load
    self.load_state_dict(torch.load(model_path))
  File "C:\Users\kavya\Anaconda3\lib\site-packages\torch\serialization.py", line 367, in load
    return _load(f, map_location, pickle_module)
  File "C:\Users\kavya\Anaconda3\lib\site-packages\torch\serialization.py", line 528, in _load
    magic_number = pickle_module.load(f)
_pickle.UnpicklingError: pickle data was truncated

Process finished with exit code 1

I had created a .pth file and added my site-packages path in that file ( C:\Users\kavya\Anaconda3\Lib\site-packages )

What can be the reason?

Should I make any other changes in the codes. Can you please help me with this?

rikeda71 commented 5 years ago

A .pth file must be trained BLSTM-CRF model. How did you create D:\CRF_Jap\model_1_jap.pth ? If you use commands such as touch D:\CRF_Jap\model_1_jap.pth to create .pth file, please train BLSTM-CRF model. Training program create .pth file such as model.pth after training step. The example of Training step is train_sample.py.