search

lonePatient / albert_pytorch

A Lite Bert For Self-Supervised Learning Language Representations
Apache License 2.0
710 stars 152 forks source link

Any usage of load model? #9

Closed xf05888 closed 5 years ago

xf05888 commented 5 years ago

I want to load ALBERT in Pytorch, so I convert albert_zh_xlarge from Tensorflow to Pytorch and succeeded. But I don't know how to load it in Pytorch, is there some usage examples of it?

fangd123 commented 5 years ago

I want to load ALBERT in Pytorch, so I convert albert_zh_xlarge from Tensorflow to Pytorch and succeeded. But I don't know how to load it in Pytorch, is there some usage examples of it?

The model's config file is located in 'configs/base.py' , just modify it. In your condition, the config file should like this:

    'bert_dir':BASE_DIR / 'pretrain/pytorch/albert_zh_xlarge',
    'albert_config_path': BASE_DIR / 'configs/albert_config_xlarge.json',
    'albert_vocab_path': BASE_DIR / 'configs/vocab.txt'

The change the value of file albert_config_xlarge.json:

 "ln_type":"postln"

ATTENTION: the value of bert_dir should be a file path instead of directory's, you can put the converted model file into the pretrain/pytorch/ folder and modify the model's file name