brightmart / albert_zh

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

https://arxiv.org/pdf/1909.11942.pdf

3.92k stars 755 forks source link

ValueError: Shape of variable bert/embeddings/LayerNorm/beta:0 ((312,)) doesn't match with shape of tensor bert/embeddings/LayerNorm/beta ([128]) from checkpoint reader #137

Open BrightHai opened 4 years ago

BrightHai commented 4 years ago

使用albert_tiny_zh_google预训练模型，args.py 文件配置：

模型目录

model_dir = os.path.join(file_path, 'albert_tiny_zh_google/')

config文件

config_name = os.path.join(file_path, 'albert_config/albert_config_tiny_google.json')

ckpt文件名称

ckpt_name = os.path.join(model_dir, 'albert_model.ckpt')

输出文件目录

output_dir = os.path.join(file_path, 'albert_tiny_zh_google/')

vocab文件目录

vocab_file = os.path.join(file_path, 'albert_tiny_zh_google/vocab.txt')

baselineHelper commented 4 years ago

同问我这边使用的albert_base的预训练模型，同样在结果中出错，报错信息 Shape of variable bert/embeddings/LayerNorm/beta:0 ((128,)) doesn't match with shape of tensor bert/embeddings/LayerNorm/beta ([768]) from checkpoint reader，麻烦作者帮忙解决下

brightmart commented 4 years ago

你这是用pytorch吗？

nocoolsandwich commented 3 years ago

我也是这个问题好像是因为这个问题