预训练模型

[glMa7]

请问你这里的bert预训练模型用的哪里的，我用的huggingface的bert-base-chinese，不但训练效果差，而且代码也一直报vocab的索引对不上的问题。

[lonePatient]

@glMa7 应该是模型下载错了吧，我使用的也huggingface的

[DuQingChen]

huggingface的可以正常使用

[ume-technology]

`Some weights of the model checkpoint at C:\BaiduNetdiskDownload\bert-base-chinese were not used when initializing BertCrfForNer: ['cls.predictions.decoder.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight']

• This IS expected if you are initializing BertCrfForNer from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing BertCrfForNer from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertCrfForNer were not initialized from the model checkpoint at C:\BaiduNetdiskDownload\bert-base-chinese and are newly initialized: ['classifier.bias', 'crf.start_transitions', 'crf.transitions', 'crf.end_transitions', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.`

我用的 transformers 版本是 4.12.5 然后加载模型的时候就会报出上面的问题, 我还没往下跑代码, 发现这个问题就先来找找看, 不知道会不会有题主说的情况出现, 总之就是说, premodel 并没有被完全加载进来, 还说我可能需要微调一下这个 pre model 才能继续用. 我换了其它的 pre model 也是一样的问题. 所以我不觉得是模型的问题, 应该是代码里处理 NER 任务的这个类: BertCrfForNer 有问题才会这样. 我准备再看看吧, 好烦, 就是想确定一个合适的任务 baseline, 但是 baseline 总是跑不出一个相对合适的结果. 很烦 ...

[Baileyswu]

我在hugging face上下载了bert-base-chinese到prev_trained_model这个目录是报下面这个错

Traceback (most recent call last): File "run_ner_crf.py", line 496, in main() File "run_ner_crf.py", line 428, in main model = model_class.from_pretrained(args.model_name_or_path, config=config) File "/Users/bytedance/Projects/BERT-NER-Pytorch/bert_pytorch/lib/python3.7/site-packages/transformers/modeling_utils.py", line 2604, in from_pretrained state_dict = load_state_dict(resolved_archive_file) File "/Users/bytedance/Projects/BERT-NER-Pytorch/bert_pytorch/lib/python3.7/site-packages/transformers/modeling_utils.py", line 450, in load_state_dict with safe_open(checkpoint_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

修复办法： scripts/run_ner_crf.sh 里的BERT_BASE_DIR=bert-base-chinese，就可以自动下载模型了，看样子git clone和这种默认下载下来的模型可能有略微的不同。