taishan1994 / BERT-BILSTM-CRF

使用BERT-BILSTM-CRF进行中文命名实体识别。
291 stars 32 forks source link

改为自己的数据集和标签后报错 #24

Open PurpLyon opened 3 months ago

PurpLyon commented 3 months ago

Traceback (most recent call last): File "D:/BERT/ner/main.py", line 189, in main(data_name) File "D:/BERT/ner/main.py", line 183, in main report = train.test() File "D:/BERT/ner/main.py", line 66, in test self.model.load_state_dict(torch.load(os.path.join(self.output_dir, "pytorch_model_ner.bin"))) File "C:\Users\liu.conda\envs\bbc\lib\site-packages\torch\nn\modules\module.py", line 2153, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for BertNer: size mismatch for linear.weight: copying a param with shape torch.Size([5, 256]) from checkpoint, the shape in current model is torch.Size([13, 256]). size mismatch for linear.bias: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.start_transitions: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.end_transitions: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.transitions: copying a param with shape torch.Size([5, 5]) from checkpoint, the shape in current model is torch.Size([13, 13]). labels文件加了几个标签之后出现的问题,估计就是喂的向量不是模型需要的,这块怎么改啊,还是要自己重新训练模型?求助大佬,找了好多设置文件没有能修改标签数量的地方。

taishan1994 commented 3 months ago

训练成功没。

PurpLyon commented 3 months ago

训练成功没。

您的数据集跑成功了,我自己的数据没有。

DYdydydyDY commented 3 months ago

训练成功没。

您的数据集跑成功了,我自己的数据没有。

你好,我也遇到了同样的问题,请问你是怎么解决的

DYdydydyDY commented 3 months ago

训练成功没。

作者你好,我也遇到了同样的问题,我想知道在哪更改标签数量,我认为是我的标签数量过多而没有更改模型造成的

taishan1994 commented 3 months ago

训练成功没。

作者你好,我也遇到了同样的问题,我想知道在哪更改标签数量,我认为是我的标签数量过多而没有更改模型造成的

labels.txt是实体标签,train.txt和dev.txt里面是每个实体标签对应的BI,再加上O。

sore-2001 commented 1 month ago

Traceback (most recent call last): File "D:/BERT/ner/main.py", line 189, in main(data_name) File "D:/BERT/ner/main.py", line 183, in main report = train.test() File "D:/BERT/ner/main.py", line 66, in test self.model.load_state_dict(torch.load(os.path.join(self.output_dir, "pytorch_model_ner.bin"))) File "C:\Users\liu.conda\envs\bbc\lib\site-packages\torch\nn\modules\module.py", line 2153, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for BertNer: size mismatch for linear.weight: copying a param with shape torch.Size([5, 256]) from checkpoint, the shape in current model is torch.Size([13, 256]). size mismatch for linear.bias: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.start_transitions: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.end_transitions: copying a param with shape torch.Size([5]) from checkpoint, the shape in current model is torch.Size([13]). size mismatch for crf.transitions: copying a param with shape torch.Size([5, 5]) from checkpoint, the shape in current model is torch.Size([13, 13]). labels文件加了几个标签之后出现的问题,估计就是喂的向量不是模型需要的,这块怎么改啊,还是要自己重新训练模型?求助大佬,找了好多设置文件没有能修改标签数量的地方。

image 朋友你好,我在更换自己的数据集时也遇到了同样的报错,您解决了吗?想请教您一下