Open orangejuice1007 opened 1 year ago
估计是BERT的版本不匹配 我刚刚也遇到了这个问题,试了下sequence_output = x[0]就可以了。 猜测原因如下: bert的输出包括last_hidden_state,pooler_output,hidden_states,attentions,默认只输出前两项,想要输出后两项需要像你说的那样修改output_hidden_states = True和output_attentions = True 代码中需要取的是last_hidden_state即最后一层的隐状态,而该版本似乎将其作为x的一个属性。而我的版本将x处理成了一个元组,故取了x的第一个元素。
估计是BERT的版本不匹配 我刚刚也遇到了这个问题,试了下sequence_output = x[0]就可以了。 猜测原因如下: bert的输出包括last_hidden_state,pooler_output,hidden_states,attentions,默认只输出前两项,想要输出后两项需要像你说的那样修改output_hidden_states = True和output_attentions = True 代码中需要取的是last_hidden_state即最后一层的隐状态,而该版本似乎将其作为x的一个属性。而我的版本将x处理成了一个元组,故取了x的第一个元素。
谢谢!我目前可以成功训练出模型,但是不知道为什么运行demo_eval.py预测的实体全部为空……
(pytorch18) z@z:~/code/nlp-notebook-master/3-2.Bert-CRF$ python demo_train.py Some weights of the model checkpoint at ./bert-base-chinese were not used when initializing BertForNER: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias'] This IS expected if you are initializing BertForNER from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model). This IS NOT expected if you are initializing BertForNER from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertForNER were not initialized from the model checkpoint at ./bert-base-chinese and are newly initialized: ['transitions', 'hidden2label.weight', 'hidden2label.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. [Train Epoch 0]: 0%| | 0/1584 [00:00<?, ?it/s] Traceback (most recent call last): File "demo_train.py", line 66, in
run()
File "demo_train.py", line 53, in run
loss = model.neg_log_likelihood(input_ids, attention_mask, label_ids, real_lens)
File "/home/z/code/nlp-notebook-master/3-2.Bert-CRF/model.py", line 137, in neg_log_likelihood
feats = self.get_features(input_ids, attention_mask)
File "/home/z/code/nlp-notebook-master/3-2.Bert-CRF/model.py", line 53, in get_features
sequence_output, pooled_output = x.last_hidden_state, x.pooler_output
AttributeError: 'tuple' object has no attribute 'last_hidden_state'
输出如上,尝试修改model.from_pretrained(model_path,output_hidden_states = True)也不行 请问是哪里出了问题?环境配置是一样的