zxlzr
commented
6 months ago 您好，模型训练的时候是基于一句话一个三元组方式训的，您可以先进行实体识别，然后对同一个句子组成1个实体对+句子输入进行多次关系分类，这样可以减轻一下上述问题。如果您想端到端抽取也可以直接使用OneKE http://oneke.openkg.cn/
Open Daniel-ChenJH opened 6 months ago
zxlzr
commented
6 months ago 您好，模型训练的时候是基于一句话一个三元组方式训的，您可以先进行实体识别，然后对同一个句子组成1个实体对+句子输入进行多次关系分类，这样可以减轻一下上述问题。如果您想端到端抽取也可以直接使用OneKE http://oneke.openkg.cn/
Daniel-ChenJH
commented
6 months ago 对同一个句子组成1个实体对+句子输入进行多次关系分类
感谢回复！我明白了，所以当前模型只能处理一句话两个实体+一个关系的简单句对吧？
另外，您的意思是two-stage，对一个复杂句提取得到多个实体后，将这些实体两两组合、一次次进行关系抽取是吗？比如上面的句子五个实体，那么就需要进行C5^2=10次关系抽取，然后取置信度高的几个？
我比较好奇当前版本模型对于多个实体的关系抽取是怎么做的，看您的意思的话，当前并不是两两组合一个个判断得到的？
zxlzr
commented
6 months ago 1 对的 2 对的 3 应该直接多个输入后，模型遍历后进行抽取（开发太久了有点忘记细节了），目前多实体关系抽取能力有限，您可以考虑根据类型过滤掉这些非常离谱的错误，您也可以自己标注一些数据进行二次训练优化下效果可能会好一些。
Daniel-ChenJH
commented
6 months ago 谢谢！我采用了您的方法，对"林忠钦，男，汉族，1957年12月6日出生，浙江省宁波市人，机械工程专家，中国工程院院士，教育部科技委先进制造学部 主任。曾任上海交通大学校长、党委副书记。"这句话单独进行两个实体的关系抽取工作，想问下这里为什么会输出“"1957年12月6日" 和 "浙江省宁波市" 在句中关系为："出生日期"，置信度为0.97。”呢？
我应该怎么理解这个结果？1、模型理解正确，没有管浙江省宁波市，而单独把"1957年12月6日" 解读为了出生日期，这跟原句子是相符的；2、模型理解错误，将"1957年12月6日" 理解为浙江省宁波市的出生日期？
类似的情况也在选择“汉族”、“1957年12月6日”两个实体进行关系抽取工作时发生了。[2024-06-06 16:57:01,502][main][INFO] - "汉族" 和 "1957年12月6日" 在句中关系为："出生日期"，置信度为0.97。
对于这种有点无厘头的两个实体之间的关系抽取，我理解应该是生成一个置信度很低的不明确关系才对？
(deepke) cjh@DESKTOP-NERJPTB:~/domain/DeepKE/example/re/standard$ python predict.py
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/core/utils.py:207: UserWarning:
Using config_path to specify the config name is deprecated, specify the config name via config_name
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/config_path_changes
warnings.warn(category=UserWarning, message=msg)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive hydra/output/custom.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive preprocess.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive train.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive embedding.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive predict.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @package directive model/lm.yaml in file:///home/cjh/domain/DeepKE/example/re/standard/conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/omegaconf/basecontainer.py:225: UserWarning: cfg.pretty() is deprecated and will be removed in a future version.
Use OmegaConf.to_yaml(cfg)
warnings.warn(
cwd: /home/cjh/domain/DeepKE/example/re/standard
use_wandb: false
preprocess: true
data_path: data/origin
out_path: data/out
chinese_split: true
replace_entity_with_type: true
replace_entity_with_scope: true
min_freq: 3
pos_limit: 30
seed: 1
use_gpu: true
gpu_id: 0
epoch: 50
batch_size: 32
learning_rate: 0.0003
lr_factor: 0.7
lr_patience: 3
weight_decay: 0.001
early_stopping_patience: 6
train_log: true
log_interval: 10
show_plot: false
only_comparison_plot: false
plot_utils: matplot
predict_plot: false
use_multi_gpu: false
gpu_ids: 0,1
vocab_size: ???
word_dim: 60
pos_size: 62
pos_dim: 10
dim_strategy: sum
num_relations: 51
fp: /home/cjh/domain/DeepKE/example/re/standard/re_bert.pth
model_name: lm
lm_file: /home/cjh/bert/chinese_wwm_pytorch
num_hidden_layers: 1
type_rnn: LSTM
input_size: 768
hidden_size: 100
num_layers: 1
dropout: 0.3
bidirectional: true
last_layer_hn: true
是否使用范例[y/n]，退出请输入: exit .... n
请输入句子：林忠钦，男，汉族，1957年12月6日出生，浙江省宁波市人，机械工程专家，中国工程院院士，教育部科技委先进制造学部 主任。曾任上海交通大学校长、党委副书记。
请输入句中需要预测关系的头实体：1957年12月6日
请输入头实体类型：Date
请输入句中需要预测关系的尾实体：浙江省宁波市
请输入尾实体类型：地点
[2024-06-06 16:53:38,447][deepke.relation_extraction.standard.tools.preprocess][INFO] - use bert tokenizer...
[2024-06-06 16:53:38,466][__main__][INFO] - device: cpu
Some weights of the model checkpoint at /home/cjh/bert/chinese_wwm_pytorch were not used when initializing BertModel: ['bert.encoder.layer.9.attention.self.value.weight', 'bert.encoder.layer.3.output.LayerNorm.bias', 'bert.encoder.layer.4.output.dense.bias', 'bert.encoder.layer.7.output.LayerNorm.bias', 'bert.encoder.layer.3.output.LayerNorm.weight', 'bert.encoder.layer.9.attention.self.key.weight', 'bert.encoder.layer.6.attention.self.query.bias', 'bert.encoder.layer.10.attention.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.key.bias', 'cls.predictions.decoder.weight', 'bert.encoder.layer.8.attention.self.query.bias', 'bert.encoder.layer.11.intermediate.dense.bias', 'bert.encoder.layer.11.attention.output.dense.weight', 'bert.encoder.layer.3.intermediate.dense.weight', 'bert.encoder.layer.5.attention.self.value.bias', 'bert.encoder.layer.10.intermediate.dense.weight', 'bert.encoder.layer.2.intermediate.dense.bias', 'bert.encoder.layer.6.attention.self.value.bias', 'bert.encoder.layer.2.output.LayerNorm.weight', 'bert.encoder.layer.6.attention.output.LayerNorm.weight', 'bert.encoder.layer.8.intermediate.dense.weight', 'cls.predictions.transform.dense.weight', 'bert.encoder.layer.9.output.dense.bias', 'bert.encoder.layer.8.attention.output.dense.bias', 'bert.encoder.layer.7.attention.self.key.bias', 'bert.encoder.layer.10.attention.output.dense.bias', 'bert.encoder.layer.10.output.dense.weight', 'bert.encoder.layer.9.attention.output.dense.weight', 'bert.encoder.layer.5.attention.self.key.weight', 'bert.encoder.layer.5.attention.self.value.weight', 'bert.encoder.layer.5.attention.output.LayerNorm.weight', 'bert.encoder.layer.1.attention.self.query.weight', 'bert.encoder.layer.9.output.LayerNorm.weight', 'cls.seq_relationship.weight', 'bert.encoder.layer.2.intermediate.dense.weight', 'bert.encoder.layer.4.output.LayerNorm.weight', 'cls.seq_relationship.bias', 'bert.encoder.layer.6.attention.self.query.weight', 'bert.encoder.layer.8.output.dense.bias', 'bert.encoder.layer.10.attention.self.key.bias', 'bert.encoder.layer.3.attention.self.query.bias', 'bert.encoder.layer.11.attention.self.query.bias', 'bert.encoder.layer.11.output.dense.weight', 'cls.predictions.bias', 'bert.encoder.layer.7.attention.output.LayerNorm.weight', 'bert.encoder.layer.7.attention.output.dense.bias', 'bert.encoder.layer.5.output.dense.weight', 'bert.encoder.layer.7.output.dense.bias', 'bert.encoder.layer.7.attention.self.query.weight', 'bert.encoder.layer.3.output.dense.weight', 'bert.encoder.layer.7.intermediate.dense.weight', 'bert.encoder.layer.11.attention.output.LayerNorm.bias', 'bert.encoder.layer.7.intermediate.dense.bias', 'bert.encoder.layer.2.attention.self.value.weight', 'bert.encoder.layer.7.attention.self.value.weight', 'bert.encoder.layer.11.attention.self.key.weight', 'bert.encoder.layer.8.attention.self.query.weight', 'bert.encoder.layer.1.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.output.dense.weight', 'bert.encoder.layer.2.attention.output.dense.weight', 'bert.encoder.layer.4.attention.self.query.bias', 'bert.encoder.layer.5.output.LayerNorm.weight', 'bert.encoder.layer.5.intermediate.dense.weight', 'bert.encoder.layer.6.attention.output.dense.weight', 'bert.encoder.layer.7.attention.output.LayerNorm.bias', 'bert.encoder.layer.11.output.LayerNorm.weight', 'bert.encoder.layer.8.output.LayerNorm.weight', 'bert.encoder.layer.8.attention.self.value.bias', 'bert.encoder.layer.10.attention.self.query.bias', 'bert.encoder.layer.5.attention.output.dense.bias', 'bert.encoder.layer.9.attention.self.key.bias', 'bert.encoder.layer.8.attention.self.value.weight', 'bert.encoder.layer.4.intermediate.dense.bias', 'bert.encoder.layer.10.intermediate.dense.bias', 'bert.encoder.layer.10.output.LayerNorm.weight', 'bert.encoder.layer.1.attention.output.dense.bias', 'bert.encoder.layer.5.output.dense.bias', 'bert.encoder.layer.10.attention.self.value.weight', 'bert.encoder.layer.10.attention.self.value.bias', 'bert.encoder.layer.8.intermediate.dense.bias', 'bert.encoder.layer.9.attention.self.query.bias', 'bert.encoder.layer.6.output.dense.weight', 'bert.encoder.layer.11.output.dense.bias', 'bert.encoder.layer.5.attention.output.dense.weight', 'bert.encoder.layer.3.attention.self.value.bias', 'cls.predictions.transform.LayerNorm.bias', 'bert.encoder.layer.9.output.LayerNorm.bias', 'bert.encoder.layer.1.output.dense.bias', 'bert.encoder.layer.10.output.LayerNorm.bias', 'bert.encoder.layer.11.attention.self.value.bias', 'bert.encoder.layer.6.output.LayerNorm.bias', 'bert.encoder.layer.6.attention.output.LayerNorm.bias', 'bert.encoder.layer.9.attention.output.LayerNorm.weight', 'bert.encoder.layer.2.output.LayerNorm.bias', 'bert.encoder.layer.2.attention.self.query.weight', 'bert.encoder.layer.8.attention.output.LayerNorm.bias', 'bert.encoder.layer.4.attention.self.query.weight', 'bert.encoder.layer.1.attention.output.dense.weight', 'bert.encoder.layer.2.attention.self.query.bias', 'bert.encoder.layer.7.attention.self.value.bias', 'bert.encoder.layer.7.attention.output.dense.weight', 'bert.encoder.layer.9.intermediate.dense.bias', 'bert.encoder.layer.9.attention.output.dense.bias', 'bert.encoder.layer.5.attention.self.query.bias', 'bert.encoder.layer.8.attention.output.dense.weight', 'bert.encoder.layer.3.intermediate.dense.bias', 'bert.encoder.layer.4.attention.output.dense.bias', 'bert.encoder.layer.7.output.dense.weight', 'bert.encoder.layer.9.attention.self.query.weight', 'bert.encoder.layer.1.attention.self.value.bias', 'bert.encoder.layer.1.output.LayerNorm.weight', 'bert.encoder.layer.10.output.dense.bias', 'bert.encoder.layer.11.attention.output.LayerNorm.weight', 'bert.encoder.layer.2.attention.output.LayerNorm.bias', 'bert.encoder.layer.11.intermediate.dense.weight', 'bert.encoder.layer.2.attention.self.key.weight', 'bert.encoder.layer.6.attention.self.key.bias', 'bert.encoder.layer.3.attention.output.LayerNorm.bias', 'bert.encoder.layer.4.attention.self.key.weight', 'bert.encoder.layer.5.attention.self.key.bias', 'bert.encoder.layer.11.output.LayerNorm.bias', 'bert.encoder.layer.8.output.LayerNorm.bias', 'bert.encoder.layer.10.attention.self.key.weight', 'bert.encoder.layer.2.output.dense.bias', 'bert.encoder.layer.4.attention.output.LayerNorm.bias', 'bert.encoder.layer.1.attention.self.key.weight', 'bert.encoder.layer.2.attention.output.dense.bias', 'bert.encoder.layer.3.attention.output.dense.bias', 'bert.encoder.layer.4.attention.output.dense.weight', 'bert.encoder.layer.2.attention.self.value.bias', 'bert.encoder.layer.1.attention.output.LayerNorm.weight', 'bert.encoder.layer.4.attention.self.key.bias', 'bert.encoder.layer.7.output.LayerNorm.weight', 'bert.encoder.layer.6.intermediate.dense.weight', 'bert.encoder.layer.9.attention.self.value.bias', 'bert.encoder.layer.5.attention.output.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'bert.encoder.layer.4.attention.self.value.weight', 'bert.encoder.layer.3.attention.self.value.weight', 'bert.encoder.layer.6.output.LayerNorm.weight', 'bert.encoder.layer.11.attention.self.query.weight', 'bert.encoder.layer.8.attention.output.LayerNorm.weight', 'bert.encoder.layer.6.attention.self.key.weight', 'bert.encoder.layer.6.attention.self.value.weight', 'bert.encoder.layer.8.attention.self.key.weight', 'bert.encoder.layer.1.output.dense.weight', 'bert.encoder.layer.10.attention.output.dense.weight', 'bert.encoder.layer.9.intermediate.dense.weight', 'bert.encoder.layer.8.attention.self.key.bias', 'bert.encoder.layer.8.output.dense.weight', 'bert.encoder.layer.3.attention.output.LayerNorm.weight', 'bert.encoder.layer.5.attention.self.query.weight', 'bert.encoder.layer.9.attention.output.LayerNorm.bias', 'bert.encoder.layer.4.attention.self.value.bias', 'bert.encoder.layer.10.attention.output.LayerNorm.weight', 'bert.encoder.layer.10.attention.self.query.weight', 'bert.encoder.layer.11.attention.self.key.bias', 'bert.encoder.layer.1.attention.self.value.weight', 'bert.encoder.layer.3.output.dense.bias', 'bert.encoder.layer.1.attention.self.query.bias', 'bert.encoder.layer.5.output.LayerNorm.bias', 'cls.predictions.transform.dense.bias', 'bert.encoder.layer.1.attention.self.key.bias', 'bert.encoder.layer.2.attention.self.key.bias', 'bert.encoder.layer.9.output.dense.weight', 'bert.encoder.layer.2.attention.output.LayerNorm.weight', 'bert.encoder.layer.4.attention.output.LayerNorm.weight', 'bert.encoder.layer.3.attention.self.query.weight', 'bert.encoder.layer.6.attention.output.dense.bias', 'bert.encoder.layer.2.output.dense.weight', 'bert.encoder.layer.11.attention.self.value.weight', 'bert.encoder.layer.1.intermediate.dense.weight', 'bert.encoder.layer.6.intermediate.dense.bias', 'bert.encoder.layer.7.attention.self.key.weight', 'bert.encoder.layer.4.output.dense.weight', 'bert.encoder.layer.4.output.LayerNorm.bias', 'bert.encoder.layer.11.attention.output.dense.bias', 'bert.encoder.layer.6.output.dense.bias', 'bert.encoder.layer.7.attention.self.query.bias', 'bert.encoder.layer.1.attention.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.key.weight', 'bert.encoder.layer.5.intermediate.dense.bias', 'bert.encoder.layer.4.intermediate.dense.weight', 'bert.encoder.layer.1.intermediate.dense.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
/home/cjh/anaconda3/envs/deepke/lib/python3.8/site-packages/torch/nn/modules/rnn.py:62: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.3 and num_layers=1
warnings.warn("dropout option adds dropout after all but last "
[2024-06-06 16:53:38,764][__main__][INFO] - model name: lm
[2024-06-06 16:53:38,764][__main__][INFO] -
LM(
(bert): BertModel(
(embeddings): BertEmbeddings(
(word_embeddings): Embedding(21128, 768, padding_idx=0)
(position_embeddings): Embedding(512, 768)
(token_type_embeddings): Embedding(2, 768)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(encoder): BertEncoder(
(layer): ModuleList(
(0): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
)
)
(pooler): BertPooler(
(dense): Linear(in_features=768, out_features=768, bias=True)
(activation): Tanh()
)
)
(bilstm): RNN(
(rnn): LSTM(768, 50, batch_first=True, dropout=0.3, bidirectional=True)
)
(fc): Linear(in_features=100, out_features=51, bias=True)
(dropout): Dropout(p=0.3, inplace=False)
)
[2024-06-06 16:53:38,848][__main__][INFO] - "1957年12月6日" 和 "浙江省宁波市" 在句中关系为："出生日期"，置信度为0.97。您好，这两个应该都是模型出错，您可以根据简单规则过滤掉这类信息，比如地名和日期不可能有出生于关系。
zxlzr
commented
5 months ago 请问您还有其他问题吗？
Daniel-ChenJH
commented
5 months ago 暂时没有了，感谢！我后面试试oneke去
1223243
commented
4 months ago 请问一下，你知道怎么使用vscode调试这个代码吗
你好，我按here中的指示完成了DeepKE-cnSchema开箱即用版的环境配置。对于一些简短的句子，往往只有两个实体、关系词明确的情况下，程序可以正确预测其中的关系。但是对于稍长、实体略多的情况，这个关系抽取给出的结果比较离谱，而且置信度还挺高。
这里是我predict.yaml的内容，其中两个模型都用的是readme中Bert-wwm的模型。
这里句子稍长，我们来看看模型得到了什么，下面是log：
可以看到，这里实体提取分割的结果很好，准确的完成了五个实体（{'林忠钦': '人物', '汉族': 'Text', '1957年12月6日': 'Date', '浙江省宁波市': '地点', '中国': '国家'}）的分割。但是关系抽取很离谱，到后面输出“ "浙江省宁波市" 和 "中国" 在句中关系为："出生地"，置信度为0.84。”、“"汉族" 和 "浙江省宁波市" 在句中关系为："民族"，置信度为0.55。”、“"1957年12月6日" 和 "浙江省宁波市" 在句中关系为："出生日期"，置信度为0.97。”等等，详见log，并且这些错误的关系提取有时候还有非常高的置信度，甚至有点像将实体胡乱组合的结果。
我不知道怎么样才能让这个模型对于关系预测更加准确、不会出现这些奇怪的关系，有人可以提供任何帮助吗？感谢！