ngl567 / RPJE

AAAI 2020: Rule-Guided Compositional Representation Learning on Knowledge Graphs
44 stars 7 forks source link

About the paper. #2

Closed Zawo1995 closed 4 years ago

Zawo1995 commented 4 years ago

Dear author,I have some questions about the paper“Rule-Guided Compositional Representation Learning on Knowledge Graphs”.

  1. About the Loss L3.Why the confidence level denoted as β only with positive energy,the paper say " The confidence levels of all the rules are considered to be penalty coefficients in optimization".Can you explain it for me. image

2.When Evaluate the model,the paper didn't use the Third Energy E3 as Triple Score,Why? image

Looking forward to your response. XD

ngl567 commented 4 years ago

Hello, they are good questions.

  1. Actually, in L3, the confidence of Rules R1 β can be regarded as the Penalty coefficient. That is to say, the energy function E_3(r, r_e) consisting of a relation pair which are associated by a rule with lower confidence will contribute less for the loss function, and then it plays a smaller role in the process of parameters update by SGD. Wheras, the energy function E_3(r, r') is not generated by rules. Thus, β is employed for ensuring the effectiveness of using rules in our model.
  2. Because the third energy function is used to evaluate the similarity between two relations, you can just view loss function L_3 corresponding to E_3 as a regularization term in the optimization. So E_3 is only used in optimization procedure but not in testing.
Zawo1995 commented 4 years ago

Great job!Thanks for your patient explanation.

Zawo1995 commented 4 years ago

Sorry for reopening the issue. I found the code in https://github.com/ngl567/RPJE/blob/596e3e146ef4893d50ee2df19af25207d2a83f83/Train_RPJE.cpp#L412 didn't use rel_pconfi (I thought it was the β).Could you explain why?

ngl567 commented 4 years ago

Thank you for pointing out this issue. Because there are too many versions of codes and I maybe uploaded an incomplete one. Now, I have fixed this issue. Please see the current version of the code.

Zawo1995 commented 4 years ago

Thanks for responsing so soon. And sorry to consult you my issue again. I guess the “while”hear is to find the neg_rel which is not satisfied R1and then gradient the negative sample,but in paper the r' just from T',no above mentioned constraint,this point mismatch the code hear. https://github.com/ngl567/RPJE/blob/8101856cb9c89243474e0583792ba9c2e713649c/Train_RPJE.cpp#L415 image Could you explain why?

ngl567 commented 4 years ago

Thank you for this question. In our codes, we indeed randomly sample a rel_rneg which cannot match any R1 rule for constraining the relation embeddings.

Zawo1995 commented 4 years ago

Thanks a lot! I think I know what you mean and the “while” here is to find the rel_neg. But what confuse me is the equation[7] form paper, Sorry for my poor English,I have to use Chinese to express my confusion here.

就,那个循环我知道是要找一个负关系r',这个r'不满足与关系rel_a的任何一条R1规则,这点,我理解。但是很奇怪的是,论文的公式7,它写的(r')∈T-这个很奇怪,因为公式8说T-是一个三元组的负例集合,而r'只是个关系啊,并且并没有代码里写的那么多的限制。

ngl567 commented 4 years ago

你说的很对,论文里给出的确实是负采样的方法,实际代码里会在负采样的基础上加上对不满足R1规则的关系的约束。