some questions

[zhangxiaowei5346]

hello, I have a few questions for you and I look forward to your answers: Q1: Your referred neighborhood are edges that connect to the source vertex. Why is this set up? Q2: why fb15k237 does not use the MLM loss？

[sanxing-chen]

Thanks for your interest! For Q1, currently, our work uses a 1-hop neighborhood to provide context to the source vertex, this is simple but effective. Indeed, a multihop neighborhood could provide more information, but the larger body of noisy information also poses more challenges. So we leave it to future work. For Q2, we did this based on the empirical result that adding the MEM loss on the FB15k237 dataset didn't improve performance. The MEM task emphasizes the information in the source vertex, but in dense KGs like FB15k237, the information in the neighborhood coulda be more helpful for link prediction.

[zhangxiaowei5346]

Thanks for your interest! For Q1, currently, our work uses a 1-hop neighborhood to provide context to the source vertex, this is simple but effective. Indeed, a multihop neighborhood could provide more information, but the larger body of noisy information also poses more challenges. So we leave it to future work. For Q2, we did this based on the empirical result that adding the MEM loss on the FB15k237 dataset didn't improve performance. The MEM task emphasizes the information in the source vertex, but in dense KGs like FB15k237, the information in the neighborhood coulda be more helpful for link prediction.

thanks for your reply！ another question，have you encountered any loss and collapse in the experiment? What do you think of this problem?

[sanxing-chen]

In small datasets and when using small-size models, such issues are rare. The optimization of transformers sometimes requires tricks, usually adjusting the learning rate could help.