Compute Head and Tail Prediction on FB15K dataset

[MuhibKhan]

I want to check the Prediction Head and Prediction Tail (Hits@10) on different types of relation like 1-to-1, 1-to-N, N-to-1 and N-to-N of TransD model like the results mentioned in the paper. How can I perform the Head and Tail Prediction on 1-to-1, 1-to-N, N-to-1 and N-to-N of relations of FB15k dataset?

[Edward-Sun]

Hi,

You can follow this nice script to split test data into 1-to-1, 1-to-N, N-to-1, and N-to-N: https://github.com/thunlp/OpenKE/blob/OpenKE-PyTorch/benchmarks/FB15K/n-n.py And set them as your test data.

For prediction head and prediction tail, you can set use test_dataset_list to [test_dataloader_head] or [test_dataloader_tail].

[MuhibKhan]

Hello. I used the script and create the text files of 1-to-1, 1-to-N, N-to-1, and N-to-N. Where I apply this, will you please elaborate it:

prediction head and prediction tail, you can set use test_dataset_list to [test_dataloader_head] or [test_dataloader_tail]?

Head and tail as (test_dataloader_head and test_dataloader_tail) are not in a separate files, so where I can find it?

[Edward-Sun]

Hi, You can

use test_dataloader_head + 1-to-1 and test_dataloader_tail+ 1-to-1 to evaluate 1-to-1 in prediction head and prediction tail,

use test_dataloader_head + 1-to-N and test_dataloader_tail+ N-to-1 to evaluate 1-to-N in prediction head and prediction tail,

use test_dataloader_head + N-to-1 and test_dataloader_tail+ 1-to-N to evaluate N-to-1 in prediction head and prediction tail,

and use use test_dataloader_head + N-to-N and test_dataloader_tail+ N-to-N to evaluate N-to-1 in prediction head and prediction tail.

Therefore, to get all 8 evaluation scores in this table, you will need to run the model for 8 times.

[MuhibKhan]

I understand a bit but having one confusion. what is test_dataloader_head and test_dataloader_tail?

Where I can find it? or I have to create it from train2id by separating all heads and tails?

[Edward-Sun]

Hi, it's in this line: https://github.com/DeepGraphLearning/KnowledgeGraphEmbedding/blob/2442e8b9bb1a40e44edabcb3cc4f145ac24b1c7b/codes/model.py#L374

[MuhibKhan]

@Edward-Sun Thanks Sir.