About Freebase dataset

[zzyzeyuan]

Hi, I try to reproduce the result in Freebase dataset and notice that it seems like the Freebase dataset used in your paper is different from the dataset in origin HGB paper. The Freebase dataset in your paper is smaller than HGB's, and I think the data distribution is also different.

But in your Freebase setting, I can not reproduce the reported result, could you give me more details about hyper-parameters of Freebase dataset?

Thanks!

[Ffffffffire]

Thanks for your attention. Due to our limited computation resources, we can't conduct experiments on the HGB Freebase. We obtained the Freebase dataset from https://github.com/liun-online/HeCo, a subset of the original Freebase knowledge graph that primarily contains movie-related data. We didn't fix the seed in our experiments and the reported results are high because it was picked from hyperparameter tuning logs. But the performances of HINormer are also competitive. You can fix the train-valid-split and tune the hyperparameters to compare HINormer with other methods. We sincerely apologize for not giving you a convenient experimental experience. Later, we might think about offering a reliable experimental setup. Once more, many thanks.