Some questions about the results

[anyingduxiao]

Hello, thank you very much for your contribution, I have some questions here, I browsed the web322log file you provided, the final result is： 2022-10-14 21:09:33,881 - root - INFO - Train Done! Evaluate on testset with saved model 2022-10-14 21:09:48,012 - root - INFO - Best h1 evaluation 2022-10-14 21:09:48,012 - root - INFO - TEST F1: 0.6879, H1: 0.7529 2022-10-14 21:10:02,328 - root - INFO - Best f1 evaluation 2022-10-14 21:10:02,328 - root - INFO - TEST F1: 0.7122, H1: 0.7578 2022-10-14 21:10:16,537 - root - INFO - Final evaluation 2022-10-14 21:10:16,537 - root - INFO - TEST F1: 0.7166, H1: 0.7596 and the results provided by your thesis： f1:70.9 h1:76.4 There are some differences, but what outcome do you end up using, how should you use this result, whether it is the best of the three results or the final round, or the average of the three Hope to get your answer, thank you

[cmavro]

Hi,

Thank you for your interest in our work. Please see a relevant discussion here.

Ideally, the best-h1 evaluation should be used for H@1 and best-f1 evaluation for F1. The problem with Webqsp is that it has only 500 validation examples, which might not be ideal. The scores in the paper were obtained with a different run -- but I provided the other log file for reproduction.

[anyingduxiao]

Thanks for the reply You mean to get two different results h1 and f1 on different parameter sets, but I think this should probably be to get both indicators on the same parameter set, for example, I value the result of h1 more, then I will use h1 the best result, and f1 is the result that matches this h1, instead of using two different parameter sets to get two best f1 and h1, hoping to get your reply.

[cmavro]

That is also an alternative.

[anyingduxiao]

ok,thanks.