search

KGQA / leaderboard

You can find the most recent KGQA benchmark numbers from publications here.
https://kgqa.github.io/leaderboard/
MIT License
105 stars 16 forks source link

add Complex Question Answering Over Temporal Knowledge Graphs #94

Closed xixi019 closed 1 year ago

xixi019 commented 1 year ago

Complex Question Answering Over Temporal Knowledge Graphs

huntila commented 1 year ago

The paper doesn't introduce any new dataset.

RicardoUsbeck commented 1 year ago
image

But it has new numbers for existing datasets.

huntila commented 1 year ago

Shall I merge the pull request https://github.com/KGQA/leaderboard/pull/108 ?

RicardoUsbeck commented 1 year ago

No, a new pull request per issue is recommended

DavidBlavid commented 1 year ago

relevant pull request: https://github.com/KGQA/leaderboard/pull/167

LongquanJiang commented 1 year ago

@DavidBlavid Are these numbers correct? I cannot find them in this pull request.

image
DavidBlavid commented 1 year ago

Because we do not differentiate between before/after, first/last and time join in cron_questions.md, i calculated the averages for the numbers. So for AE-TQ (0.542 + 0.762 + 0.737)/3 = 0.680 for Hits@1 and (0.701 + 0.793 + 0.767)/3 = 0.754 for Hits@3.

Please let me know if this is correct or if i should change it

LongquanJiang commented 1 year ago

Should we add a footnote for it? @xixi019

xixi019 commented 1 year ago

Because we do not differentiate between before/after, first/last and time join in cron_questions.md, i calculated the averages for the numbers. So for AE-TQ (0.542 + 0.762 + 0.737)/3 = 0.680 for Hits@1 and (0.701 + 0.793 + 0.767)/3 = 0.754 for Hits@3.

Please let me know if this is correct or if i should change it

I replied in this comment: https://github.com/KGQA/leaderboard/pull/167#discussion_r1228373058

DavidBlavid commented 1 year ago

I have created a new pull request based on the reviews: https://github.com/KGQA/leaderboard/pull/174