KGQA / leaderboard

You can find the most recent KGQA benchmark numbers from publications here.
https://kgqa.github.io/leaderboard/
MIT License
108 stars 16 forks source link

Add KSEM papers #52

Closed RicardoUsbeck closed 2 years ago

RicardoUsbeck commented 2 years ago
LongquanJiang commented 2 years ago

Added https://link.springer.com/chapter/10.1007/978-3-031-10983-6_15 https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11

xixi019 commented 2 years ago

@RicardoUsbeck https://link.springer.com/chapter/10.1007/978-3-031-10983-6_39 This paper is only tested on SimpleQuestionFB2M. Me and Longquan has disagreement about whether to add this dataset to our leaderboard. Add: 1. This dataset is still valid dataset and a benchmark. Not: 1. This dataset is dependent on FB2M, which is no longer used.

LongquanJiang commented 2 years ago

Not:

  1. it is a dataset for simple questions. few recent qa studies used it to evaluate their method.
  2. For Simple QA systems, their performance already reached to 100%, it is impossible to have any further improvement on it.
xixi019 commented 2 years ago

https://link.springer.com/chapter/10.1007/978-3-031-10983-6_12 is added by #61

RicardoUsbeck commented 2 years ago

I am fine with not adding the FB2M results

xixi019 commented 2 years ago

Ok, then all 4 papers apart from https://link.springer.com/chapter/10.1007/978-3-031-10983-6_39 (dependant on FB2M) are added. We can close this issue now

RicardoUsbeck commented 2 years ago

Link to merge pull?

xixi019 commented 2 years ago

https://link.springer.com/chapter/10.1007/978-3-031-10983-6_15 --> https://github.com/KGQA/leaderboard/pull/59 https://link.springer.com/chapter/10.1007/978-3-031-10983-6_11 --> https://github.com/KGQA/leaderboard/pull/59 https://link.springer.com/chapter/10.1007/978-3-031-10983-6_12 --> https://github.com/KGQA/leaderboard/pull/61 https://link.springer.com/chapter/10.1007/978-3-031-10983-6_39 : not added I have also tried to link in the pull request, somehow it does not show here

RicardoUsbeck commented 2 years ago

https://github.com/KGQA/leaderboard/pull/61 https://github.com/KGQA/leaderboard/pull/59