search

kelvin-jiang / FreebaseQA

The release of the FreebaseQA data set (NAACL 2019).
Creative Commons Attribution 4.0 International
70 stars 1 forks source link

About the associated Freebase #2

Closed lanyunshi closed 5 years ago

lanyunshi commented 5 years ago

Hi,

Thanks for your kind release of this dataset! I'm very interested in this dataset and so far I'm working on it. But I found for some questions, the SPARQL queries can't retrieve the correct answers and the inference chain is not semantically consistent with the questions' meaning. For example, the question FreebaseQA-dev-6'' isIn which country were the 1948 Winter Olympics held?'' and the inference chain is m.0blfl olympics.olympic_games.athletes..olympics.olympic_athlete_affiliation.country'' orm.0blfl olympics.olympic_games.participating_countries''. However, neither of these two relation paths expresses the accurate meaning of the question. And these two relation paths in Freebase returns a bunch of entities besides ``m.06mzp'', which leads to the F1 < 1. (I retrieved the query within the entire Freebase instead of the associated one.)

Does this mean that the dataset must be used together with the associated subset of Freebase, so that only ``m.06mzp'' is retrieved?

Best, Yunshi

kelvin-jiang commented 5 years ago

Hi Yunshi,

Thanks for your feedback. Regarding the specific inferential chain you mentioned, it seems that that one was labelled incorrectly. These triples were annotated by hand, so from time to time, there will be some inferential chains that are not completely accurate. Sorry about that.

Regarding your second question, Freebase is set up so that the relation paths are not always unique. For example, for your node m.0blfl, which is referring to the 1948 Winter Olympic Games, all countries participating at those games would be connected through the olympics.olympic_games.participating_countries predicate.

Kelvin

lanyunshi commented 5 years ago

I see, thank you for your kind explanation! I will close this issue :)