Open yhshu opened 3 years ago
In my opinion, the best way to evaluate your system would be to execute the gold SPARQL query (the one given in the dataset) and compare it with the answers retrieved by your system.
Purely from the structure/template of the SPARQL, there might be literals as answers. However, we removed all SPARQL and its associated question while creating this dataset, which returned a literal.
I’m working on a KBQA system on LC-QuAD 1.0. I would like to ask some questions about the evaluation. Should I include the literal in golden answers when I execute the golden SPARQL against DBpedia 1604? I noticed that all lambda variables in golden SPARQLs are ‘?uri’. Since there’s no FILTER in SPARQL for literals, does it mean answers are just URIs? Thanks.