search

facebookresearch / KILT

Library for Knowledge Intensive Language Tasks
MIT License
910 stars 91 forks source link

Additional KB fields #37

Open daltonj opened 3 years ago

daltonj commented 3 years ago

I would like to request an extension to the KB structure, closer to the one released as part of the SLING Wikiflow or TREC CAR and/or data pipelines: https://github.com/google/sling/blob/master/doc/guide/wikiflow.md http://trec-car.cs.unh.edu/datareleases/v2.4-release.html

In particular, access to precomputed inlinks and incoming anchor text for a page, the occurrence of the page coming from disambiguation pages as well as redirects. These all provide rich name variety and signals commonly used in entity retrieval and linking tasks.

It seems like they could be taken from the output of WikiFlow or the CAR pipeline if the same wiki dump is used.

Are there plans to add/support these rich fields?

fabiopetroni commented 3 years ago

Hey @daltonj, many thanks for your message and request. It would be awesome to have all the info you listed in the json. Any chance you would be happy to help? :)