KGQA / leaderboard

You can find the most recent KGQA benchmark numbers from publications here.
https://kgqa.github.io/leaderboard/
MIT License
111 stars 16 forks source link

systems.md vs. systems_updated.md #275

Open Artur-Galstyan opened 4 months ago

Artur-Galstyan commented 4 months ago

Hi everyone, sorry for the late response! Xi had recently messaged me about the difference between the systems.md file and the systems_updated.md file. So, I thought I'd clarify it here.

(bear in mind though that the project was 9 months ago for me and some details have become...foggy)

The former systems.md file was a malformed table. For a few rows, the delimiter | was missing, which caused certain rows to "overflow" into the next row. But I didn't want to delete the file, so instead I went through the entire file, fixed all the formatting issues as well as fill out missing data if there was some.

That resulted in the systems_updated.md file. This is also the one that is being rendered on the frontend. You can verify it here:

Click me!

async function loadFromGithub(db: string | null, dataset: string | null, systems = false) {
    let mainPath = systems ? `systems_updated.md` : `${db}/${dataset}.md`;
    let headerPath = systems ? `^systems_updated.md` : `${db}/^${dataset}.md`;
    let footerPath = systems ? `$systems_updated.md` : `${db}/$${dataset}.md`;

I also noted this in the tutorial here.

I'm sorry if this resulted in confusion in the team.

Cheers :)