Generate result table - Githubissues

pykeen / benchmarking

📊 Results from the reproducibility and benchmarking studies presented in "Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" (http://arxiv.org/abs/2006.13365)

MIT License

35 stars 4 forks source link

Generate result table #8

Closed mali-git closed 4 years ago

mali-git commented 4 years ago

Generate result tables with the results of the reproducibility studies which also reflects the variance, e.g., 50.2 +- 0.25

[x] Make TSV files
[x] Make full latex table
[x] Generate PDFs
[x] Solve width problem

mberr commented 4 years ago

a quick & dirty variant can be found at https://github.com/mali-git/pykeen_experimental_results/blob/8e892c5ca7519cffd9edb90100b6a6d03069c25e/reproducibility/summarise.py

cthoyt commented 4 years ago

I made a bit of a cleaner implementation in 5cab604 that outputs tex files and also TSVs. See files in https://github.com/mali-git/pykeen_experimental_results/tree/master/reproducibility/summaries

cthoyt commented 4 years ago

I've done a few variants of this (figured out why the latex wasn't working- needed an include and also the plus minus sign isn't allowed in latex. I replaced that with $\pm$ )

Problem is these tables are huge... even if you break them up to three different ones for avg, best, and worst, it still overflows off the page...

mali-git commented 4 years ago

Maybe we can align the tabels vertically over the whole page?

mberr commented 4 years ago

or \scalebox / smaller font size?

cthoyt commented 4 years ago

I split the tables in half between hits@k and not hits@k. There's a new PDF at https://github.com/mali-git/pykeen_experimental_results/blob/master/reproducibility/summaries/results.pdf