Collect additional metrics during collation

pykeen / benchmarking

📊 Results from the reproducibility and benchmarking studies presented in "Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" (http://arxiv.org/abs/2006.13365)

MIT License

35 stars 4 forks source link

Collect additional metrics during collation #14

Open mberr opened 4 years ago

cthoyt commented 4 years ago

Ahh its not so easy since there are groupby operations and all of the later scripts rely on the hits@10 being the only column that isn't getting grouped

mberr commented 4 years ago

Hm, so we would need to re-run the collation every time we want to have the plots for a new metric?

cthoyt commented 4 years ago

No, we just need to generalize the pipeline just a bit