search

pykeen / benchmarking

📊 Results from the reproducibility and benchmarking studies presented in "Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" (http://arxiv.org/abs/2006.13365)
MIT License
35 stars 4 forks source link

Remove default setting for 'inverse_triples' #12

Closed mali-git closed 4 years ago

mali-git commented 4 years ago

Not every 'study.json' file has the entry 'create_inverse_triples', e.g.: ablation/results/rotate/wn18rr/random/adam/2020-04-25-19-04_217bcf38-2101-461b-9593-b133b4201e6a/0000_wn18rr_rotate/study.json

But a default value is set:

https://github.com/mali-git/pykeen_experimental_results/blob/cd2f6a3a58faf8ffdfaf049b1d0755984fef473d/ablation/collate.py#L117

which distorts the configuration. The correct information can be extracted from 'pipeline_config.json' which is contained in the 'best_pipeline' directory

cthoyt commented 4 years ago

Is this something that still needs addressing?