Open kaushik333 opened 1 year ago
@cthoyt @mberr could you please help me out?
@kaushik333 I would suggest you start by carefully searching through existing issues and discussions on this tracker - there are other people who have had similar issues we've been able to address.
For future information, we maintain this project in our free time and don't have time to address all issues. Demonstrating you've put in significant effort ahead of time is a good way to get us interested, probably not badgering us.
That being said, I'll suggest you use the same version of PyKEEN that accompanies the paper if you want to make a faithful reproduction. Many improvements have been made in the mean time that affect results.
Describe the bug
I am trying to reproduce Table 9 in the paper (https://arxiv.org/pdf/2006.13365.pdf). I am making using of hyperparameters json file in the folder (https://github.com/pykeen/pykeen/tree/master/src/pykeen/experiments). For example, for conve algorithm and fb15k237 dataset, I use (https://github.com/pykeen/pykeen/blob/master/src/pykeen/experiments/conve/dettmers2018_conve_fb15k237.json).
Using these hyperparameters, I am able to perfectly reproduce ConvE and TuckER algorithms, but I am not able to reproduce the other algorithms. When I mention "reproduce" I mean the row R in this table (not pub -- because the paper clearly says for algos in orange, pub and R do not match).
I might be missing something here, can you please help me out?
How to reproduce
I am actually using raytune to launch experiments parallely:
Environment
Additional information
Code to replicate my environment:
Issue Template Checks