Help Needed for Reproducing `rel-trial` `condition-sponsor-run`

snap-stanford / relbench

RelBench: Relational Deep Learning Benchmark

https://relbench.stanford.edu

MIT License

223 stars 41 forks source link

Help Needed for Reproducing `rel-trial` `condition-sponsor-run` #273

Closed george-1019 closed 2 weeks ago

george-1019 commented 3 weeks ago

I'm running idgnn_link.py to reproducerel-trial task condition-sponsor-run.

python idgnn_link.py --dataset rel-trial --task condition-sponsor-run --lr 0.0001 --aggr mean --num_neighbors 64 --num_layers 4

I got the result of 10.2 for val and test, which seems lower than the reported value. Could you please let me know if the hyperparameters are appropriate? Thanks

rishabh-ranjan commented 2 weeks ago

Hi, for idgnn_link.py on rel-trial we keep the default parameters, except for --num_layers=4.

If you check the script, the defaults are: lr=0.001, aggr=sum, num_neighbors=128

Please let us know if you are able to reproduce the reported results under these settings. Feel free to re-open the issue otherwise.