Closed stanlo229 closed 2 years ago
The purpose of our benckmarks in the paper is to prove that in a fair situation, that is, all methods use a same group of splited dataset, our method is better than the previous method.
Different group of splited datasets will lead to different benchmark scores, so if you can't reach the score in our paper in one run, please test a few more groups of splitted dataset. Then you can find some groups can reach and exceed the benchmarks scores in our paper.
Last but not least, we only seleted the splits that the distrbutions of train-valid-test sets is closed.
We considered 100/200/300 configurations initialized depending on the dataset.
Hi Yuquan Li, Thanks for your previous reply. I had another question: What are the configurations to reproduce your results in the paper? I.e. when I run
python3 glam.py --n_init_configs ...
What do I set for each dataset?