search

yuanqing-wang / rum

Random walk unit with memory
MIT License
16 stars 3 forks source link

Reproducability Issue #18

Open Zehong-Wang opened 1 week ago

Zehong-Wang commented 1 week ago

I have been working with the Cora dataset and using the script node_classification/tune.py to tune the hyperparameters. My results consistently range between 76% and 81%, and I would appreciate any guidance on the hyperparameter settings used to achieve the reported 84.1% performance.

yuanqing-wang commented 1 week ago

Hi @Zehong-Wang , Thanks for reaching out! How many sample trials did you run with that script? The test metric was obtained by completing the tuning experiment of many trials and selecting the best one according to the validation metric.

Zehong-Wang commented 1 week ago

Thank you for the reply. I just naively follow the provided script in script/node_classification/tune.sbatch. I think there are 100 trials by default.