Closed hyp1231 closed 3 years ago
Hi @hyp1231,
Below is my previous result log
0 0.6695203812316668 0.6639126131844449
1 0.6689575398079525 0.6578154819531342
2 0.688679302806011 0.6857136163297752
3 0.6836998338581497 0.6732908715258754
4 0.69713972798318 0.6849932255983389
5 0.6776616371025179 0.6742576016349859
6 0.678085934406664 0.6687597803949993
7 0.6714889003635183 0.6592118772987136
8 0.6725745492007931 0.6691230880465453
9 0.6807927650869883 0.669925179267206
CUDA version:
NVIDIA-SMI 450.51.05 Driver Version: 450.51.05 CUDA Version: 11.0
Also, I just put the environment file here.
Could it be environmental issue? Lets figure it out. Any update please let me know. Thanks!
@yyou1996 Thank you for the fast and comprehensive reply. I'll try to upgrade the driver of my devices and do experiments as soon as possible.
Please also try lr=1e-2
and lr=1e-4
here. Due to potential environmental issue, a little bit of tuning is supposed to be required.
Thanks for your advice!
When tuning lr=1e-4
, results come to 67.34 ± 1.15, which is close to the results in the literature.
Details of result.log
0 0.6725159200324586 0.66603653940776
1 0.6657212107373389 0.6568356122380342
2 0.6687794432662423 0.6648277830180185
3 0.6985625261307801 0.68720311294303
4 0.6759798551079974 0.6692231385248839
5 0.6765493005622117 0.6736804953394853
6 0.7022692012839785 0.6951517668704265
7 0.6763894721533189 0.6615853768109135
8 0.6853784068489189 0.683059879327427
9 0.6850071688755396 0.6764286988851322
BTW, the experiments are carried out in NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1
. Due to the difficulty of upgrading, I have not tested on CUDA 11.0 yet and still have no idea whether it's an environmental issue. If I find available devices for testing in the future, I'll update the results here.
All in all, thanks so much for the kindly reply and instruction. Just feel free to close this comment. :D
That's great! Good luck with your following experiments!
Hi @yyou1996,
I wonder how to install torch=1.0.1
with cuda=11.0
or cuda=10.1
. I found that the conda way only supports cuda 9.0 and 10.0. Thanks!
# CUDA 9.0
conda install pytorch==1.0.1 torchvision==0.2.2 cudatoolkit=9.0 -c pytorch
# CUDA 10.0
conda install pytorch==1.0.1 torchvision==0.2.2 cudatoolkit=10.0 -c pytorch
Hi @ha-lins,
I try torch.version.cuda
then it outputs 9.0.176
. I will defer this problem to others since I am not expert in environment config...
Hi @yyou1996,
Thanks for the amazing work and released code. They are really interesting.
However, I find that it's hard to reproduce the results of Table. 5 on PPI dataset, where GraphCL gets 67.88 ± 0.85 ROC-AUC. I follow the instruction in README to run the script
cd ./bio; ./finetune.sh
in two versions of PyG.The results and details of
result.log
of my reproducing experiments are as following:ROC-AUC = 63.95 ± 1.05
In this experiment, I had
torch_geometric == 1.0.3, torch == 1.0.1
and all the codes are as original.ROC-AUC = 63.43 ± 0.86
In this experiment, I had
torch_geometric == 1.6.3, torch == 1.7.1
, and I changed the codes a little bit following #14.Results of my experiments are calculated according to the rightest column of
result.log
, which are results oftest_acc_hard_list
following Hu et.al. ICLR 2020 [1]. Just to make sure if I have missed some important details to reproduse the results presented in the literature. Looking forward to your reply, thanks![1] Strategies for Pre-training Graph Neural Networks. Weihua Hu, Bowen Liu et.al. ICLR 2020. arxiv.