Reproducing ZINC result with v2

[chjjma]

Hi, First, thanks for the code. I tried to reproduce the result of ZINC subset with graphormer v2 code, but my best validation loss was 0.178 with parameters in the given script with test loss 0.164, and 0.186 with parameters in the paper with test loss 0.157. The latter script's code is as follows:

CUDA_VISIBLE_DEVICES=0 fairseq-train \
--user-dir ../../graphormer \
--num-workers 16 \
--ddp-backend=legacy_ddp \
--dataset-name zinc \
--dataset-source pyg \
--task graph_prediction \
--criterion l1_loss \
--arch graphormer_slim \
--num-classes 1 \
--attention-dropout 0.1 --act-dropout 0.1 --dropout 0.0 \
--optimizer adam --adam-betas '(0.9, 0.999)' --adam-eps 1e-8 --clip-norm 5.0 --weight-decay 0.01 \
--lr-scheduler polynomial_decay --power 1 --warmup-updates 40000 --total-num-update 400000 \
--lr 2e-4 --end-learning-rate 1e-9 \
--batch-size 256 \
--fp16 \
--data-buffer-size 20 \
--encoder-layers 12 \
--encoder-embed-dim 80 \
--encoder-ffn-embed-dim 80 \
--encoder-attention-heads 8 \
--max-epoch 10000 \
--keep-best-checkpoints 1 \
--save-dir ./ckpts

Thanks in advance.

[zhengsx]

Will look into it.

[chjjma]

Do you have any update?

[hyp1231]

Same here. I used the given scripts on ZINC subset, then got an best validation loss 0.1750 and corresponding test MAE 0.1406. I'll continue trying on other hyperparameters.

@chjjma By the way, have you successfully reproduced the results on ZINC subset? Thx!

[Voldet]

Same here. I'm sure I used the original code and there is no problem with the environment configuration. However, the test MAE result I reproduced on the ZINC subset is 0.178.

[HalvesChen]

Same here :(