Questions about experimental results

[yeondukim]

Hi @yuyangw , thanks for sharing your nice works! :)

I've pre-trained the mix-aug GIN model from scratch, and got the finetuned results on QM7 database.

In supported yaml files, the number of max epoch is set to 100. BTW, when I checked tensorboard logs, the training seemed not to be sufficient. So I've tried to finetune the models for 1k epochs on QM7, and got MAE results as 63.4±0.89 for 3 runs (c.f., 87.2±2.0 in the paper).

Target:  u0_atom Test loss: 0.2730964422225952 Test MAE: 62.36752 Best Valid metric: 80.150856 @ epoch: 400
Target:  u0_atom Test loss: 0.2805113196372986 Test MAE: 64.06087 Best Valid metric: 80.38328 @ epoch: 385
Target:  u0_atom Test loss: 0.2787664532661438 Test MAE: 63.66238 Best Valid metric: 82.99878 @ epoch: 295

(pick the model when the valid metric shows minimum value) Have you ever experienced about finetuning the models for more epochs?

Here are the detailed configuration for finetuned results: (trained on Tesla V100-SXM2-32gb)

batch_size: 4096
dataset:
  data_path: data/qm7/qm7.csv
  num_workers: 4
  splitting: scaffold
  target: u0_atom
  task: regression
  test_size: 0.1
  valid_size: 0.1
epochs: 1000
eval_every_n_epochs: 5
fine_tune_from: ckpt/pubchem_Apr25_12-12-28/checkpoints/model.pth
fp16_precision: false
gpu: cuda:0
init_base_lr: 5.0e-05
init_lr: 0.0005
log_every_n_steps: 50
model:
  drop_ratio: 0.3
  emb_dim: 300
  feat_dim: 512
  num_layer: 5
  pool: mean
  pred_act: softplus
model_type: gin
task_name: qm7
weight_decay: 1e-6

Thanks in advance! :)

Sincerely, YKim

[yuyangw]

Hi YKim,

Thanks for your interest in our work!

It is possible that training for more epochs leads to better results. In our work, we set the maximum epoch as 100 due to the limitation of computational resources and consistency with supervised GCN/CIN baselines.

Hope this helps.

Best, Yuyang

[yeondukim]

Thank you for your sincere reply! :) It could be helpful.

YKim