search

divelab / DIG

A library for graph deep learning research
https://diveintographs.readthedocs.io/
GNU General Public License v3.0
1.82k stars 281 forks source link

Evaluation script used in Comenet on OC20 #173

Closed fmocking closed 1 year ago

fmocking commented 1 year ago

Hi again. Thank you for the great work. How did you evaluate the trained model (including the ID, OOD Ads, OOD Cat, OOD Both subsets) on the validation set using the OCP framework on validation set? Is it possible to share commands/scripts used for this process?

Best regards,

limei0307 commented 1 year ago

Hi @fmocking, to evaluate the model on the validation sets (ID, OOD Cat, OOD Ads, OOD both), you should

  1. update the validation dataset in this line, https://github.com/Open-Catalyst-Project/ocp/blob/main/configs/is2re/all/base.yml#L8, the default validation set is ID,
  2. then use this line to test your performance, python main.py --mode predict --config-yml configs/is2re/all/comenet/comenet.yml --checkpoint checkpoints/[TIMESTAMP]/checkpoint.pt

Thanks