Un/self-supervised learning with GraphGym

[moetayuko]

🚀 The feature, motivation and pitch

Dear developers,

GraphGym is a very useful tool for experiment management and a good skeleton for new research projects. However, it seems that the current GraphGym is designed for supervised tasks only. The workflow of unsupervised tasks usually differs from supervised tasks in the following aspects:

• Dataset: full dataset (w/o labels) might be used rather than train/val/test split
• Multi-stage training: different modules might need to be trained with different inputs in the given order
• Objective: Pretrain and inference might use different loss functions
• Evaluation protocol: Finetune, linear probing or other lightweight training procedures might be necessary before obtaining predicted labels for calculating metrics
• ...

Thanks to the modularized design of GraphGym, it is possible to implement unsupervised projects but with many ad hoc hacks. It would be nice if its flexibility can be improved to make unsupervised learning easier.

Alternatives

No response

Additional context

No response

[rusty1s]

Thanks for sharing! This is definitely of interest to us, and we could come up together with a clear roadmap to get there. @JiaxuanYou for additional thoughts.