search

Shen-Lab / GraphCL

[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
MIT License
541 stars 103 forks source link

Contrastive learning implementation in unsupervised_Cora_Citeseer #35

Closed vpozdnyakov closed 2 years ago

vpozdnyakov commented 2 years ago

Hello, I have read the paper "Graph Contrastive Learning with Augmentations" with the appendix, but implementation of contrastive learning in unsupervised_Cora_Citeseer looks a bit unclear for me. Could you please clarify a few details?

  1. What is the discriminator with a bilinear layer in unsupervised_Cora_Citeseer/layers/discriminator.py? What is its purpose?
  2. Why do you use cross entropy instead of contrastive loss in unsupervised_Cora_Citeseer/execute.py? Is it still called contrastive learning?
yyou1996 commented 2 years ago

Hi @vpozdnyakov,

Thanks for your questions. @yongduosui please give some comments on it, thanks!

yongduosui commented 2 years ago

Please check paper DEEP GRAPH INFOMAX[1], this paper maximizing mutual information between patch representations and corresponding high-level summaries of graphs. We also add augmentation graphs information to maximizing mutual information, which is equal to optimize GraphCL loss. You can check and compare the theoretical proof in our paper Appendix section with the paper DEEP GRAPH INFOMAX[1] for more details.

[1] Petar Veliˇckovi´c, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint arXiv:1809.10341, 2018.

vpozdnyakov commented 2 years ago

@yongduosui thanks a lot for comment