search

Shen-Lab / GraphCL

[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
MIT License
541 stars 103 forks source link

Question about the unsupervised_TU gsimclr.py #40

Closed scottshufe closed 2 years ago

scottshufe commented 2 years ago

Hi GraphCL team, thanks for your excellent work.

I have some questions about the loss function in unsupervised_TU/gsimclr.py: in your article eq.(3), you claim that the positive pairs and negative pairs are composed of augmentations, but in the code I find that you use the original sample and one augmentation to form a positive pair. I don't understand the differences between these two methods and wish you could give me some suggestions or explanation. Thank you in advance.

yyou1996 commented 2 years ago

Hi @scottshufe,

Sorry for the late reply. Double augmentations are implemented for experiments except unsupervised_TU due to some implementation issue then (e.g. please refer to https://github.com/Shen-Lab/GraphCL/tree/master/semisupervised_TU).

scottshufe commented 2 years ago

Hi, Mr. You @yyou1996. Thanks for your reply. I think I have implemented the code of double augmentations, but I need to figure out the differences between single augmentation and double augmentations...If you have any ideas on this question, I would love to hear your opinion :smile:

CynthiaLaura6gf commented 2 years ago

Thanks for your excellent codes, I have some problems with the loss function. what's the difference between the loss function in the class simclr and the loss function in the class of GcnInfomax.

scottshufe commented 2 years ago

Thanks for your excellent codes, I have some problems with the loss function. what's the difference between the loss function in the class simclr and the loss function in the class of GcnInfomax.

I think the loss in GcnInfomax is the same as DGI (2019 ICLR) which generalizes DeepInfomax (2019 ICLR) from images to graphs. It aims to maximize the mutual information between local patches and the global graph. And GraphCL (or simclr) aims to maximize the mutual information between the augmentations.

yyou1996 commented 2 years ago

@scottshufe I feel in small datasets it differs little, while things might change in large-scale datasets. The positive or negative influence depends on whether the augmentation is rational for the downstream.

scottshufe commented 2 years ago

Got it. Thanks again, Mr. You. Maybe I should read more GCL articles and run some experiments to better understand these questions.