search

CRIPAC-DIG / AUG-MAE

Code for AAAI'24 paper "Rethinking Graph Masked Autoencoders through Alignment and Uniformity”.
12 stars 1 forks source link

Question about formula 9 #2

Open RX28666 opened 1 week ago

RX28666 commented 1 week ago

Thank you for your fascinating work! I have several question this paper:

  1. Regarding the training of the mask generator, in Formula 9, does the probability represent the binary classification of whether a specific node will be masked or not? If so, is the purpose of using Gumbel-Softmax just to generate discrete mask vectors (0 or 1) while maintaining differentiability, otherwise the mask vector can be decided directly by the probability.

  2. Besides, I am curious if you try the link prediction task?

  3. Did you try use other GNN model for mask generator? Since GraphMAE highly depends on GAT and it performs worse if other GNN models are used.

  4. Did you try sampling (neighbor sampling or subgraph sampling)?

Thanks.

AzureLeon1 commented 1 week ago

Thanks for your interest in our work! Regarding your questions:

  1. The "prob" in Equation 9 is the probability vector for each node being masked. A binary vector "m" (as shown in Equation 10) is then sampled from this vector using the Gumbel-Softmax, which indicates whether each node is masked. The Gumbel-Softmax is used to make the sampling operation differentiable.
  2. We followed the experimental setup of GraphMAE and did not attempt the link prediction task.
  3. We experimented with different GNN architectures as the mask generator. On most datasets, the GNN architecture of the mask generator we used is consistent with that of the encoder & decoder. You can refer to configs.yml for details.
  4. We did not try these sampling methods.