search

liugangcode / GREA

[KDD'22] Source codes of "Graph Rationalization with Environment-based Augmentations"
https://arxiv.org/abs/2206.02886
MIT License
34 stars 5 forks source link

Model design choice #2

Closed rowedenny closed 7 months ago

rowedenny commented 10 months ago

Thanks for release the code for this amazing work. I have two questions about the model design and its optimization:

  1. In this paper, GNN_1 and GNN_2 are two separated modules, I am wondering if they can unified to be one, i.e., MLP_1 directly leverages the intermediate representation from GNN_2?
  2. I notice that the optimization particularly adopts the alternative training, is there any reason for that?

Thank you.

liugangcode commented 9 months ago

Thank you for your question.

  1. Yes, you can unify the representation space for rationalization and prediction. While we haven't tested this idea yet, it could be a promising direction if you observe performance improvements and can find theoretical motivation for it!

  2. Alternative training is more efficient because some parts of the model parameters are disabled for updating. The training process should be flexible, so you can also try the joint training approach. In our experience, we did not observe a significant difference in model prediction accuracy between these two methods.