search

ma-compbio / Hyper-SAGNN

hypergraph representation learning, graph neural network
MIT License
86 stars 21 forks source link

Question about static embeddings #11

Closed yuzhenmao closed 1 year ago

yuzhenmao commented 1 year ago

Hi, thank you for your work! I notice here: https://github.com/ma-compbio/Hyper-SAGNN/blob/be978daade0305798c45f27d3ba3c66a5f4c5a59/Code/Modules.py#L612 The output is static1, but later, the input is static, which then overwrites static1: https://github.com/ma-compbio/Hyper-SAGNN/blob/be978daade0305798c45f27d3ba3c66a5f4c5a59/Code/Modules.py#L615 Is this correct? If so, the output static1 is redundent.

Thank you for your help!

ruochiz commented 1 year ago

That is correct. We later on did more ablation studies on additional datasets and go with this choice (i.e. skip some layer normalization + MLP in the self.mul_head_attn for static embeddings. ). But because I still want to keep the option of using the static1 from the self.mul_head_attn, so instead of changing that function, I just chose to overwrite the variable. Sorry about the confusion.

yuzhenmao commented 1 year ago

Makes sense to me and thank you for your reply!