search

lrjconan / GRAN

Efficient Graph Generation with Graph Recurrent Attention Networks, Deep Generative Model of Graphs, Graph Neural Networks, NeurIPS 2019
MIT License
469 stars 96 forks source link

Loss with multiple permutations #10

Closed adamoyoung closed 4 years ago

adamoyoung commented 4 years ago

Looking at the calculation of the loss in the function mixture_bernoulli_loss, it seems like the the loss for multiple orderings of the adjacency is summed. According to the paper, the goal is to optimize $$\log( p(G, \pi_1) + p(G, \pi_2) )$$ but it appears as if what is actually being optimized is $$\log( p(G, \pi_1) p(G, \pi_2) ) = \log( p(G, \pi_1) ) + \log( p(g, \pi_2) )$$. Is this intended? The second expression is a lower bound on the first but it was not explicitly mentioned in the paper afaik.

lrjconan commented 4 years ago

Looking at the calculation of the loss in the function mixture_bernoulli_loss, it seems like the the loss for multiple orderings of the adjacency is summed. According to the paper, the goal is to optimize log(p(G,π1)+p(G,π2)) but it appears as if what is actually being optimized is log(p(G,π1)p(G,π2))=log(p(G,π1))+log(p(g,π2)) . Is this intended? The second expression is a lower bound on the first but it was not explicitly mentioned in the paper afaik.

Thanks for your interest! The code implements what has been described in the paper. The summation inside the log is implemented by this line.