search

weihua916 / powerful-gnns

How Powerful are Graph Neural Networks?
MIT License
1.16k stars 223 forks source link

Dropout in last layer #16

Open foxtrotmike opened 3 years ago

foxtrotmike commented 3 years ago

Thank you for your work. You have used dropout prior to computing the output from each layer. What is the role of this dropout? See: https://github.com/weihua916/powerful-gnns/blob/f2626e7ed9c81d40e315f657c9c5c1611e071cc9/models/graphcnn.py#L225

weihua916 commented 3 years ago

It's just a regularization.

foxtrotmike commented 3 years ago

Thank you for your response - Tome, it makes sense to regularize using dropout at layers before the last one. Dropout at the prediction output forces prediction scores of an example to become zero and also scales up the prediction scores of other examples in the batch. Please see: "Furthermore, the outputs are scaled by a factor of \frac{1}{1-p}" in https://pytorch.org/docs/stable/generated/torch.nn.Dropout.html It would be great if you could help me understand this a bit better. Thanks, again.