Closed gngdb closed 6 years ago
The Linear layer uses this:
KLD_element = -0.5 * self.weight_logvar + 0.5 * (self.weight_logvar.exp() + self.weight_mu.pow(2)) - 0.5
But the convolutional layers use this:
KLD_element = -self.weight_logvar + 0.5 * (self.weight_logvar.exp().pow(2) + self.weight_mu.pow(2)) - 0.5
The second appears to match equation 8 of the paper, so is it just a mistake in the Linear layer?
Ah, appears to only be an issue in my fork (of a fork). Should've read the code before opening this issue, sorry!
The Linear layer uses this:
But the convolutional layers use this:
The second appears to match equation 8 of the paper, so is it just a mistake in the Linear layer?