Closed burntcobalt closed 3 years ago
Hi Lewis,
In Flipout (Wen et al. 2018 ) method, the operation is performed separately with the mean and perturbation component as shown below (Eqn(4) in https://arxiv.org/pdf/1803.04386.pdf). I hope this helps to clarify your question.
outputs
outputs = F.conv1d(x,
weight=self.mu_kernel,
bias=self.mu_bias,
stride=self.stride,
padding=self.padding,
dilation=self.dilation,
groups=self.groups)
perturbed_outputs
perturbed_outputs = F.conv1d(x * sign_input,
bias=bias,
weight=delta_kernel,
stride=self.stride,
padding=self.padding,
dilation=self.dilation,
groups=self.groups) * sign_output
outputs + perturbed_outputs
return outputs + perturbed_outputs, kl
Best, Ranganath
Ah, I see. Thank you, Ranganath.
Hi,
Shouldn't mu_kernel be added to delta_kernel in the code below?
Best, Lewis