Hi, I find some bugs of my early implementation when using QDropout.
In the backward, the gradient should also be divided by the 1-p factor.
In the validation step (self.training = False), we can directly use the forward of nn.Dropout since dropout performs different in training and validation steps.
Hi, I find some bugs of my early implementation when using QDropout.
1-p
factor.self.training = False
), we can directly use the forward ofnn.Dropout
since dropout performs different in training and validation steps.Please help check the modification.