Closed LeavesLei closed 3 years ago
Hi,
You can do that but it will hurt the performance in some way as there will not be any regularisations or constraints induced and parameters will be more free. In many cases, over time KL becomes 0 or too high and there are ways to counter that like KL annealing.
Hi, I want to know whether I can drop the KL term, i.e., set beta=0 in the loss function. Will it hurt the training process? Does It still belong to Bayes by Backprop? Thx and looking forward to your reply 😄.