Open pkulwj1994 opened 3 years ago
Hi again,
In theory, the layer might become non-invertible, but it will be very unlikely to happen (won't really happen in practice).
If you get an error I would suspect that this is due to NANs appearing somewhere else which tends to crash the slogdet
function.
One possibility is of course parameterize the matrix differently to ensure that it is truly impossible for it to become non-invertible.
Hope that helps, and thanks for your question!
I am reading and reimplementing your great codebase, Didrick. This repo is such a nice work on Normalizing Flow generative experiments!
In vanilla implementation of bijection transform conv1x1, I find that the conv1x1 kernel (weight) is not strictly invertible with guarantee, which may also lead the torch.slogdet() function to diverse(use of svd for non-invertible kernel matrix) and result in Error. Same problem also arise from Glow paper but author does not give refinement method for non-invertible kernel. My question is: have you any refinements of conv1x1 kernel, just like refinements planar flow and sylvester flow gives to force transform to be invertible all the time. I think the use of LU or Householder tricks for kernel will retrict the power of Conv1x1 transform and thus not applausive.
Codes for Conv1x1 layer is copied below:
Great Thanks Didrik!