Permutation Function for FactorVAE

[DexterJZ]

According to the original paper by Kim et al., the permutation function permutes across the batch for each dimension. In the case here, if B, D = z.size(), def permute_latent(self, z: Tensor) should permute z along the dimension of B, i. e., z[i, j] = z[new_indices[i], j], where new_indices = torch.randperm(B).

[mistycheney]

I also think this is an error. The permutation should be applied to the batch dimension, rather than the factor dimension.