search

danieltan07 / dagmm

My attempt at reproducing the paper Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection
400 stars 106 forks source link

cholesky_cpu: U(2,2) is zero, singular U. #15

Open jiejunnan opened 4 years ago

jiejunnan commented 4 years ago

using my dataset,i have tried many solutions to solve it,but all failed

mil-hasegawa commented 4 years ago

@jiejunnan Hello, this issue comes from that cov_diag degenerates to 0. So, you should set large value of lambda_cov_diag (i.e. from 0.1 to 1.0, default is 0.005).

I also faced the same issue, and I avoided this error after I set lambda_cov_diag = 1.0 with 10 epochs. (when over 20 epochs, loss value goes to nan... in my dataset)

The paper says your issue in p.6

DAGMM also has the singularity problem as in GMM: trivial solutions are triggered when the diagonal entries in covariance matrices degenerate to 0.

Then, the author decided to penalize 1 / cov_k. I hope you will succeed in your dataset.

AmmarKamoona commented 3 years ago

@jiejunnan Hello, this issue comes from that cov_diag degenerates to 0. So, you should set large value of lambda_cov_diag (i.e. from 0.1 to 1.0, default is 0.005).

I also faced the same issue, and I avoided this error after I set lambda_cov_diag = 1.0 with 10 epochs. (when over 20 epochs, loss value goes to nan... in my dataset)

The paper says your issue in p.6

DAGMM also has the singularity problem as in GMM: trivial solutions are triggered when the diagonal entries in covariance matrices degenerate to 0.

Then, the author decided to penalize 1 / cov_k. I hope you will succeed in your dataset.

Hi there,

I have the same problem on my dataset. the latent dimension I use is high 8, 16.

mil-hasegawa commented 3 years ago

In my experience, a high latent dimension tends to result in this error. I recommend using one or two dimensions for latent space to avoid this error.
According to the paper, the authors used one dimension for latent space.

Fig.1 in the original paper shows the relationship between one dimension latent feature and reconstruction error. Before applying this model, you could draw the same figure( one-dim latent feature and reconstruction error). If you find no difference or cannot train an autoencoder-based model, you should not take this model.

Thanks for reading.