search

lukasruff / Deep-SVDD-PyTorch

A PyTorch implementation of the Deep SVDD anomaly detection method
MIT License
698 stars 197 forks source link

Center of volume #21

Open oattao opened 4 years ago

oattao commented 4 years ago

We don't need to update value of c (center of volume) during the training?

snowood1 commented 4 years ago

We don't need to update value of c (center of volume) during the training?

I think in the code he didn't update R ( in deepSVDD_trainer.py)

oattao commented 4 years ago

Do you think that he was wrong?

snowood1 commented 4 years ago

Do you think that he was wrong?

Yes. I think this pytorch version is incomplete compared to the orginal version written in Theano. In this version, one class training and block coordinate optimization is not finished.

jhl13 commented 4 years ago

Do you think that he was wrong?

Yes. I think this pytorch version is incomplete compared to the orginal version written in Theano. In this version, one class training and block coordinate optimization is not finished.

deepSVDD_trainer.py line 94 self.R.data = torch.tensor(get_radius(dist, self.nu), device=self.device)the

I think R is updated. However, according to the paper, the dist should be sorted, but I don't find any code to do this.

jhl13 commented 4 years ago

We don't need to update value of c (center of volume) during the training?

Yes, We don't need to update the value of c (center of volume) during the training.

BenedictGreen commented 4 years ago

Hi, thanks for the discussion. I also noticed the same problem. The paper said that "we found that fixing c in the neighborhood of the initial network outputs made SGD convergence faster and more robust ---(in 3.3 Proposition 1)".I think that mean that 'center' don't need to update.

Was that mean that regard 'center' as a constant made no difference to the result. In other words, theoretically define the 'center' as any constant which satisfy the right shape could also made the model work.

AM I right?

GabrielDornelles commented 2 years ago

As you first pre-train the model with an autoencoder, you have pretty good representations in the latent space. When you start training the Encoder only part, you could find two scenarios:

  1. You only have good samples of your class, and thus, taking the mean of this points in the latent space will just do it, since you want this C point to be close to every other point of good sample.
  2. You have good and anomalous samples, and averaging them all will converge the center C to the most represented type in the dataset (like you can see some specific views of planes in cifar-10, or specific shapes of numbers in mnist).

So, no, I don't see any reason why you would update the center during training, but the radius on the other hand, is what makes the edge between anomalous or not, and it is updated during training.

But I don't think you cant initialize it anyway, they initialize and keep it as the mean of one forward pass on your training data.

ZPP2000 commented 1 year ago

Hi,I have a question about this code .when there is no pretraining process, what should i do about Initializing the center of the sphere?