About the clustering loss

[1173206772]

During my training, the clustering loss did not drop(remain the same),while the loss of CNN remained decreased . I don't know whether this result is correct, and whether the clustering loss will decrease during training. And since the clustering results are used as pseudo-labels to train the network, does it mean that the best results of the trained network can only achieve the effect of the clustering algorithm? I hope you can answer my questions, very grateful ！！

[1173206772]

The data set I used for training is very small, only a few thousand.

[GBoy2509]

I'm in the same situation as you, and the loss of CNN is fluctuating, staying around 1. I'm looking forward somebody can answer the question as well.

[mathildecaron31]

Hi @1173206772

I have not experimented with very small datasets so I am not sure I have much insights to share unfortunately.

A low clustering loss does not necessarily mean that the clustering is semantically good and I would not be too concerned if it does not decrease so much.

Do you have a downstream task that could help you to inspect the quality of the features you're training or the quality of the clustering you obtain ? I think using downstream tasks and transfers is better to assess the training than inspecting the training loss.

Hope that helps a bit