search

jumpynitro / MPCC

MPCC: Matching Priors and Conditional for Clustering. Official implementation
4 stars 0 forks source link

reproduce the reported results #2

Open fengwang99feng opened 3 years ago

fengwang99feng commented 3 years ago

Thank you for releasing the code!

I attempted to reproduce the reported results on cifar10. The best run gave an accuracy of 46% with your default setting, which is quite far away from the reported 67%.

Could you release the settings that reproduce the reported results? Thanks!

xuaiyishen2012 commented 3 years ago

Hello!

I am interested in this paper and I attempt to run this code. But I can't find the entrance of this code and I don't know where to start.

Could you give me a hand? Thanks. @fengwang99feng

jumpynitro commented 3 years ago

Hello, sorry for the late response. This a cleaned up version of the original code that I used for the paper. The examples scripts are also not the same. I'm rerunning the results to corroborate if there is a difference in the results.

jumpynitro commented 3 years ago

Hello. I checked the code, the training and the results are similar to the reported in the paper. I excuted the script of Cifar10 example and it gave me around 60 % acc. However the configuration was not the same from the paper experiments, I missed the "--concat" flag. With this change I excuted the script again and I obtained around 67 % acc. You can check the "logs" file to observe your results, please observe "Acc" that is the clustering accuracy reported.

I suspect you are not measuring the clustering accuracy as we explained in the paper. I suspect that you are not rearranging the prediction indexes of the clusters, note since is an unsupervised model "cluster index 0" will not be the same to "label 0". To correct this you need to use the labels. Note this is the only part of the code that you use the labels. To help you I uploaded a jupyter notebook with the model weights that obtained high accuracy. I included the prediciton of the clusters and how the indexed are changed to alligned with the real label indexes.

Hopes this helps, Greetings.