search

LiJunnan1992 / DivideMix

Code for paper: DivideMix: Learning with Noisy Labels as Semi-supervised Learning
MIT License
529 stars 83 forks source link

Ask for hyperparameters for experiments in Table1. #17

Closed huizhang0110 closed 4 years ago

huizhang0110 commented 4 years ago

Hi, @LiJunnan1992 Thanks for your excellent work and I really interested in it. But I cannot get the results claimed in Table .1 by using the default hyperparameters, as shown in the following table. So could you show the hyperparameters setting for experiments in Table .1?

CIFAR_ResNet18 default setting (p_threshold=0.5, lambda_u=25, T=0.5, alpha=4)    
cifar10-sym-20% 90.77/91.06    
cifar10-sym-50% 94.87/94.87    
cifar10-sym-80% 92.87/93.05    
cifar10-sym-90% error    
       
Pre-ResNet18   paper claim  
cifar10-sym-20% 91.63/92 95.7/96.1  
cifar10-sym-50% 94.91/94.91 94.4/94.6  
cifar10-sym-80%   92.9/93.2  
cifar10-sym-90% 50.68/69.84 75.4/76.0 overfit
huizhang0110 commented 4 years ago

Oh, I see it in Appendix B.

huizhang0110 commented 4 years ago

Hi @LiJunnan1992, I use the hyper-params setting in Appendix B and get cool performance as in Table. 1 but there is a difference in the high noisy ratio (sym-90). I do not know why. As shown in the following table. And are there any other hyper-params that affect the performance and the reason why DivideMix is unstable with the high noisy ratio (sym-90). x

LiJunnan1992 commented 4 years ago

Hi, maybe you could try a different random seed and see if the result improves? High noise ratio result could have larger variance, but I haven't observed the drop as you observed.