search

YU1ut / MixMatch-pytorch

Code for "MixMatch - A Holistic Approach to Semi-Supervised Learning"
MIT License
636 stars 129 forks source link

Have you tested the codes with other models? #7

Open shuxjweb opened 5 years ago

shuxjweb commented 5 years ago

Firstly, thanks for you great work. I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

blankWorld commented 5 years ago

Hi, @shuxjweb. Did you trained on SVHN dataset? I have trained on SVHN dataset with same experiment setting as the paper, but there is always 1% point accuracy lower than paper's result.

shuxjweb commented 5 years ago

@blankWorld I only tested it on cifar10, and the accuracy in paper also has a fluctuate range, see B.1 and B.2.

ClayZhang0403 commented 5 years ago

Firstly, thanks for you great work. I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

have u try another models? like res2net? resnxet?

shuxjweb commented 5 years ago

@ClayZhang0403 I just tested the codes with wideresnet and resnet50, but resnet50 behaved much worse than wideresnet.

sudalvxin commented 5 years ago

@ClayZhang0403 I just tested the codes with wideresnet and resnet50, but resnet50 behaved much worse than wideresnet.

different network may need different parameters, such as batch size and lr

dddzg commented 4 years ago

@shuxjweb

Firstly, thanks for you great work. I have done several experiments using your codes, and most experimental results are satisfied. For cifar10, the best acc is 0.8982 with 250 labels, 0.9438 with 4000labels. However, these results are based on the model of wideresnet, which is utilized in paper. The accuracy woud be much worse when I alternate it with resnet50. The best acc is 0.7384 with 250 labels, 0.8219 with 4000labels. I wonder why different models produce such different results ？

Could you share us with the hyper-parameters? I only get 86% acc with 250 labels by the default hyper-parameters.

berzentine commented 4 years ago

Has anybody tried replicating for CIFAR 100 using this code? I get 6-7% less accuracy than the reported numbers for n=10000 labelled samples

sailist commented 4 years ago

Has anybody tried replicating for CIFAR 100 using this code? I get 6-7% less accuracy than the reported numbers for n=10000 labelled samples

cifar100 need larger model, in this paper, they use 135 filters per layer in wideresnet when training cifar100...