hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI
Apache License 2.0
334 stars 102 forks source link

[image] replace torchvision resnet with cifar-specific resnet #131

Closed FrankLeeeee closed 2 years ago

FrankLeeeee commented 2 years ago

The model convergence performance in the ResNet+CIFAR10 example is much lower (below 90% testing accuracy) than the results reported in the ResNet paper. This is because that the author has designed slightly different model architectures for CIFAR and ImageNet experiments. The model provided by torchvision is for ImageNet and thus gives poor performance on CIFAR10. I have added CIFAR10-ResNet implementation in this example and produced 95% testing accuracy.