Low CIFAR-100 accuracy

[0xsamgreen]

Has anyone run this on CIFAR-100? I modified the code just enough to download and train on it, changing nothing else, and after running augment.py on two GPUs, I got 65% top-1 and 88% top-5 accuracy on the validation set. 65% top-1 is state of the art for 2014. Current SOTA for CIFAR-100 is 91.3 for top-1 accuracy. https://benchmarks.ai/cifar-100

Here's the search command: python search.py --name cifar100 --dataset cifar100 --gpus all --batch_size 96 --workers 8 --print_freq 10 --w_lr 0.05 --w_lr_min 0.002 --alpha_lr 0.0006

Here's the augment command: python augment.py --name cifar100_1 --dataset cifar100 --genotype "Genotype(normal=[[('skip_connect', 0), ('skip_connect', 1)], [('skip_connect', 0), ('skip_connect', 1)], [('skip_connect', 0), ('skip_connect', 1)], [('skip_connect', 0), ('skip_connect', 1)]], normal_concat=range(2, 6), reduce=[[('avg_pool_3x3', 0), ('avg_pool_3x3', 1)], [('skip_connect', 2), ('max_pool_3x3', 0)], [('skip_connect', 2), ('avg_pool_3x3', 0)], [('skip_connect', 2), ('avg_pool_3x3', 0)]], reduce_concat=range(2, 6)

Has anyone else tried to train CIFAR-100?

[0xsamgreen]

Turns out this is a known limitation of DARTS...

[khanrc]

Even if this is known limitation, the results looks too strange. I think you can get better results by single-gpu search.

[0xsamgreen]

Thanks, I will try this.

[heartInsert]

Turns out this is a known limitation of DARTS...

What do you mean about the limitation ?

I know there is some new paper like pc_darts and darts+ based on darts , have you tried them ?