panlinchao commented 3 weeks ago

Hello, this research is very meaningful. However, when I reproduced the performance of the default code (with default parameters) in CIFAR80N, I found that the performance of Sym-80% (33.72) and Asym-40% (49.42) is quite different from the performance reported in the paper (42.57 and 60.87 respectively). Is there anything I need to modify when running these cases? Thank you!

P.S. The figure below shows the test results of Asym-40% at different epochs. results

shengmengmeng commented 3 weeks ago

Thank you for your interest in our work. You need to double-check your experimental environments such as the python and pytorch versions, and secondly you need to check that your run command is correct, we have repeatedly confirmed that the direct run of this code gives correct results, which may fluctuate slightly due to random seeds, but will not give you results like yours. You can try the following run command again with our source code:

Sym-80%-CIFAR100N: python main.py --warmup-epoch 20 --epoch 100 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type symmetric --closeset-ratio 0.8 --lr-decay cosine:20,5e-5,100 --opt sgd --dataset cifar100nc --momentum_scs 0.999 --momentum_scr 0.99 --alpha 1.0 --aph 0.95 --gpu 0

Asym-40%-CIFAR100N: python main.py --warmup-epoch 20 --epoch 150 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type asymmetric --closeset-ratio 0.4 --lr-decay cosine:20,5e-5,140 --opt sgd --dataset cifar100nc --momentum_scs 0.9 --momentum_scr 0.9 --alpha 1.0 --aph 0.99 --gpu 0

panlinchao commented 3 weeks ago

Thanks for your valuable training parameters, which allowed me to successfully reproduce the performance of the CIFAR100NC dataset. However, when adapting these parameters to the CIFAR80NO dataset, especially in the Sym-80% case, the results are quite different from those reported in the paper, as shown in the figure below.

If it is not too much trouble, could you kindly share the hyperparameters tailored for the dataset CIFAR80NO at the Sym-80% case?

Thanks for your time and help.

shengmengmeng commented 3 weeks ago

Sym-80%-CIFAR80NO:python main.py --warmup-epoch 20 --epoch 100 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type symmetric --closeset-ratio 0.8 --lr-decay cosine:20,5e-4,100 --opt sgd --dataset cifar80no --momentum_scs 0.95 --momentum_scr 0.85 --alpha 1.0 --aph 0.99 --gpu 7

shengmengmeng / SED

Questions about the hyperparameters #1

Sym-80%-CIFAR100N: python main.py --warmup-epoch 20 --epoch 100 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type symmetric --closeset-ratio 0.8 --lr-decay cosine:20,5e-5,100 --opt sgd --dataset cifar100nc --momentum_scs 0.999 --momentum_scr 0.99 --alpha 1.0 --aph 0.95 --gpu 0

Asym-40%-CIFAR100N: python main.py --warmup-epoch 20 --epoch 150 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type asymmetric --closeset-ratio 0.4 --lr-decay cosine:20,5e-5,140 --opt sgd --dataset cifar100nc --momentum_scs 0.9 --momentum_scr 0.9 --alpha 1.0 --aph 0.99 --gpu 0

Sym-80%-CIFAR80NO:python main.py --warmup-epoch 20 --epoch 100 --batch-size 128 --lr 0.05 --warmup-lr 0.05 --noise-type symmetric --closeset-ratio 0.8 --lr-decay cosine:20,5e-4,100 --opt sgd --dataset cifar80no --momentum_scs 0.95 --momentum_scr 0.85 --alpha 1.0 --aph 0.99 --gpu 7