I ran the experiments for colored MNIST three times and got the following results:
python train.py with server_user colored_mnist skewed3 severity4
valid_accs_d aligned 0.5823 valid_accs_d skewed 0.7024
valid_accs_d aligned 0.6067 valid_accs_d skewed 0.6851
valid_accs_d aligned 0.6767 valid_accs_d skewed 0.6791
avg: 62.19 and 68.88
Same issue for CIFAR
python train.py with server_user corrupted_cifar10 type0 skewed3 severity4
valid_accs_d aligned 0.5980 valid_accs_d skewed 0.3731
valid_accs_d aligned 0.5760 valid_accs_d skewed 0.3493
avg: 58.7 36.12
Am I making any mistake or are you using any other set of hyperparameters?
Hi,
I ran the experiments for colored MNIST three times and got the following results: python train.py with server_user colored_mnist skewed3 severity4 valid_accs_d aligned 0.5823 valid_accs_d skewed 0.7024 valid_accs_d aligned 0.6067 valid_accs_d skewed 0.6851 valid_accs_d aligned 0.6767 valid_accs_d skewed 0.6791
avg: 62.19 and 68.88
Same issue for CIFAR python train.py with server_user corrupted_cifar10 type0 skewed3 severity4 valid_accs_d aligned 0.5980 valid_accs_d skewed 0.3731 valid_accs_d aligned 0.5760 valid_accs_d skewed 0.3493
avg: 58.7 36.12
Am I making any mistake or are you using any other set of hyperparameters?