I ran the code of Shapley-NAS on CIFAR-10 and was unable to reproduce the results claimed in the paper.
I got similar results (~96.96 % top 1 accuracy) with both the provided genotype and a newly searched one.
This is far from the results mentioned in the paper (97.53 % average with 0.04 std).
This is consistent with the results another user obtained in issue #2 .
I used the commands provided in the README and the default hyperparameters.
Have I done something wrong?
I included the log files for information.
Hi,
I ran the code of Shapley-NAS on CIFAR-10 and was unable to reproduce the results claimed in the paper. I got similar results (~96.96 % top 1 accuracy) with both the provided genotype and a newly searched one. This is far from the results mentioned in the paper (97.53 % average with 0.04 std). This is consistent with the results another user obtained in issue #2 . I used the commands provided in the README and the default hyperparameters. Have I done something wrong? I included the log files for information.
Thanks
log_provided_genotype_C10.txt log_searched_genotype_C10.txt