search

rois-codh / kmnist

Repository for Kuzushiji-MNIST, Kuzushiji-49, and Kuzushiji-Kanji
Creative Commons Attribution Share Alike 4.0 International
695 stars 96 forks source link

Reporting results #2

Closed hysts closed 4 years ago

hysts commented 5 years ago

Hi, I trained some models with KMNIST dataset, and results are here. Using Cutout, shake-shake-26 2x96d model could reach test accuracy of 99.34% as a median of 3 runs.

tkasasagi commented 5 years ago

Thank you so much. We will check it out. :)

mxbi commented 5 years ago

Hi @hysts, thank you for your results. I was wondering if you were willing to run the same model on the MNIST dataset so that we have a comparison number I can add to the table?

hysts commented 5 years ago

Sure! I'll let you know once it finishes.

hysts commented 5 years ago

Hi, I ran the model on the MNIST dataset three times with different random seeds, and the results are 99.75%, 99.76%, and 99.79% accuracy. So, the median of 3 runs is 99.76%. The accuracy of their ensemble is 99.78%, which is slightly worse than the best model. For KMNIST, the results of three runs were 99.37%, 99.34%, and 99.30%, and the ensemble result was 99.39%.

mxbi commented 4 years ago

Added them all (despite the massive delay, I apologise). It would be appreciated if you could rerun on the new dataset version with minor fixes as it may produce a slightly different score.