Closed codec-abc closed 7 years ago
What the error rate the CNTK tool tell you when testing after the training? If the error rate is high, I guess that your data is not poured into CNTK using correct reader format. Could you show some more detailed information?
Sorry, I ditn't explained myself clearly. I trained the various models using the documentation and it worked fine. The error rate was exactly those stated. Then I made a paint like application where the drawing can be send to the model. The problem is that the prediction were not so good.
I made more tests and it seems that using a the ConvNet_MNIST helps. Moreover, I noticed that it had difficulties if the digit drawn was a bit too small or the value were not saturated enough. I expected the network to be more robust for those cases.
03_OneConvdropout is a model in GettingStarted and it's deliberately kept extremely simple. As a result the classification accuracy is bad. In our examples the best model for MNIST would be ConvNet_MNIST.py, which is state-of-the-art with no data augmentation.
If you want to further improve accuracy, to handle your own data, the best way is to add your own training data. Another way is to use data augmentation. For example: https://www.microsoft.com/en-us/research/publication/best-practices-for-convolutional-neural-networks-applied-to-visual-document-analysis/.
Thanks for the links. I can confirm that using the ConvNet model helps. Moreover the other problem that I had (and probably even worse than using a too simple model) is that I drawn on a bigger texture than the model use as input. When I resized it I lost too much information making the prediction hard to do. Anyways, thanks for your help and sorry for the inconvenience.
Hi,
I tried to test the different sample models for the MNIST data set using my own data (by modifying the EvalClient project)
It run fine but the result seems wrong wrong way too often. As an example using the following data with the trained 03_OneConvdropout model:
I got the following result: -2.277135 -7.73908 -1.059328 -2.477858 -6.769311 7.046165 4.724845 -9.132572 10.03633 3.570117
While the data seems to represent a "fine" six digit it has a pretty low score. On average I would say that for digit that "look correct" I have an error rate of 10% to 50 % (depending on which digit) which is way more than the most basic model.
Is this something expected or am I doing something wrong (which clearly could be since I am little to none experience in machine learning)?