Prediction is not same as in Teachable web

[ananthakrish98]

When i tried to predict data using the exported saved model from teachable machine in my local using their suggested code, i couldn't get the same accuracy as that of teachable machine.

[sryu1]

Hi @ananthakrish98 what snippet did you use?

[sryu1]

if it was keras or opencv keras, I have a pull request open right now, and it should provide better results using the code snippet from the pending pr.

[ananthakrish98]

I was using keras

[sryu1]

Yeah take a look at my pull request or my tmc repository and try that code, if the predictions are still wrong, can you just show me what you get for the website then the code?

[sryu1]

Did it work now @ananthakrish98?

[sryu1]

The code snippet is now updated in the teachable machine website now.

[ChoiBoHyeon]

I have the same problem. Did you solve it?

[sryu1]

Yes, it should be solved...

[sryu1]

which snippet did you use?

[ChoiBoHyeon]

What is snippet? I am not good at English... sry..

[sryu1]

A snippet is a example/small part of a code. The snippets for python are here: https://github.com/googlecreativelab/teachablemachine-community/tree/master/snippets/markdown/image/tensorflow

[ChoiBoHyeon]

I tested it with the opencv code of the link you sent me. But the recognition rate is still low.

[sryu1]

So you tried both keras and opencv snippet? Keras may have a higher recognition rate, if opencv has a lower recognition rate, could you please tell me and I'll try to fix it.

[ChoiBoHyeon]

Yes, I've used both. This is a link that includes my project code and model. https://github.com/DCU-CapstoneDesign-WhiteCane/WhiteCane-AI

[sryu1]

I'll take a look at that a bit later when I get the time to.

[ChoiBoHyeon]

Oh, thank you. Thank you for your kindness.

[ChoiBoHyeon]

Did you find a solution?

[sryu1]

Hi, I did look through your code and nothing seems wrong... The only thing I'd suspect is most likely the teachablemachine code itself, since it originally uses tfjs, and when you get the keras model, it converts a tfjs model to a keras model and something might've gone wrong with that... I'm so sorry I wasn't able to help you much.

[ChoiBoHyeon]

I see. Thank you again for your kindness!

[miminashi]

I'm facing the same problem. I'm also using keras (not OpenCV)

[sryu1]

Wow, ok, I'll take a look by trying it out when I get the time to.

[sryu1]

Hi @ChoiBoHyeon @miminashi @ananthakrish98 Could you try the snippets from here? I updated some snippets and this might help.

[miminashi]

@sryu1 Thanks! I'll try it.

[sryu1]

Does it work @miminashi?

[Mallikarjun29]

@sryu1 I tried the keras snippets from your repository. I'm still facing the same problem. I think the problem is conversion from tfjs to keras model.

[miminashi]

@sryu1 I have not yet been able to test this quantitatively, but I cannot tell the difference in terms of experience. I am writing a test for my model right now. Please give me some time.

[miminashi]

According to @sryu1 's snippet, I modified my code and tested.

- image = ImageOps.fit(image, size, Image.Resampling.LANCZOS)
+ image = ImageOps.fit(image, size)

The entire source code is here: https://github.com/GUND-ARM/favrial/blob/main/ml/main.py

Test script here: https://github.com/GUND-ARM/favrial/blob/main/test_model.sh

Test steps:

• I have prepared annotated images
  • Class 0 (label: SULEMIO) has about 4000 images
  • Class 1 (label: NOTSULEMIO) has about 8000 images
• I excluded 100 images of each for testing
• I uploaded the remaining images to TeachableMachine and train it
• For each class, counted the number of images that were answered correctly out of 100 and expressed as a ratio

I am an novice in machine learning, so please let me know if there are any problems with my testing methods.

Test Result:

• SULEMIO: class0
• NOTSULEMIO: class1
• SULEMIO(sryu1): class0 with modified python code
• NOTSULEMIO(sryu1): class1 with modified python code

epoch	SULEMIO	NOTSULEMIO	SULEMIO(sryu1)	NOTSULEMIO(sryu1)
50	0.79	0.75	0.79	0.75
60	0.65	0.86	0.67	0.85
80	0.67	0.86	0.70	0.85
100	0.69	0.87	0.66	0.87
150	0.74	0.81	0.71	0.80
200	0.76	0.81	0.72	0.84
250	0.71	0.80	N/A	N/A
300	0.73	0.84	0.73	0.86

There did not seem to be much difference between the results with and without Image.Resampling.LANCZOS specified.

[sryu1]

When you were testing it, did you record the results with the teachable machine's website results?

[mohasaid]

Hello,

I'm facing the same issue mentioned here, the prediction of my program is not the same as in the teachable web. In my case, I'm generating an audio model, then I convert it to a Keras model using tensorflowjs_converter (because I'm unable to download it as a Keras model) and I run it on a Python project.

Related to this issue I have two questions:

• Is there any tutorial about how to create an audio classifier model in the teachable machine and then run it in Python? I'm interested in listening to my microphone and making the classification of the audio data every five seconds for instance.
• I noticed in the snippet of the website that when you click on the download model (for the TensorFlow.js option) it uses tensorflow-models/speech-commands which has not a similar library in Python, and maybe that's the problem? I have also tried using Tflite Interpreter but I did not achieve it.

[vish-n-u-k]

I am also facing the same issue, I am using it for image recognition , it works perfectly on teachable web but in my app it sometimes fail. Should I train my model more would that help? Or should I do something else.