search

googlecreativelab / teachable-machine-v1

Explore how machine learning works, live in the browser. No coding required.
https://g.co/teachablemachine
Apache License 2.0
3.85k stars 600 forks source link

Confidence of trained classes not updating when an unknown object appears #22

Open nadendlaprasad opened 6 years ago

nadendlaprasad commented 6 years ago

Once i trained the model if i put an unknown object in front of the camera the confidence of last detected class not changing from 100 percent. I mean ideally when it detects an unknown object the confidence of the classes should be 0 percent. please let me know if i am wrong here ?

HalfdanJ commented 6 years ago

Great question. I understand why you intuitively would think so, but they way to think about it is, say you teach it two classes, all that the model knows about are those two classes, and it will always try and categorize whatever it sees in one of those two classes. One way to get around this is by adding a 3rd class that is used as a "unknown" class where you show it samples of things that dont belong to either of the two "true" classes, and keep showing it samples whenever it incorrectly categorizes something as either "a" or "b" instead of unknown.

Does that help?

nadendlaprasad commented 6 years ago

But it's very difficult to train all the unknown objects for the "unknwon"class. I am planning to implement an use case where we'll be training two classes and if any unknown object shows up then the confidence of both the classes should be below 90. isn't it possible ?

HalfdanJ commented 6 years ago

It is not possible with the current KNN classifier implementation. Currently its setup determine which cluster of training images the current live image is the most similar to, but not determine how close it is. But I think it should be possible to tune that algorithm to get closer to what you want, though I havent tried.

It might be easier to do with the codebase in the boilerplate version of this project: https://github.com/googlecreativelab/teachable-machine-boilerplate

nadendlaprasad commented 6 years ago

thank you. I'll look into this

nadendlaprasad commented 6 years ago

could you please let me know the class or API method where i can write my custom code to do this ?

HalfdanJ commented 6 years ago

You would have to implement it directly with the deeplearn.js layer of the project. I unfortunately don't have any more specific pointers at time being for how to do it