Open modcarroll opened 2 years ago
Cesar will own this, he experimented with OpenCV, will train an object detection model for the RPi
@cmaciel is getting 1.1-1.5 FPS with a pre-trained model (MobileNet, has 90 classes of objects) on a RPi 3. He will re-train the model to focus only on objects found inside a house to get better results.
Oops, the model was trained but the data labels were missing! :D
Need to create a list of common household items to train a new model. Maybe focus the model on rock, paper, scissors? Maybe tjbotlib has generic functions for loading local models and running inference, and then we create a separate recipe that uses that API to load a rock/paper/scissors model locally in order to play that game.
I propose a two-part solution for the elimination of Watson Visual Recognition:
see()
functionality from tjbotlib and add a look()
function which just takes a picture and returns the filename in the filesystem (e.g. somewhere in /tmp
). Which is to say, I think this is what takePhoto()
already does, so maybe we just rename that function.rock-paper-scissors
that uses @cmaciel 's on-device vision model. But we likely don't want to bundle the whole model inside the tjbot repo because it's likely big (right?) so maybe the recipe downloads the model from somewhere on first run (e.g. test if it's in the filesystem, if not, then download it). We'd need a permalink for hosting the model somewhere.@cmaciel if this plan makes sense to you, I'd like to close this card and open two new cards to cover 👆 . Thanks!
Watson Visual Recognition is deprecated, so we should find a (preferably) open source replacement