KenEucker / biketag-api

The javascript client for interacting with BikeTag games.
https://keneucker.github.io/biketag-api/
GNU Affero General Public License v3.0
10 stars 5 forks source link

Add bicycle inside image detection for new BikeTag Post submissions. #206

Open KenEucker opened 11 months ago

KenEucker commented 11 months ago

Is your feature request related to a problem? Please describe. Currently, Players of BikeTag can upload any image to the site.

Describe the solution you'd like We can use ML, specifically ml5.js, to detect whether or not a bicycle is present in the image. If a bicycle is not present, the Player should see a similar popup to when an image is submitted outside of the boundary of a game: "The photo you uploaded does not appear to have a bicycle in it. Please upload a photo with a bicycle clearly visible."

Additionally, since BikeTag Posts are not "selfies", we could detect whether or not a "face" is present in the photo and reject is based on that. The API method that handles this, validateBikeTagImage, should return a boolean for valid and an error object, which is undefined if it is a valid image.

Describe alternatives you've considered We've talked about this in the past and some solutions were discussed then. Now, after looking into it for a few minutes, I've chosen ml5.js. Other solutions are available and anything that is free will be considered.

Additional context ML5JS - Object Detector

Upvote & Fund

Fund with Polar

KenEucker commented 11 months ago

Here's the relevant ML5JS code:

// When the model is loaded
function modelLoaded() {
  console.log('Model Loaded!');
}

// Create a ObjectDetector method
const objectDetector = ml5.objectDetector('cocossd', {}, modelLoaded);

// Detect objects in the image element
objectDetector.detect(image, (err, results) => {
  console.log(results); // Will output bounding boxes of detected objects
});