Closed hwang26 closed 2 years ago
https://github.com/hwang26/image-classifier/issues/1
A Beginner's Guide to Machine Learning with ml5.js
https://www.youtube.com/watch?v=jmznx0Q1fP0&ab_channel=TheCodingTrain
https://itp.nyu.edu/adjacent/issue-3/ml5-friendly-open-source-machine-learning-library-for-the-web/
p5js
library
setup(), draw()
ml5js
library
image classifier
uses tfjs
the "5" is a tribute to "p5"
tfjs, keras
python version compiles to C++
tfjs runs all in js
lower level library than ml5js
algorithms
datasets
models
eg. pretrained model
MobileNet
[x] Maybe include a tooltip on how our pretrained model works and what dataset it uses
- For transparency reasons
https://www.youtube.com/watch?v=3MqJzMvHE3E&ab_channel=TheCodingTrain
Coding Challenge #158: Shape Classifier Neural Network with ml5.js
Generate Model Dataset
setup()
draw()
Drawing circles
saveFrame()
saves pngs
save 100 frames
draw square, triangle
use rotate() and random() to change up alignment so that they're not all perfectly aligned
rectmode(center)
[-] Should we use a pretrained model or should we use our own datasets?
- We could provide some popular datsets that can be switched in and out on the frontend?
- Let's just use DoodleNet and keep it simple for now
preload()
setup()
see in browser?
create neuralnetwork
shapeClassifier = ml5.neuralNetwork(options)
shape of data?
size of image
128 x 128
channels of image (colors, alpha value, greyscale)
specify task as CNN
set debug to true
automatic visualization of loss function
shows us the error over time during training
add data for circle, squares, triangle
training dataset has "input" and "target"
shapeClassifier.addData(iinput, target)
normalize data
train network
shapeClassifier.train({epochs: 50}, finishedTraining)
callback finishedTraining is just a console.log()
[-] Training takes a while, if we don't do preprocessing, we may want to include a loading bar or icon
- We won't need to do this bc we can just upload the pretrained model
Training is not working super well
reduce resolution
increase stroke weight
reduce random() so the images don't bleed off the edge
reduce input size of image for classifier options
64x64
No way to reverse training data from neural network itself
Upload the pretrained model
p5 sketch
set up sketch and model
shapeClassifier.load(modelDetails, modelLoaded)
pass an image to NN
mousePressed()
create input graphics object
input.copy(..)
image(..)
want to constantly classify drawn image
want to see the results in the page
clear button
https://www.youtube.com/watch?v=ABN_DWnM5GQ&t=229s&ab_channel=TheCodingTrain
ml5.js: Classifying Drawings with DoodleNet
Pretrained model DoodleNet
doodleClassifier = ml5.imageClassifier('DoodleNet', modelReady)
get model ready
classify the canvas
ml5 error first callback pattern
returns objects with classification ideas
cat drawing is not working
guess: dataset is consistent
increase stroke weight
Frontend interface for ml5.js DoodleNet Classifier
https://www.youtube.com/watch?v=jmznx0Q1fP0&ab_channel=TheCodingTrain
https://www.youtube.com/watch?v=3MqJzMvHE3E&ab_channel=TheCodingTrain
https://www.youtube.com/watch?v=ABN_DWnM5GQ&ab_channel=TheCodingTrain