ml5js / ml5-library

Friendly machine learning for the web! 🤖
https://ml5js.org
Other
6.49k stars 901 forks source link

Support plain arrays for KNNClassifier #255

Closed shiffman closed 5 years ago

shiffman commented 5 years ago

I am looking at @yining1023's wonderful KNN Classifier examples from Machine Learning for the Web. It would be useful for the KNNClassifier's classify() and addExample() methods to be able to accept a tensor or a plain array (and then internally make the tensor). This would allow the following:

  const poseArray = poses[0].pose.keypoints.map(p => [p.score, p.position.x, p.position.y]);
  // Create a tensor2d from 2d array
  const logits = ml5.tf.tensor2d(poseArray);
  // Add an example with a label to the classifier
  knnClassifier.addExample(logits, label);

to turn into:

  const poseArray = poses[0].pose.keypoints.map(p => [p.score, p.position.x, p.position.y]);
  // Add an example with a label to the classifier
  knnClassifier.addExample(poseArray, label);

This is also relevant to recent discussions with @mayaman as there may be an easier way to get a flat array out of the the detected pose rather than having to dig into the object and use a higher order function like map().

It's also useful to note that with the MobileNet FeatureExtractor example + KNN the infer() function outputs a tensor which gets passed directly into addExample(). I wonder if there might be a way of asking PoseNet for the tensor of results directly here for optimization purposes? This is lower priority, however, as the user is likely going to want to draw the results so needs access to the data anyway.

cvalenzuela commented 5 years ago

Makes sense. The same can be applied for every future set of methods that can work with arrays, in that case making a util function that checks for all types of valid inputs would be useful.

Also, in line with https://github.com/ml5js/ml5-library/issues/245, it would be great to just pass the name of the body part

yining1023 commented 5 years ago

Closed in #256