Compatibility with Electron (KNN Image Classification)

[jwoa]

Hey,

I am new to ML5 and I just want to say that this library is awesome. I've had no problem working with the library in the Chrome browser but I have issues when trying to run the code in Electron.

I am currently experimenting with Electron and I wanted to implement the KNN Image Classification demo into it. However, after following the instructions on the demo page I run into this error when I run the code, this happens as soon as the page loads:

Uncaught TypeError: n.captureStream is not a function at HTMLVideoElement.n.onplay (ml5.js:1) n.onplay @ ml5.js:1

All the buttons from the example show up along with the video canvas but when I click on "Train A" or "Train B" I get this error:

ml5.js:1 Uncaught (in promise) Error: Requested texture size [0x0] is invalid. at Object.e.validateTextureSize (ml5.js:1) at p (ml5.js:1) at Object.e.createMatrixTexture (ml5.js:1) at t.createMatrixTexture (ml5.js:1) at t.acquireTexture (ml5.js:1) at t.uploadToGPU (ml5.js:1) at t.getTexture (ml5.js:1) at t.fromPixels (ml5.js:1) at t.fromPixels (ml5.js:1) at t.fromPixels (ml5.js:1)

Here's an image in case the error is hard to understand:

I'm extremely new to Electron and ML5 so sorry if this is something that is easily solvabe. Any help at all would be appreciated, thank you!

[cvalenzuela]

Hi @jwoa,

It looks like the webcam is not being initialized in Electron. How are you creating a video stream?

glad you liked the library 😄

[jwoa]

@cvalenzuela Hey! I used the same code from the example and it works to an extent. The webcam shows up in my electron window:

<div id="videoContainer"><video></video></div>

let knn;
let video;

function setup() {
  noCanvas();
  video = createCapture(VIDEO).parent('videoContainer');
  // Create a KNN Image Classifier
  knn = new ml5.KNNImageClassifier(2, 1, modelLoaded, video.elt);
  createButtons();
}

[cvalenzuela]

You need to wait until the video has loaded before adding images to the classifier. The Requested texture size [0x0] is invalid is triggered because of adding an image without content

[jwoa]

Any idea on how I would do that through the code? Or is it actually a matter of me just waiting a while before clicking on the training buttons?

[cvalenzuela]

You can provide a callback when creating the method. This function will execute once the model and video are loaded. See this example: https://github.com/ml5js/ml5-examples/blob/master/p5js/KNNImageClassification/sketch.js#L19

[cvalenzuela]

The KNN method is now deprecated, closing this for now.