Uncaught Type Error in program

[MarkKocherovsky]

I'm trying to use webcame frames in my model.predict function, but it is returning the following error in the console:

Uncaught (in promise) TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'
    at Object.cropTo (canvas.js:60)
    at CustomMobileNet.<anonymous> (custom-mobilenet.js:286)
    at step (custom-mobilenet.js:49)
    at Object.next (custom-mobilenet.js:30)
    at custom-mobilenet.js:24
    at new Promise (<anonymous>)
    at ./node_modules/@teachablemachine/image/dist/custom-mobilenet.js.__awaiter (custom-mobilenet.js:20)
    at CustomMobileNet../node_modules/@teachablemachine/image/dist/custom-mobilenet.js.CustomMobileNet.predict (custom-mobilenet.js:280)
    at index.js:135

My code is as follows:

    getXCoord(args, util) {
        //A1 or A2 returns 0
        //B1 or B2 returns 1
        this.importModel()
            .then(model => {
                const frame = this.runtime.ioDevices.video.getFrame({
                    format: Video.FORMAT_IMAGE_DATA,
                    dimensions: Scratch3PalmReaderBlocks.DIMENSIONS
                });
                console.log(frame);
                const prediction = model.predict(frame);  //line 135!!!
                console.log(prediction);
                return prediction;
            });
    }

line 135 is const prediction = model.predict(frame); frame is logged as follows:

ImageData {data: Uint8ClampedArray(691200), width: 480, height: 360} data: Uint8ClampedArray(691200) [125, 116, 117, 255, 125, 117, 121, 255, 122, 114, 121, 255, 120, 112, 121, 255, 119, 110, 122, 255, 119, 110, 124, 255, 125, 111, 123, 255, 126, 113, 123, 255, 128, 115, 124, 255, 123, 114, 115, 255, 115, 113, 110, 255, 112, 116, 110, 255, 117, 115, 111, 255, 114, 115, 110, 255, 116, 120, 114, 255, 116, 114, 108, 255, 118, 121, 112, 255, 113, 120, 108, 255, 114, 116, 104, 255, 111, 117, 104, 255, 111, 120, 105, 255, 113, 124, 109, 255, 111, 122, 109, 255, 113, 124, 111, 255, 112, 124, 111, 255, …] height: 360 width: 480 __proto__: ImageData

The following is also logged between the ImageData and the error, which may be of use:

Promise {<rejected>: TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not o…}
__proto__: Promise
[[PromiseState]]: "rejected"
[[PromiseResult]]: TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)' at Object.cropTo (http://localhost:8601/lib.min.js:19569:9) at CustomMobileNet.<anonymous> (http://localhost:8601/lib.min.js:18821:49) at step (http://localhost:8601/lib.min.js:18584:23) at Object.next (http://localhost:8601/lib.min.js:18565:53) at http://localhost:8601/lib.min.js:18559:71 at new Promise (<anonymous>) at ./node_modules/@teachablemachine/image/dist/custom-mobilenet.js.__awaiter (http://localhost:8601/lib.min.js:18555:12) at CustomMobileNet../node_modules/@teachablemachine/image/dist/custom-mobilenet.js.CustomMobileNet.predict (http://localhost:8601/lib.min.js:18815:16) at http://localhost:8601/lib.min.js:416103:32
message: "Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'"
stack: "TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'↵    at Object.cropTo (http://localhost:8601/lib.min.js:19569:9)↵    at CustomMobileNet.<anonymous> (http://localhost:8601/lib.min.js:18821:49)↵    at step (http://localhost:8601/lib.min.js:18584:23)↵    at Object.next (http://localhost:8601/lib.min.js:18565:53)↵    at http://localhost:8601/lib.min.js:18559:71↵    at new Promise (<anonymous>)↵    at ./node_modules/@teachablemachine/image/dist/custom-mobilenet.js.__awaiter (http://localhost:8601/lib.min.js:18555:12)↵    at CustomMobileNet../node_modules/@teachablemachine/image/dist/custom-mobilenet.js.CustomMobileNet.predict (http://localhost:8601/lib.min.js:18815:16)↵    at http://localhost:8601/lib.min.js:416103:32"
__proto__: Error

[MarkKocherovsky]

@dalelane I know that you have some experience with this, so if you have time, I would very much appreciate your help.

[dalelane]

I think the error message you've got is pretty good here.

The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'

The predict method is expecting to receive a browser picture resource (such as an <img> element or a <canvas> element or a <video> element. The error message is giving you the full list of what it can support.

You're giving it an ImageData. That isn't in the list of things that it can support.

[MarkKocherovsky]

You're giving it an ImageData. That isn't in the list of things that it can support.

I'm still getting the same error with this code:

    getXCoord(args, util) {
        //A1 or A2 returns 0
        //B1 or B2 returns 1
        this.importModel()
            .then(model => {
                const frame = this.runtime.ioDevices.video.getFrame({
                    format: Video.FORMAT_IMAGE_DATA,
                    dimensions: Scratch3PalmReaderBlocks.DIMENSIONS
                });
                console.log(frame);
                const webcamImage = createImageBitmap(frame);
                console.log(webcamImage)
                const prediction = model.predict(webcamImage);
                console.log(prediction);
                return prediction;
            });
    }

console.log(webcamImage) is returning this:

Promise {<fulfilled>: ImageBitmap}
__proto__: Promise
[[PromiseState]]: "fulfilled"
[[PromiseResult]]: ImageBitmap
height: 360
width: 480
__proto__: ImageBitmap