how to use jsmpeg.player output in object detection code.

[sazzad15-1779]

can anyone help me to use jsmpeg streaming in object detection function. const player = new JSMpeg.Player("ws://localhost:9999", { canvas: canvas, autoplay: true, loop: true, decodeFirstFrame: true, }); how to get video data from this streaming?

[phoboslab]

What is "video data"? Raw pixels?

Depending on the disableGl option, you can get pixel data from

• player.renderer.canvas through toDataURL() (very slow)
• player.renderer.gl (in WebGL mode) through readPixels()
• player.renderer.context (in Canvas2D mode) through getImageData()

If performance is important and black & white is enough, you may hook into the renderer interface and forward the luma plane (y) to your object detection.

If you need color information, you may also hook into the Canvas2D renderer here and forward the Uint8TypedArray with the RGBA data this.imageData.data without going through the Canvas API first. This may be slower than doing the color space conversion in WebGL and using readPixels() though.