How to run segmentation in a Worker

[berkon]

On the documentation page: https://developers.google.com/mediapipe/solutions/vision/image_segmenter/web_js it is mentioned that segmentForVideo() can also be run in a worker in order to improve performance:

I couldn't find a working example for this. The problem is that importScripts() is not supported in workers, but the mediapipe npm module seems to use this function.Thus I'm getting:

I'm running this all in the renderer process of an Electron app.

[schmidt-sebastian]

This is a high priority work item for us. We will hopefully have an update for you soon.

[schmidt-sebastian]

I am able to run our Tasks API in a Web Worker. Can you share your example somewhere?

Here are my two files: https://gist.github.com/schmidt-sebastian/bb9b06988c8d4e09370770fbe3e9478f

[berkon]

@schmidt-sebastian, this is the link to my Electron demo repository. Just connect a webcam an run it. It should work out of the box. The general issue that I have is that the frame rate is not good enough at 1280 x 720. So I hope the Web Worker solution will speed it up.

https://github.com/berkon/background-removal-new

Is there a new / specific version of the library which I have to use, or do I just need to apply your example code? BTW: The application where I need too add this functionality runs on plain JS. Is there a non-Typescript way to use it?

[schmidt-sebastian]

I might be missing something here, but I don't see the code that transfers the image from the webcam to the worker. You also need to transfer the result back. The link I pasted shows you how to do this via ImageData, but using ImageBitmap would be much faster.

[berkon]

The example which I've posted is the initial way I implemented it (without Worker, because I couldn't get it to work).