Blur Processor Worker Does Not Close

[MitchTalmadge]

What happened and what did you expect to happen?

After calling BackgroundBlurVideoFrameProcessor.create() followed by processor.destroy() once the processor has been created, the Dedicated Worker does not close and the message unknown operation close is emitted by the worker.

Here I have repeated the example 10 times by use of this codepen, and you can see that the workers are still alive in the Chrome task manager:

Have you reviewed our existing documentation?

• [X] GitHub issues
• [X] FAQs page
• [X] README page
• [X] Feature-specific guides

Reproduction steps

1. Create a BackgroundBlurVideoFrameProcessor
2. Destroy it.
3. Check the Chrome task manager and see that the background worker is still there.

Try my example: https://codepen.io/mitchtalmadge/pen/podWbox (you need to use the F12 console, not the codepen console, or you won't see the message.)

Amazon Chime SDK for JavaScript version

2.26.1

What browsers are you seeing the problem on?

Google Chrome

Browser version

98.0.4758.82

Meeting and Attendee ID Information.

No response

Browser console logs

iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:24.937Z [INFO] BackgroundBlurProcessor - BackgroundBlur processor successfully created
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:24.937Z [INFO] BackgroundBlurProcessor - BackgroundBlur spec: {
  "paths": {
    "worker": "https://static.sdkassets.chime.aws/bgblur/workers/worker.js?assetGroup=sdk-2.26&sdk=2.26.1&ua=chrome-98",
    "wasm": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm.wasm?assetGroup=sdk-2.26&sdk=2.26.1&ua=chrome-98",
    "simd": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm-simd.wasm?assetGroup=sdk-2.26&sdk=2.26.1&ua=chrome-98"
  },
  "model": {
    "path": "https://static.sdkassets.chime.aws/bgblur/models/selfie_segmentation_landscape.tflite?assetGroup=sdk-2.26&sdk=2.26.1&ua=chrome-98",
    "input": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 3
    },
    "output": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 1
    }
  },
  "assetGroup": "sdk-2.26"
}
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:24.937Z [INFO] BackgroundBlurProcessor - BackgroundBlur options: {
  "blurStrength": 15,
  "logger": {
    "name": "BackgroundBlurProcessor",
    "level": 1
  },
  "reportingPeriodMillis": 1000,
  "filterCPUUtilization": 30
}
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:24.937Z [INFO] BackgroundBlurProcessor - start initializing the processor
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.658Z [INFO] BackgroundBlurProcessor - received initialize message: {
  "msg": "initialize",
  "payload": true
}
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.658Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur worker
4acf1927-2aa2-49e1-9c05-cf159ce38083:4 INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
4acf1927-2aa2-49e1-9c05-cf159ce38083:4 INFO: adding 0 filters
4acf1927-2aa2-49e1-9c05-cf159ce38083:4 INFO: model successfully loaded
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.759Z [INFO] BackgroundBlurProcessor - received load model message: {
  "msg": "loadModel",
  "payload": 2
}
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.760Z [INFO] BackgroundBlurProcessor - successfully loaded background blur worker segmentation model
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.760Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur processor
iframeConsoleRunner-d0f3648046d2aaca07bd0037b9e061a26c74a8a999b75672ad6a638cca641472.js:1 2022-02-14T19:44:25.760Z [INFO] BackgroundBlurProcessor - background blur frame process destroyed
4acf1927-2aa2-49e1-9c05-cf159ce38083:4 unknown operation close

[MitchTalmadge]

Here is where close is emitted:

https://github.com/aws/amazon-chime-sdk-js/blob/2f9ab450288ea0371eb5e3cb41bdb9ca8d4134ae/src/backgroundfilter/BackgroundFilterProcessor.ts#L396-L407

In the worker (viewed through Chrome F12 Sources tab) you can see where the switch statement does not include "close", but it does have "stop" which calls self.close:

i.addEventListener("message", (function(t) {
        let n = t.data.payload;
        switch (t.data.msg) {
        case "initialize":
            ...
            break;
        case "loadBlur":
            ...
            break;
        case "blur":
            {
                ...
                break
            }
        case "loadModel":
            ...
            break;
        case "predict":
            {
                ...
                break
            }
        case "destroy":
            o.destroy();
            break;
        case "stop":
            self.close();
            break;
        default:
            console.log(`unknown operation ${t.data.msg}`)
        }
    }
))

[MitchTalmadge]

As a temporary fix, I was able to force-send the stop message, which does close the worker as seen here:

https://codepen.io/mitchtalmadge/pen/OJOxRgY

[richhx]

Hi Mitch! Thanks for reporting this problem. As you pointed out, the wrong message is being sent (i.e. stop should've been sent instead of close to the WebWorker. I'll submit a PR in a bit with a unit test.