search

aws / amazon-chime-sdk-js

A JavaScript client library for integrating multi-party communications powered by the Amazon Chime service.
Apache License 2.0
717 stars 472 forks source link

Video goes black screen after selecting blur background #2038

Open dathacky opened 2 years ago

dathacky commented 2 years ago

What happened and what did you expect to happen?

After selecting blur background, I noticed the video went black before rendering it made our customer unhappy, is there any way to fix this?

https://user-images.githubusercontent.com/37299929/154604895-8a6010e1-d205-4c89-9d6b-9e5e6f9bef45.mp4

Have you reviewed our existing documentation?

Reproduction steps

  1. Join chime demo
  2. Choose Blur Backgroud 10%
  3. Choose Blur Background 40% -> black screen

Amazon Chime SDK for JavaScript version

2.27.0

What browsers are you seeing the problem on?

Chrome

Browser version

Version 98.0.4758.102

Meeting and Attendee ID Information.

No response

Browser console logs


VM76:2 2022-02-18T02:21:12.886Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 2022-02-18T02:21:13.887Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 2022-02-18T02:21:14.298Z [INFO] SDK - local clock skew estimate=20ms from ping-pong time=101ms
VM76:2 2022-02-18T02:21:14.887Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 [DEMO] clicking video filter Background Blur 10% CPU
VM76:2 [DEMO] Switching to: a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c
VM76:2 [DEMO] Choosing video filter Background Blur 10% CPU
VM76:2 2022-02-18T02:21:15.302Z [INFO] BackgroundBlurProcessor - processor is  supported
VM76:2 2022-02-18T02:21:15.304Z [INFO] BackgroundBlurProcessor - Using browser-provided background blur
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - blur strength set to 15
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - background blur amount set to 0
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - BackgroundBlur processor successfully created
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - BackgroundBlur spec: {
  "paths": {
    "worker": "https://static.sdkassets.chime.aws/bgblur/workers/worker.js?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "wasm": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm.wasm?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "simd": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm-simd.wasm?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98"
  },
  "model": {
    "path": "https://static.sdkassets.chime.aws/bgblur/models/selfie_segmentation_landscape.tflite?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "input": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 3
    },
    "output": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 1
    }
  },
  "assetGroup": "sdk-2.26"
}
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - BackgroundBlur options: {
  "filterCPUUtilization": 10,
  "blurStrength": 15,
  "logger": {
    "name": "BackgroundBlurProcessor",
    "level": 1
  },
  "reportingPeriodMillis": 1000
}
VM76:2 2022-02-18T02:21:15.305Z [INFO] BackgroundBlurProcessor - start initializing the processor
VM76:2 2022-02-18T02:21:15.889Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 2022-02-18T02:21:16.428Z [INFO] BackgroundBlurProcessor - received initialize message: {
  "msg": "initialize",
  "payload": true
}
VM76:2 2022-02-18T02:21:16.429Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur worker
2adbbb16-2e74-4065-ba31-f5e824a20e19:4 INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
printChar @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
b @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
$func1355 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x2050b7
$func640 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0xa95fb
$func713 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0xb4f66
$func405 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x4ba07
$func1190 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1af170
$func1231 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1b90b9
(anonymous) @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
N @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
(anonymous) @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
(anonymous) @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
r.onload @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
load (async)
u @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
(anonymous) @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
o @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
$v @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1962be
i._cwt_model_load_model @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
loadModel @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
(anonymous) @ 2adbbb16-2e74-4065-ba31-f5e824a20e19:4
2adbbb16-2e74-4065-ba31-f5e824a20e19:4 INFO: adding 0 filters
2adbbb16-2e74-4065-ba31-f5e824a20e19:4 INFO: model successfully loaded
VM76:2 2022-02-18T02:21:16.469Z [INFO] BackgroundBlurProcessor - received load model message: {
  "msg": "loadModel",
  "payload": 2
}
VM76:2 2022-02-18T02:21:16.469Z [INFO] BackgroundBlurProcessor - successfully loaded background blur worker segmentation model
VM76:2 2022-02-18T02:21:16.469Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur processor
VM76:2 2022-02-18T02:21:16.470Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/chooseVideoInputDevice with transform device
VM76:2 2022-02-18T02:21:16.470Z [INFO] SDK - Choosing video transform device [object Object]
VM76:2 2022-02-18T02:21:16.470Z [INFO] SDK - video transform device uses previous stream
VM76:2 2022-02-18T02:21:16.472Z [INFO] SDK - processing pipeline input stream settings {"aspectRatio":1.7777777777777777,"deviceId":"a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c","frameRate":15,"groupId":"0e0ae47ddb218df004d4c42b4ffdd7157704fc2cd8e56318aa0d0ed085efc21e","height":540,"resizeMode":"crop-and-scale","width":960}
VM76:2 2022-02-18T02:21:16.509Z [INFO] BackgroundBlurProcessor - frame counter setting frame rate to 15
VM76:2 2022-02-18T02:21:16.509Z [INFO] BackgroundBlurProcessor - background blur source width: 960
VM76:2 2022-02-18T02:21:16.509Z [INFO] BackgroundBlurProcessor - background blur source height: 540
VM76:2 2022-02-18T02:21:16.509Z [INFO] BackgroundBlurProcessor - background blur amount set to 15
VM76:2 2022-02-18T02:21:16.537Z [INFO] SDK - successfully replaced video track
VM76:2 video tile updated: {
  "tileId": 1,
  "localTile": true,
  "localTileStarted": false,
  "isContent": false,
  "active": true,
  "paused": false,
  "poorConnection": false,
  "boundAttendeeId": "f5832c8b-1b17-eb24-d9a6-8aaa99173036",
  "boundExternalUserId": "9d3dac59#dat",
  "boundVideoStream": {},
  "boundVideoElement": {},
  "nameplate": null,
  "videoStreamContentWidth": 960,
  "videoStreamContentHeight": 540,
  "videoElementCSSWidthPixels": 1050,
  "videoElementCSSHeightPixels": 591,
  "devicePixelRatio": 2,
  "videoElementPhysicalWidthPixels": 2100,
  "videoElementPhysicalHeightPixels": 1182,
  "streamId": null
}
VM76:2 2022-02-18T02:21:16.539Z [INFO] SDK - binding video tile 1 to video-27
VM76:2 2022-02-18T02:21:16.539Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/bindVideoElement {"tileId":1,"videoElementId":"video-27"}
VM76:2 setting nameplate to dat
VM76:2 setting attendeeId to f5832c8b-1b17-eb24-d9a6-8aaa99173036
VM76:2 2022-02-18T02:21:16.539Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:16.539Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:16.539Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.540Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:16.653Z [INFO] SDK - video transform device processing started
VM76:2 2022-02-18T02:21:16.886Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 [DEMO] background filter CPU utilization high: 39%
VM76:2 2022-02-18T02:21:17.886Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 [DEMO] background filter CPU utilization high: 37%
VM76:2 [DEMO] clicking video filter Background Blur 30% CPU
VM76:2 [DEMO] Switching to: a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c
VM76:2 2022-02-18T02:21:18.619Z [INFO] BackgroundBlurProcessor - background blur frame process destroyed
VM76:2 [DEMO] Choosing video filter Background Blur 30% CPU
VM76:2 2022-02-18T02:21:18.626Z [INFO] SDK - video transform device processing stopped
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - processor is  supported
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - Using browser-provided background blur
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - blur strength set to 15
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - background blur amount set to 0
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - BackgroundBlur processor successfully created
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - BackgroundBlur spec: {
  "paths": {
    "worker": "https://static.sdkassets.chime.aws/bgblur/workers/worker.js?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "wasm": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm.wasm?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "simd": "https://static.sdkassets.chime.aws/bgblur/wasm/_cwt-wasm-simd.wasm?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98"
  },
  "model": {
    "path": "https://static.sdkassets.chime.aws/bgblur/models/selfie_segmentation_landscape.tflite?assetGroup=sdk-2.26&sdk=2.26.0&ua=chrome-98",
    "input": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 3
    },
    "output": {
      "height": 144,
      "width": 256,
      "range": [
        0,
        1
      ],
      "channels": 1
    }
  },
  "assetGroup": "sdk-2.26"
}
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - BackgroundBlur options: {
  "filterCPUUtilization": 30,
  "blurStrength": 15,
  "logger": {
    "name": "BackgroundBlurProcessor",
    "level": 1
  },
  "reportingPeriodMillis": 1000
}
VM76:2 2022-02-18T02:21:18.643Z [INFO] BackgroundBlurProcessor - start initializing the processor
2adbbb16-2e74-4065-ba31-f5e824a20e19:4 unknown operation close
VM76:2 2022-02-18T02:21:18.887Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 2022-02-18T02:21:19.036Z [INFO] BackgroundBlurProcessor - received initialize message: {
  "msg": "initialize",
  "payload": true
}
VM76:2 2022-02-18T02:21:19.037Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur worker
bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4 INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
printChar @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
b @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
$func1355 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x2050b7
$func640 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0xa95fb
$func713 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0xb4f66
$func405 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x4ba07
$func1190 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1af170
$func1231 @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1b90b9
(anonymous) @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
N @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
(anonymous) @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
(anonymous) @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
r.onload @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
load (async)
u @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
(anonymous) @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
o @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
$v @ _cwt-wasm-simd-9d918e502f01731a.wasm:0x1962be
i._cwt_model_load_model @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
loadModel @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
(anonymous) @ bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4
bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4 INFO: adding 0 filters
bed5c37f-7f7f-4615-8ae1-409eeb5cc493:4 INFO: model successfully loaded
VM76:2 2022-02-18T02:21:19.071Z [INFO] BackgroundBlurProcessor - received load model message: {
  "msg": "loadModel",
  "payload": 2
}
VM76:2 2022-02-18T02:21:19.071Z [INFO] BackgroundBlurProcessor - successfully loaded background blur worker segmentation model
VM76:2 2022-02-18T02:21:19.071Z [INFO] BackgroundBlurProcessor - successfully initialized the background blur processor
VM76:2 2022-02-18T02:21:19.072Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/chooseVideoInputDevice with transform device
VM76:2 2022-02-18T02:21:19.072Z [INFO] SDK - Choosing video transform device [object Object]
VM76:2 2022-02-18T02:21:19.072Z [INFO] SDK - Switched from previous VideoTransformDevice
VM76:2 2022-02-18T02:21:19.072Z [INFO] SDK - video transform device needs new intrinsic device
VM76:2 2022-02-18T02:21:19.072Z [INFO] SDK - DefaultVideoTransformDevice: detach stopping input media stream
VM76:2 2022-02-18T02:21:19.073Z [INFO] SDK - requesting new video device with constraint {"video":{"deviceId":{"exact":"a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c"},"width":{"ideal":960},"height":{"ideal":540},"frameRate":{"ideal":15}}}
VM76:2 2022-02-18T02:21:19.110Z [INFO] SDK - got video device for constraints {"video":{"deviceId":{"exact":"a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c"},"width":{"ideal":960},"height":{"ideal":540},"frameRate":{"ideal":15}}}
VM76:2 2022-02-18T02:21:19.110Z [INFO] SDK - apply processors to transform
VM76:2 2022-02-18T02:21:19.110Z [INFO] SDK - processing pipeline input stream settings {"aspectRatio":1.7777777777777777,"deviceId":"a143b3800185a195bde09066998bb1524184cbf716640bad6b029039744e508c","frameRate":15,"groupId":"0e0ae47ddb218df004d4c42b4ffdd7157704fc2cd8e56318aa0d0ed085efc21e","height":540,"resizeMode":"crop-and-scale","width":960}
VM76:2 2022-02-18T02:21:19.168Z [INFO] BackgroundBlurProcessor - frame counter setting frame rate to 15
VM76:2 2022-02-18T02:21:19.168Z [INFO] BackgroundBlurProcessor - background blur source width: 960
VM76:2 2022-02-18T02:21:19.168Z [INFO] BackgroundBlurProcessor - background blur source height: 540
VM76:2 2022-02-18T02:21:19.168Z [INFO] BackgroundBlurProcessor - background blur amount set to 15
VM76:2 2022-02-18T02:21:19.178Z [INFO] SDK - successfully replaced video track
VM76:2 video tile updated: {
  "tileId": 1,
  "localTile": true,
  "localTileStarted": false,
  "isContent": false,
  "active": true,
  "paused": false,
  "poorConnection": false,
  "boundAttendeeId": "f5832c8b-1b17-eb24-d9a6-8aaa99173036",
  "boundExternalUserId": "9d3dac59#dat",
  "boundVideoStream": {},
  "boundVideoElement": {},
  "nameplate": null,
  "videoStreamContentWidth": 960,
  "videoStreamContentHeight": 540,
  "videoElementCSSWidthPixels": 1050,
  "videoElementCSSHeightPixels": 591,
  "devicePixelRatio": 2,
  "videoElementPhysicalWidthPixels": 2100,
  "videoElementPhysicalHeightPixels": 1182,
  "streamId": null
}
VM76:2 2022-02-18T02:21:19.182Z [INFO] SDK - binding video tile 1 to video-27
VM76:2 2022-02-18T02:21:19.183Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/bindVideoElement {"tileId":1,"videoElementId":"video-27"}
VM76:2 setting nameplate to dat
VM76:2 setting attendeeId to f5832c8b-1b17-eb24-d9a6-8aaa99173036
VM76:2 2022-02-18T02:21:19.183Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:19.184Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:19.184Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:19.184Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.185Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.185Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:19.185Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:19.186Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.186Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.187Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:19.189Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/hasStartedLocalVideoTile null -> true
VM76:2 2022-02-18T02:21:19.189Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getLocalVideoTile
VM76:2 2022-02-18T02:21:19.190Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.190Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getAllVideoTiles
VM76:2 2022-02-18T02:21:19.214Z [INFO] SDK - video transform device processing started
VM76:2 2022-02-18T02:21:19.887Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 [DEMO] background filter CPU utilization high: 40%
VM76:2 2022-02-18T02:21:20.888Z [INFO] SDK - API/DefaultAudioVideoFacade/eeecc3b0-26a1-420d-839c-70958b872713/f5832c8b-1b17-eb24-d9a6-8aaa99173036/getRTCPeerConnectionStats null
VM76:2 [DEMO] background filter CPU utilization high: 41%
xuesichao commented 2 years ago

Hi @dathacky. There are two types of device: Device or VideoTransformDevice. The original camera is of Device type. The VideoTransformDevice is a device that applies processing to another device. So background blur function is actually a VideoTransformDevice. It applies background blur processing to the original devices.

When you switch the background blur filter between different CPU utilization, the actual logic here is JS SDK is switching between different devices.

The reason you observe the black video tile is that the video tile is not removed during the switching, but the camera stream is stopped for a while.

hekmatba commented 2 years ago

Hi, sorry for the inconvenience and thank you for reporting this. We will look into this and respond to this ticket. Thank you.

dathacky commented 2 years ago

@xuesichao @hekmatba Thank you, hope it gets fixed soon

dathacky commented 2 years ago

@hekmatba Hi, hope you are still interested in this issue

dathacky commented 2 years ago

@hekmatba @ltrung I hope this ticket is not forgotten

michhyun1 commented 2 years ago

@dathacky I think this is not a bug, but rather a side effect of selecting a different device. @xuesichao mentioned the behavior above, which is expected when switching between devices.

mrnav90 commented 2 years ago

@michhyun1 So this is not a bug, can we create a fake track ( stream ) when device selecting? Before, we're already used Jitsi Library and also happen that so my team create a fake track replace when device selecting..

michhyun1 commented 2 years ago

@mrnav90 what is the purpose of a fake stream? When you switch from one device to another device, we need to turn off one device and then select and turn on the next device. Currently, that includes switching between a regular video device and a VideoTransformDevice that is applying a video filter to a regular video device.