Closed Kstar0722 closed 2 years ago
Hey @Kstar0722 can you please provide steps to reproduce, code snippet on how you're adding the processor, and also a room sid?
@charliesantos sorry it was my mistake, I guess. I published local track first and then add background lik
const track = await createLocalVideoTrack({
deviceId: this.selectedVideoInputDevice
})
await this.currentRoom.localParticipant.publishTrack(track)
if (args.backgroundEnabled) {
await this.addBackgroundToVideoTrack(track, args.backgroundUrl)
}
but the issue fixed I changed order like add background first and then publish the local track. like
const track = await createLocalVideoTrack({
deviceId: this.selectedVideoInputDevice
})
if (args.backgroundEnabled) {
await this.addBackgroundToVideoTrack(track, args.backgroundUrl)
}
await this.currentRoom.localParticipant.publishTrack(track)
but I wonder it should work if we publish track first and then add background. b/c when we need to toggle background (enable/disable), we have to just remove/add background processer in created track without create new track.
Hi @Kstar0722 , adding background after publishing track should work. Can you please provide a code snippet on how you're adding the background? That is, please show this.addBackgroundToVideoTrack(track, args.backgroundUrl)
implementation. Also, are you getting any log messages? Can you please enable debug mode on twilio-video and provide logs?
Hi @charliesantos ,
sure, addBackgroundToVideoTrack triggers attachBackgroundToTrack.
@Kstar0722 thank you for providing the code snippet. Nothing stands out to me so far. I have a couple of requests
// Save this in your hooks or app state, or wherever your video track can access them
let blurProcessor;
let virtualBackgroundProcessor;
// Call this on your app start up
async function initializeVideoProcessors() {
blurProcessor = new GaussianBlurBackgroundProcessor({...});
virtualBackgroundProcessor = new VirtualBackgroundProcessor({...});
await blurProcessor.loadModel();
await virtualBackgroundProcessor.loadModel();
}
Now your addBackgroundToVideoTrack
can be simpler
async function addBackgroundToVideoTrack(track, backgroundUrl) {
try {
const backgroundImage = await loadImage(backgroundUrl);
virtualBackgroundProcessor.backgroundImage = backgroundImage;
track.addProcessor(virtualBackgroundProcessor);
} catch {
track.addProcessor(blurProcessor);
}
}
async function loadImage(url) {
return new Promise((resolve, reject) => {
const image = new Image();
image.onload = () => resolve(image);
image.onerror = reject;
image.src = url;
});
}
Thanks @charliesantos , I will try to update my code like above.
Hi @charliesantos I try to update my code like your solution. but it's not working.
Can you please enable debug mode on twilio-video and provide logs? Can you also push up your changes to a repo so I can take a look?
ok, I will try to enable debug mode and give you callSid, maybe tomorrow morning but I can't share my code since it's private.
@charliesantos I checked it and seems the problem is on our end. I will close the ticket since there's no issue in this lib. thanks for your help.
@Kstar0722 I'm glad this is now resolved. What was the issue you found? It might help others in case they run into a similar issue.
It works VirtualBackgroundProcessor in local track, but I see blank screen in the remote track with VirtualBackgroundProcessor I checked remote video track object. and there's no processor(processor: undefined)
Expected behavior:
Does not showing remote video track with background
Actual behavior:
Need to show remote video track with background
Software versions: