Open thlee1122 opened 1 year ago
I managed to find the root cause for this.
The default pipeline
for the video processors is set to WebGL2
, but the default processor options in twilio-video
is set to { inputFrameBufferType: 'offscreencanvas', outputFrameBufferContextType: '2d' }
, which are options for Canvas2D.
There are 2 ways to fix this:
track.addProcessor
track.addProcessor(processor, { inputFrameBufferType: "video", outputFrameBufferContextType: "webgl2" });
processor = new VirtualBackgroundProcessor({
pipeline: Pipeline.Canvas2D,
...options
});
I only managed to find this out by reading through the example.
*Canvas2D might not have full functionality with Safari - the Gaussian Blur didn't work for Safari 16.5.1 MacOS (haven't tested others)
@alex93293 Thank you for this! This looks like a work around. Is the Twilio video processor team going to release an actual fix for this issue?
Also worked for me! Does anyone knows which one should we be using? (webgl2 or canvas2d) I mean, since the defaults are different for @twilio/video-processors
and twilio-video
.
As a sidenote, I'm sticking with canvas2d for now, since webgl2 completely ignores the fitType
option (and having a ImageFit.Contain
is must for me).
I had the same issue when I upgraded to v2.0.0.
The following seemed to work for me but it was painfully slow. so I ended up using Canvas2D pipeline option instead.
track.addProcessor(processor, { inputFrameBufferType: "video", outputFrameBufferContextType: "webgl2" });
Edit: It seems for higher resolution it is slow. width: 1280, height: 720, if I change width:640, height:480. it seems to be okay.
I found a bug with WebGL2. We recently upgraded the twilio-video-processor to the latest version, 2.0.0. With the above-updated version, the WebGL2 pipeline video background blur functionality throws a TypeError: can't access property "VERTEX_SHADER", gl is null.
Below is the screenshot of the error:
Software versions: