twilio / twilio-video-processors.js

Twilio Video Processors is a collection of video processing tools which can be used with Twilio Video JavaScript SDK to apply transformations and filters to a video track.
Other
33 stars 22 forks source link

Enable Multiple Processors #68

Open dionatawijaya opened 1 year ago

dionatawijaya commented 1 year ago

What is the feature that you would like to see? Please describe. Using multiple processors for my video

Is your feature request related to a problem? Please describe. I'm trying to add virtual background and also mirror the video but I can't do it together.

Describe alternatives you've considered I try to flip my background by using CSS img.style.transform = 'scaleX(-1)' but it's not possible since the processor doesn't apply the CSS styling.

Additional context Here's my code for now, with this code, I can only have the background and not mirror.

    videoTrack = await createLocalVideoTrack({
      width: 360,
      height: 640,
      frameRate: 24
    });
    const bg = new VirtualBackgroundProcessor({
      assetsPath: '/tsFlow/',
      backgroundImage: img,
      maskBlurRadius: 5,
      pipeline: Pipeline.WebGL2
    });

    await bg.loadModel();

    const videoProcessor = {
      processFrame: (inputFrame: any, outputFrame: any) => {
        const ctx = outputFrame.getContext('2d');
        ctx.save();
        ctx.scale(-1, 1);
        ctx.drawImage(inputFrame, 0, 0, -inputFrame.width, inputFrame.height);
        ctx.restore();
      }
    };

    try {
      videoTrack.addProcessor(bg, {
        inputFrameBufferType: 'video',
        outputFrameBufferContextType: 'webgl2'
      });

      videoTrack.addProcessor(videoProcessor);
    } catch {}
    videoTrack.attach();