AgoraIO-Community / AgoraWebSDK-NG

The Next Generation of Agora Web SDK
https://agoraio-community.github.io/AgoraWebSDK-NG/
161 stars 75 forks source link

Custom Video Source not working for canvas.captureStream() #19

Closed jinglinwishlife closed 4 years ago

jinglinwishlife commented 4 years ago

I'm trying to create a MediaStreamTrack from a Canvas element then publish it to a video call. local video track is playing correctly, but other users could not receive the video track. captureStream() from video element works fine though.

Code:

    // ... file from <input type="file"/> 
    var url = URL.createObjectURL(file);
    var img = new Image;
    img.src = url;

    img.onload = () => {
        const canvas = document.createElement('canvas');
        canvas.width = 1280;
        canvas.height = 720;
        const ctx = canvas.getContext('2d');
        ctx.drawImage(img, 0, 0);
        var stream = canvas.captureStream();
        var customStream = stream.getVideoTracks()[0];

        // ...createClient
        // ...client.join
        // ...createAudioTrack
        this.localVideoTrack = AgoraRTC.createCustomVideoTrack({
            mediaStreamTrack: customStream,
        });

        this.client.publish([this.localVideoTrack, this.localAudioTrack]);
    }
disoul commented 4 years ago
  1. You should always call ctx.drawImage in requestAnimationFrame or setInterval to trigger the video encoder to produce frames.

  2. Pass a framerate param to canvas.captureStream(), just like canvas.captureStream(30)