EricssonResearch / openwebrtc

A cross-platform WebRTC client framework based on GStreamer
http://www.openwebrtc.org
BSD 2-Clause "Simplified" License
1.8k stars 537 forks source link

FPS parameter is ignored in Android #706

Open LanaSina opened 6 years ago

LanaSina commented 6 years ago

Hello,

I am using openwebrtc for android and I am faced with a strange issue where the stream is sometimes dropped without any error in the log (this happens in my code as well as in the original demo code). Google suggests that when there is too much data transmitted this can happens and that the solution is to drop the fps.

Thankfully there is a variable for that in the library:

PeerConnectionParameters params = new PeerConnectionParameters( true, false, displaySize.x, displaySize.y, fps, 1, VIDEO_CODEC_VP9, true, 1, AUDIO_CODEC_OPUS, true); webRtcClient = new WebRtcClient((WebRtcClient.RtcListener) mWebRtcContext, mSocketAddress, params, VideoRendererGui.getEGLContext());

PeerConnectionParameters takes a fps value. Unfortunately this value seems to be ignored by webrtc - even setting it to 1 does not change anything. Indeed, commenting out the entire parameter setting in the demo code of WebRtcClient does not change anything:

`private void setCamera() { localMS = factory.createLocalMediaStream("ARDAMS"); if (pcParams.videoCallEnabled) { MediaConstraints videoConstraints = new MediaConstraints(); //comment out to see what happens //videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxHeight", Integer.toString((pcParams.videoHeight)))); //videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxWidth", Integer.toString((pcParams.videoWidth)))); //videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxFrameRate", Integer.toString(pcParams.videoFps))); //videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("minFrameRate", Integer.toString(pcParams.videoFps)));

        VideoCapturer videoCapturer = getVideoCapturer();
        videoSource = factory.createVideoSource(videoCapturer, videoConstraints);
        localMS.addTrack(factory.createVideoTrack("ARDAMSv0", videoSource));
    }

    //AudioSource audioSource = factory.createAudioSource(new MediaConstraints());
    //localMS.addTrack(factory.createAudioTrack("ARDAMSa0", audioSource));

    mListener.onLocalStream(localMS);
}`

Having found that the settings are ignored, how can I change the fps? I tried to follow the trail to see where the video settings are decided but after going through several wrapper classes I was unable to find out. My only hint is this in the log:

V/MediaCodecVideoEncoder: setRates: 1992 kbps. Fps: 30

Hundreds of lines like this one, but searching for MediaCodecVideoEncoder in the path did not yield anything, so I suppose it is somewhere in the binaries?