Quick question about video streaming

[RaniRaven]

I am trying to follow the path in the tests that allows for streaming actual video into the peer. I am getting to the stage where both peers are connected, each is setting right the SDP offer and answer to the remote and local. The webrtc session is up for sure, but nothing is being streamed. I am probably missing something trivial, but I cannot understand what is supposed to be the "streamer" that actually streams the video. Unfortunately there is no test showing this including video, only data channel is being illustrated, Isn't there supposed to be an event loop which makes the streaming of video frames there ? or is it done by the native functions behind the scenes, like by the call to the VideoSource.start() ?

Can someone shed some light on it ? Cause I think I am quite close to success there.

[devopvoid]

Unfortunately there is no test showing this including video, only data channel is being illustrated

That's because the tests must be able to run on headless servers which have no cameras or microphones connected.

As of now, you can stream webcams. Custom video sources is doable too, but haven't done that so far.

Have a look at the code snippets in #25. Once you call videoSource.start() your webcam should turn on.

[RaniRaven]

I must be missing something , as this is my implementation : void TestVideoRTC() throws Exception { //TestPeerConnection caller = new TestPeerConnection(factory); TestPeerConnection callee = new TestPeerConnection(factory);

    // set video
    VideoDeviceSource videoSource = new VideoDeviceSource();
    videoSource.setVideoCaptureDevice(MediaDevices.getVideoCaptureDevices().get(0));
    VideoTrack videoTrack = factory.createVideoTrack("videoTrack", videoSource);

    List<String> streamIds = new ArrayList<>();
    streamIds.add("stream-0");

    // config
    RTCIceServer iceServer = new RTCIceServer();
    iceServer.urls.add("stun:stun.l.google.com:19302");

    RTCConfiguration config = new RTCConfiguration();
    config.iceServers.add(iceServer);
    config.bundlePolicy = RTCBundlePolicy.MAX_BUNDLE;
    config.rtcpMuxPolicy = RTCRtcpMuxPolicy.NEGOTIATE;
    config.iceTransportPolicy = RTCIceTransportPolicy.ALL;
    TestPeerConnection caller = new TestPeerConnection(factory, config);

    callee.setRemotePeerConnection(caller);
    caller.setRemotePeerConnection(callee);

    RTCRtpSender videoSender = caller.getPeerConnection().addTrack(videoTrack, streamIds);

    //factory.createPeerConnection(config, null);

    caller.getPeerConnection().setConfiguration(config);
    RTCSessionDescription goodOffer = null;
    TestCreateDescObserver callerObserver = new TestCreateDescObserver();
    caller.getPeerConnection().createOffer(new RTCOfferOptions(), callerObserver);
    goodOffer = callerObserver.get();
    RTCSessionDescription offer = caller.getPeerConnection().getRemoteDescription();
    caller.getPeerConnection().setLocalDescription(goodOffer, new TestSetDescObserver());
    callee.setRemoteDescription(goodOffer); //getPeerConnection().setRemoteDescription(offer,null);
    RTCSessionDescription goodAnswer = null;
    TestCreateDescObserver calleeObserver = new TestCreateDescObserver();
    callee.getPeerConnection().createAnswer(new RTCAnswerOptions(), calleeObserver);
    goodAnswer = calleeObserver.get();
    callee.getPeerConnection().setLocalDescription(goodAnswer, new TestSetDescObserver());
    goodAnswer = calleeObserver.get();
    RTCSessionDescription answer = callee.getPeerConnection().getLocalDescription();
    System.out.println("Caller state :  " + caller.getPeerConnection().getConnectionState());
    System.out.println("Callee state :  " + callee.getPeerConnection().getConnectionState());
    callee.getPeerConnection().setLocalDescription(goodAnswer, new TestSetDescObserver());
    caller.setRemoteDescription(goodAnswer); //getPeerConnection().setRemoteDescription(answer, null);

    //caller.setRemotePeerConnection(callee);
    //callee.setRemotePeerConnection(caller);

    //callee.setRemoteDescription(caller.createOffer());
    //caller.setRemoteDescription(callee.createAnswer());
    RTCRtpTransceiverInit init = new RTCRtpTransceiverInit();
    init.direction = RTCRtpTransceiverDirection.RECV_ONLY;

    VideoDeviceSource videoSource1 = new VideoDeviceSource();
    VideoTrack track1 = factory.createVideoTrack("videoTrack", videoSource);
    VideoTrackSink sink = new VideoTrackSink() {
        @Override
        public void onVideoFrame(VideoFrame frame) {
            System.out.println("Got frame : " + frame.toString());
        }
    };
    init.streamIds = streamIds;
    RTCRtpTransceiver transceiver = callee.getPeerConnection().addTransceiver(track1, init);

    System.out.println("Caller state :  " + caller.getPeerConnection().getConnectionState());
    System.out.println("Callee state :  " + callee.getPeerConnection().getConnectionState());

    //transceiver.

    try {
        caller.waitUntilConnected();
        callee.waitUntilConnected();
    } catch (InterruptedException exc) {
        System.out.println("Got interupted : " + exc.getStackTrace());
    }
    System.out.println("Caller state :  " + caller.getPeerConnection().getConnectionState());
    System.out.println("Callee state :  " + callee.getPeerConnection().getConnectionState());

    try {
        videoSource.start();
        System.out.println("[Start] Caller state :  " + caller.getPeerConnection().getConnectionState());
        System.out.println("[Start] Callee state :  " + callee.getPeerConnection().getConnectionState());
    } catch (Exception exc) {
        System.out.println("Exc : " + exc.getMessage() +  "  - " + exc.getStackTrace());
    }

    Thread.sleep(1500);

    /*caller.sendTextMessage("Hello world");
    callee.sendTextMessage("Hi :)");

    Thread.sleep(500);

    assertEquals(List.of("Hello world"), callee.getReceivedTexts());
    assertEquals(List.of("Hi :)"), caller.getReceivedTexts());*/

    caller.close();
    callee.close();
}

For some reason everything is connected but I am not getting the frames of the video at the callee. What am I missing here, as I followed all the steps in the sample ?

[devopvoid]

Hi @RaniRaven, did you manage to run your example successfully? Maybe the timeout Thread.sleep(1500); is too short.

[RaniRaven]

No actually I get a weird problem that the track is LIVE on both side (both peers) but I don;t get the onframe event. Now I tried to upgrade to the 0.5.0 version of the dll (pom), and all crashes.

[RaniRaven]

I've good news now, I have finally been able to set a working sample there. Currently working in the model of ICE gathering all. I'll see about the full trickling model, later. I am still a little bit confused about setting the audio stream, since the model is not clear there, how do I start ? Is there any sample by the way of how to stream a file from a filesystem using those sources ?

[kinsleykajiva]

@RaniRaven any updates yet on this ?

[devopvoid]

Streaming a file is not supported yet, but will be possible in the near future. For upcoming progress see #87