Call DesktopCapturer.captureFrame() got EXCEPTION_UNCAUGHT_CXX_EXCEPTION

[sansking]

Hello devopvoid: Thanks for develop this great project! Could you please help me of the confusion I've got? I wanted to get a local screencapture stream with java,and I use some api of this in v2,maven dependency is dev.onvoid.webrtc:webrtc-java:0.2.0 my code likes this(in a common class's main method):

        // to check library correctly loaded
        PeerConnectionFactory factory = new PeerConnectionFactory();
        DesktopCapturer capturer = new DesktopCapturer() {
            @Override
            public void dispose() {
                super.dispose();
            }
        };
//        capturer.captureFrame();
//        DesktopCaptureCallback callback = (result, frame) -> {};
//        capturer.start(callback);

when I call dev.onvoid.webrtc.media.video.desktop.DesktopCapturer's methods(except for dispose()), there is a c/++ error, the captureFrame() mehod got EXCEPTION_ACCESS_VIOLATION (0xc0000005), others got EXCEPTION_UNCAUGHT_CXX_EXCEPTION (0xe06d7363) error;

    I've tried:
        1. change jdk version from 11.0-11.9, and it behaves all the same.
        2. change to ScreenCapturer or WindowCapturer, same result.
        3. create your javafx demo's DesktopSourcesView, same error;

Any suggession is appreciated. If more details could help, just tell me, thank you!

    some error logs:
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_UNCAUGHT_CXX_EXCEPTION (0xe06d7363) at pc=0x00007ffb27bc3b29, pid=10708, tid=18096
#
# JRE version: Java(TM) SE Runtime Environment 18.9 (11.0.9+7) (build 11.0.9+7-LTS)
# Java VM: Java HotSpot(TM) 64-Bit Server VM 18.9 (11.0.9+7-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [KERNELBASE.dll+0x43b29]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows

[sansking]

some more logs may help (when call methods except for captureFrame())

Internal exceptions (3 events):
Event: 0.251 Thread 0x000001db0f031800 Exception 'java/lang/NoSuchMethodError'{0x0000000621336c38}: java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(Ljava/lang/Object;IIII)Ljava/lang/Object;> (0x0000000621336c38) thrown at [./open/src/hotspot/share/interpreter/linkResolver.cpp, l
Event: 0.493 Thread 0x000001db0f031800 Exception 'java/lang/NoSuchMethodError'{0x00000006213f5aa0}: java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(Ljava/lang/Object;Ljava/lang/Object;IIII)Ljava/lang/Object;> (0x00000006213f5aa0) thrown at [./open/src/hotspot/share/interpreter/l
Event: 1.072 Thread 0x000001db0f031800 Exception 'java/lang/NoClassDefFoundError'{0x00000006210e6be8}: dev/onvoid/webrtc/media/DesktopSource> (0x00000006210e6be8) thrown at [./open/src/hotspot/share/classfile/systemDictionary.cpp, line 217]

when call captureFrame(), it changes like this:

Internal exceptions (2 events):
Event: 0.297 Thread 0x000001dec7198000 Exception < 'java/lang/NoSuchMethodError'{0x00000006211116f8}: java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(Ljava/lang/Object;IIII)Ljava/lang/Object;> (0x00000006211116f8) thrown at [./open/src/hotspot/share/interpreter/linkResolver.cpp, l
Event: 0.508 Thread 0x000001dec7198000 Exception < 'java/lang/NoSuchMethodError'{0x0000000620e064d8}: java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(Ljava/lang/Object;Ljava/lang/Object;IIII)Ljava/lang/Object;> (0x0000000620e064d8) thrown at [./open/src/hotspot/share/interpreter/l

and sorry for a wrong info about my code, the way I create DesktopCapturer is

  DesktopCapturer capturer = new ScreenCapturer(){}; 

not

 DesktopCapturer capturer = new DesktopCapturer() {
            @Override
            public void dispose() {
                super.dispose();
            }
        };

which just throw a java excpeiton like java.lang.NullPointerException: Object handle is null

[devopvoid]

Hi, thanks for the report. The desktop capture feature is not fully implemented. But I'm working on a new update and desktop capture will work when the update is finished. A new client that demonstrates the features will be provided as well. Should not take long to be published.

[sansking]

thanks for your response and your work, hope you are doing well!

[polyn0m]

@devopvoid What is the rough horizon for the next version?

[devopvoid]

@polyn0m The next version will have:

• [ ] macOS camera capturer, currently there is none
• [ ] desktop and window capture implementation
• [ ] demo update
• [ ] not sure for the next version: better support for custom media sources

I plan to finish it in the next two weeks.

[sansking]

@devopvoid hello, I got some trouble when I use these APIs,could you please help me?

I create two peers in java, and they send the offer and receive the answer successfully, and eventually both peers are in STABLE status, and then I thought I should gather and exchange candidates, but I don't know how to GATHER LOCAL CANDIDATES WITH THE ICE SERVER I provide, I guess it was automatically perfomed after the answer and offer was exchanged, but it doesn't, and I don't find methods to gather these things other than crate a new candidate manually, is there any way to do such a thing, or is there any mistakes in my codes? any response is appreciated.

my code seems like this:

RTCIceServer iceServer = getMyIceServer();
// this holder just create and hold an RTCPeerConnection with my iceServer and an observer just print event and status
PeerConnectionHolder.init(iceServer,new WssPeerObserver());
// WssHolder hold the methods to exchange infos from peers and signal-server
WssHolder.setMessageHandler(message->{
    JSONObject jsonMessage = JSON.parseObject(message);
    // if receive 'ready' signal from signal-server, one peer create an offer and send to the other peer
    if("ready".equals(jsonMessage.get("type"))){
        PeerConnectionHolder.getPeerConnection().createOffer(new RTCOfferOptions(),new CreateSessionDescriptionObserver(){
            @Override
            public void onSuccess(RTCSessionDescription description) {
                sendOffer(description);
                PeerConnectionHolder.getPeerConnection().setLocalDescription(description,defaultSessionDescObserver);
            }
            @Override
            public void onFailure(String error) {}
        });
    }
    if("offer".equals(jsonMessage.get("type"))){
        RTCSessionDescription desc = new RTCSessionDescription(RTCSdpType.OFFER,((Map)jsonMessage.get("payload")).get("sdp").toString());
        PeerConnectionHolder.getPeerConnection().setRemoteDescription(desc,defaultSessionDescObserver);
        PeerConnectionHolder.getPeerConnection().createAnswer(new RTCAnswerOptions(), new CreateSessionDescriptionObserver() {
            @Override
            public void onSuccess(RTCSessionDescription description) {
                sendAnswer(description);
                PeerConnectionHolder.getPeerConnection().setLocalDescription(description,defaultSessionDescObserver);
            }
            @Override
            public void onFailure(String error) {}
        });
    }
    if("answer".equals(jsonMessage.get("type"))){
        RTCSessionDescription desc = new RTCSessionDescription(RTCSdpType.ANSWER,((Map)jsonMessage.get("payload")).get("sdp").toString());
        PeerConnectionHolder.getPeerConnection().setRemoteDescription(desc,defaultSessionDescObserver);
    }
});
String url = "wss://my-signal-server-url";
WssHolder.init(url);

[devopvoid]

Hi @sansking, usually the PeerConnection creates the local ICE candidates right after the offer/answer was created and set localy. Make sure you set the callback to obtain the candidates from the PeerConnection and the candidates are sent properly via the signaling channel.

I think your Websocket message handler is missing the 'candidate' message handling?! if("candidate".equals(jsonMessage.get("type")))

One more thing, the 'ready' and 'offer' signals: I would swap the calls in the onSuccess callback. First, PeerConnectionHolder.getPeerConnection().setLocalDescription(), then sendOffer/Answer(description).

[sansking]

@devopvoid : Hi, thanks for your help, I've tried as you said, and it seems something still not prepared well. I then tried to run a test function to show my problem as below(the only dependency is dev.onvoid.webrtc:webrtc-java:0.2.0):

public class PeerStateTest {
    static final SetSessionDescriptionObserver defaultSessionDescObserver = new SetSessionDescriptionObserver() {
        public void onSuccess() {System.out.println("success");}
        public void onFailure(String error) {System.out.println(error);}
    };
    public static void main(String[] args) {
        PeerConnectionFactory factory = new PeerConnectionFactory();
        RTCIceServer iceServer = new RTCIceServer();
        iceServer.urls = Arrays.asList("stun:stun.l.google.com:19302");
        RTCConfiguration rtcConfiguration = new RTCConfiguration();
        rtcConfiguration.iceServers = Arrays.asList(iceServer);
        RTCPeerConnection peerConnection = factory.createPeerConnection(rtcConfiguration, new PeerConnectionObserver(){
            // all other state change print messages like below
            public void onSignalingChange(RTCSignalingState state){System.out.println("onSignalingChange:"+state);}
            public void onIceCandidate(RTCIceCandidate candidate) {System.out.println("onIceCandidate:"+candidate);}
        });
        // the offer&answer sdp produced by createOffer/createAnswer method  in last test
        String mockOfferSdp = "v=0\no=- 909530456377251177 2 IN IP4 127.0.0.1\ns=-\nt=0 0\na=msid-semantic: WMS\n";
        peerConnection.setRemoteDescription(new RTCSessionDescription(RTCSdpType.OFFER, mockOfferSdp), defaultSessionDescObserver);
        String mockAnswerSdp = "v=0\no=- 8914950224124104193 2 IN IP4 127.0.0.1\ns=-\nt=0 0\na=msid-semantic: WMS\n";
        peerConnection.setLocalDescription(new RTCSessionDescription(RTCSdpType.ANSWER, mockAnswerSdp), defaultSessionDescObserver);
    }
}

and I got these resultes in my console:

onSignalingChange:HAVE_REMOTE_OFFER
success
onSignalingChange:STABLE
success

as shown above,I guess when I call setLocalDescription,the onIceCandidate in PeerConnectionObserver will be automatically triggered after a while, and I could got the local candidate, and print something like "onIceCandidate:some-candidate", but it didn't, ALL other callback function in PeerConnectionObserver doesn't triggered except for onSignalingChange; so even you hava said

Make sure you set the callback to obtain the candidates from the PeerConnection and the candidates are sent properly via the signaling channel

I'm not sure which callback could obtain the candidates cause none is triggered, if any advice, thank you!

[devopvoid]

You were missing the local media (audio/video) associated with your PeerConnection. I used your minimal example and added the bold part in the following snippet:

SetSessionDescriptionObserver defaultSessionDescObserver = new SetSessionDescriptionObserver() {
    public void onSuccess() {System.out.println("success");}
    public void onFailure(String error) {System.out.println(error);}
};

PeerConnectionFactory factory = new PeerConnectionFactory();

RTCIceServer iceServer = new RTCIceServer();
iceServer.urls.add("stun:stun.l.google.com:19302");

RTCConfiguration rtcConfiguration = new RTCConfiguration();
rtcConfiguration.iceServers.add(iceServer);

RTCPeerConnection peerConnection = factory.createPeerConnection(rtcConfiguration, new PeerConnectionObserver() {
    @Override
    public void onSignalingChange(RTCSignalingState state) {
        System.out.println("onSignalingChange:" + state);
    }

    @Override
    public void onIceCandidate(RTCIceCandidate candidate) {
        System.out.println("onIceCandidate:" + candidate);
    }
});


// Add audio.
AudioOptions audioOptions = new AudioOptions();
audioOptions.echoCancellation = true;
audioOptions.noiseSuppression = true;

AudioSource audioSource = factory.createAudioSource(audioOptions);
AudioTrack audioTrack = factory.createAudioTrack("audioTrack", audioSource);

peerConnection.addTrack(audioTrack, List.of("stream"));


// Create offer.
peerConnection.createOffer(new RTCOfferOptions(), new CreateSessionDescriptionObserver() {
    @Override
    public void onSuccess(RTCSessionDescription description) {
        System.out.println(description);

        peerConnection.setLocalDescription(description, defaultSessionDescObserver);
    }

    @Override
    public void onFailure(String error) {
        System.err.println(error);
    }
});

I hope this helped you.

[sansking]

@devopvoid It works and I gather candidates from the turn/stun server successful, so I could exchange infos across remote explorer, thank you very much!

[TimWebb]

@polyn0m The next version will have:

• [ ] macOS camera capturer, currently there is none
• [ ] desktop and window capture implementation
• [ ] demo update
• [ ] not sure for the next version: better support for custom media sources

I plan to finish it in the next two weeks.

Looks like a great update coming out! Would you have an approx update on timing?

[devopvoid]

This issue was fixed with 0314dec