ICE exchange does not work properly when a native client creates an offer to a browser

ChrisTrenkamp commented 10 months ago

Godot version

4.1.3.stable.official

Plugin version

v1.0.3

System information

Windows 11

Issue description

~~This was already posted here.~~ Deleted because it wasn't a bug, it was PEBKAC.

The ICE exchange seems to work fine with native-to-native clients. It also works when the browser calls the create_offer method and exchanges with a native client.

However, the ICE exchange does not work when the native client calls the create_offer method and exchanges it with a browser client. The same happens with browser-to-browser ICE exchanges as well.

I know nothing about how the ICE exchange works, so I'm just relying on what the server logs are printing.

Native-to-native ICE exchange:

server sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=rtc 3720359899 0 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic:WMS *\r\na=setup:actpass\r\na=ice-ufrag:8Fwn\r\na=ice-pwd://rsvLKicpvi5121cj4BFf\r\na=ice-options:ice2,trickle\r\na=fingerprint:sha-256 90:7D:3A:C2:1D:02:D5:1C:56:A1:C3:BC:E8:16:4C:14:F8:DD:DE:8E:68:60:50:16:D5:E2:57:D4:E0:C6:50:0D\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=mid:0\r\na=sendrecv\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","type":"offer"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 2122317823 LOCAL_IP 50825 typ host"}

client sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=rtc 3574127925 0 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic:WMS *\r\na=setup:active\r\na=ice-ufrag:shmG\r\na=ice-pwd:/DNWIlsirTHNf7D19t9VN5\r\na=ice-options:ice2,trickle\r\na=fingerprint:sha-256 F9:C4:3C:7B:E5:BE:90:43:A4:CB:52:0F:C8:ED:B7:EA:7D:27:4B:DD:0D:D4:E2:1E:42:34:B9:CF:28:5F:26:5F\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=mid:0\r\na=sendrecv\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","type":"answer"}
client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 2122317823 LOCAL_IP 50827 typ host"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:3 1 UDP 1686109695 PUBLIC_IP 50825 typ srflx raddr 0.0.0.0 rport 0"}
client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:2 1 UDP 1686109951 PUBLIC_IP 50827 typ srflx raddr 0.0.0.0 rport 0"}

Browser-to-native ICE exchange (browser calls create_offer):

server sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0 4820537007877044315 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=fingerprint:sha-256 B8:3E:53:3F:80:97:26:DC:AB:3A:14:83:5E:BF:FA:74:9B:C5:4B:3D:6F:8B:50:8E:0F:92:FE:B2:F7:64:F8:6C\r\na=group:BUNDLE 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=sendrecv\r\na=ice-pwd:c2012f35316d7ddf8003dfb3b50197e0\r\na=ice-ufrag:efc32a9d\r\na=mid:0\r\na=setup:actpass\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n","type":"offer"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:0 1 UDP 2122252543 740e96d7-c7ba-49c5-877d-a9198be5737d.local 55873 typ host"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:2 1 TCP 2105524479 740e96d7-c7ba-49c5-877d-a9198be5737d.local 9 typ host tcptype active"}
client sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=rtc 1812742799 0 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic:WMS *\r\na=setup:active\r\na=ice-ufrag:MNIr\r\na=ice-pwd:DP0PfkbSNdLPqVBmMAebEX\r\na=ice-options:ice2,trickle\r\na=fingerprint:sha-256 55:C3:07:EF:55:DE:08:9B:83:5B:B1:12:9C:74:F2:75:FD:CF:DB:40:C8:F8:1C:42:B0:B7:6A:1A:BF:9E:BA:3C\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=mid:0\r\na=sendrecv\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","type":"answer"}
client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 2122317823 LOCAL_IP 55875 typ host"}
client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:2 1 UDP 1686109951 PUBLIC_IP 55875 typ srflx raddr 0.0.0.0 rport 0"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 1686052863 PUBLIC_IP 55873 typ srflx raddr 0.0.0.0 rport 0"}

Native-to-browser ICE exchange (native calls create_offer):

server sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=rtc 4254617398 0 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic:WMS *\r\na=setup:actpass\r\na=ice-ufrag:ubR7\r\na=ice-pwd:vac2KQQMKexeI9RuLPcUyw\r\na=ice-options:ice2,trickle\r\na=fingerprint:sha-256 35:31:5F:95:8A:35:82:17:89:BC:20:62:A1:8E:73:B5:ED:FE:37:AA:8D:13:71:67:D2:4C:32:E2:5E:B5:7B:68\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=mid:0\r\na=sendrecv\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","type":"offer"}
server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 2122317823 LOCAL_IP 50773 typ host"}

client sent {"cmd":"setup_webrtc_session","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-99.0 4212232542675602986 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=fingerprint:sha-256 20:20:02:B2:E4:50:BF:03:0C:86:67:8C:71:42:6D:F6:77:CA:F3:77:D7:70:15:C6:DF:F4:CC:AA:99:55:B3:89\r\na=group:BUNDLE 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=sendrecv\r\na=ice-pwd:af2c222b3842cd2ebd7cab2fda0e00a9\r\na=ice-ufrag:fde2a9b1\r\na=mid:0\r\na=setup:active\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n","type":"answer"}

server sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:2 1 UDP 1686109951 PUBLIC_IP 50773 typ srflx raddr 0.0.0.0 rport 0"}

In the native-to-browser scenario, after the client (browser) sets up its session, it does not send these packets:

client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:1 1 UDP 2122317823 LOCAL_IP 55875 typ host"}
client sent {"cmd":"add_ice_candidate","index":0,"media":"0","name":"candidate:2 1 UDP 1686109951 PUBLIC_IP 55875 typ srflx raddr 0.0.0.0 rport 0"}

Is this something that the webrtc plugin does, or is this something the built-in browser WebRTC capabilities handle? Is this intended functionality in the browser?

Steps to reproduce

Download and install the Go compiler
Download the WebRTC GDExtension
Extract the webrtc folder into the root of the project.
Start the dedicated server by running go build ; ./webrtc-test-server in the server folder.
Open the Godot project and run the HTML5 export to the server/static directory.
Open a browser to http://localhost:8080 (the websocket will fail to upgrade if you use 127.0.0.1),
Run a native client.
In the browser client, enter a server name and click Host. In the native client, enter the same server name and click Connect. It should succeed and the two clients can talk to each other.
Now refresh the browser and restart the native client, but this time, host in the native client and connect in the browser. The WebRTC ICE exchange does not complete and the browser and native client cannot talk to each other.

Minimal reproduction project

webrtc-godot.zip

Faless commented 10 months ago

I can't seem to be able to reproduce using the webrtc_signaling demo, tested with both Chromium and Firefox

ChrisTrenkamp commented 10 months ago

Indeed, I took the signaling demo, created a HTML5 export of it, and tweaked the NodeJS server to serve the HTML5 export with a static file endpoint, and everything seems to work fine.

I guess I'm doing something wrong in my demo, but it's not immediately obvious what it is.

ChrisTrenkamp commented 10 months ago

I just realized that the WebRTCPeerConnection.ice_candidate_created signal is not triggering, and it's because I forgot to call mp_connections.get_peer(MultiplayerPeer.TARGET_PEER_SERVER).get("connection").set_local_description(type, sdp) in the client. After adding that line, it works in the browser.

Out of curiosity, why do the native clients work fine without setting the local description?

Faless commented 10 months ago

Out of curiosity, why do the native clients work fine without setting the local description?

IIRC that is a quirk in the library we use, i.e. there is (was?) no way to create an offer without calling set_local_description.

We should investigate if that changed in recent versions, or if we can ask upstream to allow that.

Faless commented 10 months ago

See: https://github.com/godotengine/webrtc-native/blob/cf23e5e74038249e6c050c5610db108efd86ae85/src/WebRTCLibPeerConnection.cpp#L233-L239

godotengine / webrtc-native