search

sepfy / libpeer

WebRTC Library for IoT/Embedded Device using C
MIT License
804 stars 108 forks source link

TURN server support on esp32 #40

Closed rjjrbatarao closed 6 months ago

rjjrbatarao commented 10 months ago

I've tested the esp32 and works pretty well the only problem I have was it does not work on different internet provider maybe this is caused by NATs and would require TURN server for it to work.

sepfy commented 10 months ago

Hello. Thanks for your test. Actually I'm developing TURN server feature and hope I can push the code next week. May i know your test environment? Is using chrome of desktop to watch the camera video of ESP32? And both connect to different router?

rjjrbatarao commented 10 months ago

Hi thank you in advance im testing this on chrome and connects on different routers

sy1vain commented 10 months ago

I think I have the same issue. On the esp monitor I see it has two candidates, a local ip and a remote ip. But on the webpage I only see the local ip address as a remote candidate, but the logs seem to indicate it sends both IPs over mqtt. But I could be wrong. I have only been playing with it for 20 minutes.

If it indeed receives an external candidate, but that not end up on the other peer, they won't be able to open a connection.

sepfy commented 9 months ago

hello, i just pushed the code to solve an issue of stun port resolving. It could be that this issue is causing connection failed. I also added support for turn server. You can set the turn server through the following config.

{ .urls = "turn:turnserver.xxx.com:3478",
  .username = "",
  .credential = "",
},
rjjrbatarao commented 9 months ago

hi I've tested it with this setup laptop---->router/isp<-----esp-cam everything works fine with stun and turn, i can see some transactions on my turn server but with this setup laptop---->cellular-4G-lte/isp_a esp-cam---->router/isp_b it is stucked on webrtc: PeerConnectionState: 2 note: im browsing the webrtc page on the laptop to test the camera

sepfy commented 9 months ago

I think when using the same router, they are using the LAN IP as a candidate, so it works fine. I use the same environment as you, and I can connect successfully (4G + router). Have you tried using TURN Server on the browser side? Also could you provide your SDP to me? You can find these information from the log of ESP32. onicecandidate: v=0 ... and DATA={"jsonrpc":"2.0","method":"PEER_ANSWER.... Thanks

rjjrbatarao commented 9 months ago

im using your site for testing, how to do this add TURN Server on the browser side? setup laptop---->cellular-4G-lte/isp_a esp-cam---->router/isp_b

here is from the esp32 monitor

onicecandidate: v=0
o=- 1495799811084970 1495799811084970 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: iot
a=group:BUNDLE datachannel
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:datachannel
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 36:CB:AE:D2:ED:AD:86:8D:7F:7A:06:5A:0D:1B:11:A5:38:E5:1B:18:DC:95:18:3B:3D:7B:A2:E0:6E:7E:E3:02
a=setup:actpass
a=ice-ufrag:UC5V
a=ice-pwd:UC5VM7drRr1PCMpKNuYY3oKW
a=candidate:1 1 UDP 255 0.0.0.0 0 typ relay raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 1691198719 124.105.97.61 52644 typ srflx raddr 0.0.0.0 rport 0
a=candidate:3 1 UDP 2113929471 10.0.0.229 0 typ host
a=candidate:4 1 UDP 2127406335 10.0.0.229 52644 typ host

I (175718) MQTT: MQTT_EVENT_DATA
TOPIC=webrtc/esp32-30c6f704a954/jsonrpc
DATA={"jsonrpc":"2.0","method":"PEER_ANSWER","params":"dj0wDQpvPS0gNjM5MTc0Njg5ODgyNTYxODMyMCAyIElOIElQNCAxMjcuMC4wLjENCnM9LQ0KdD0wIDANCmE9Z3JvdXA6QlVORExFIGRhdGFjaGFubmVsDQphPW1zaWQtc2VtYW50aWM6IFdNUw0KbT1hcHBsaWNhdGlvbiAxNjg3MSBVRFAvRFRMUy9TQ1RQIHdlYnJ0Yy1kYXRhY2hhbm5lbA0KYz1JTiBJUDQgMTc1LjE3Ni4yOS45OA0KYT1jYW5kaWRhdGU6Mjc1OTk5MzYgMSB1ZHAgMjEyMjI2MDIyMyAxOTIuMTY4LjU2LjEgNTU5ODAgdHlwIGhvc3QgZ2VuZXJhdGlvbiAwIG5ldHdvcmstaWQgMg0KYT1jYW5kaWRhdGU6MzEyMzAyODcyIDEgdWRwIDIxMjIxOTQ2ODcgMTkyLjE2OC40My43IDU1OTgxIHR5cCBob3N0IGdlbmVyYXRpb24gMCBuZXR3b3JrLWlkIDEgbmV0d29yay1jb3N0IDEwDQphPWNhbmRpZGF0ZTozMjkwNjgzNTAgMSB1ZHAgMTY4NTk4NzA3MSAxNzUuMTc2LjI5Ljk4IDE2ODcxIHR5cCBzcmZseCByYWRkciAxOTIuMTY4LjQzLjcgcnBvcnQgNTU5ODEgZ2VuZXJhdGlvbiAwIG5ldHdvcmstaWQgMSBuZXR3b3JrLWNvc3QgMTANCmE9Y2FuZGlkYXRlOjQyNzkyMzQ3NzIgMSB0Y3AgMTUxODI4MDQ0NyAxOTIuMTY4LjU2LjEgOSB0eXAgaG9zdCB0Y3B0eXBlIGFjdGl2ZSBnZW5lcmF0aW9uIDAgbmV0d29yay1pZCAyDQphPWNhbmRpZGF0ZTozOTYzMDYyNjY4IDEgdGNwIDE1MTgyMTQ5MTEgMTkyLjE2OC40My43IDkgdHlwIGhvc3QgdGNwdHlwZSBhY3RpdmUgZ2VuZXJhdGlvbiAwIG5ldHdvcmstaWQgMSBuZXR3b3JrLWNvc3QgMTANCmE9aWNlLXVmcmFnOmpJY1cNCmE9aWNlLXB3ZDp6eFhOcC96Q2VKYmgvMGw1b0NDRC8ybDcNCmE9aWNlLW9wdGlvbnM6dHJpY2tsZQ0KYT1maW5nZXJwcmludDpzaGEtMjU2IDA4OkVDOjdEOjQzOjkxOjgyOjcyOjg0OkFFOkIzOjA2OjU5OkI3OkJGOjVFOjYwOjQ1OjRDOkIyOjZBOkFBOkQwOkE3Ojg1OjY0OjYyOkNGOkRCOkVGOkUzOjlCOkE2DQphPXNldHVwOmFjdGl2ZQ0KYT1taWQ6ZGF0YWNoYW5uZWwNCmE9c2N0cC1wb3J0OjUwMDANCmE9bWF4LW1lc3NhZ2Utc2l6ZToyNjIxNDQNCg==","id":183}
I (175848) webrtc: PeerConnectionState: 2
E (459348) esp-tls-mbedtls: read error :-0x004C:
E (459348) transport_base: esp_tls_conn_read error, errno=Socket is not connected
E (459348) mqtt_client: esp_mqtt_handle_transport_read_error: transport_read() error: errno=128
I (459358) MQTT: MQTT_EVENT_ERROR
I (459368) MQTT: Last error code reported from esp-tls: 0x0
I (459368) MQTT: Last tls stack error number: 0x4c
I (459378) MQTT: Last captured errno : 0 (Success)
E (459378) mqtt_client: mqtt_process_receive: mqtt_message_receive() returned -1
I (459398) MQTT: MQTT_EVENT_DISCONNECTED
I (474398) MQTT: Other event id:7
I (476548) MQTT: MQTT_EVENT_CONNECTED
I (476548) MQTT: sent subscribe successful, msg_id=57494
I (476848) MQTT: MQTT_EVENT_SUBSCRIBED, msg_id=57494

Here is from the chrome debug

disconnected
?deviceId=esp32-30c6f704a954:399 v=0
o=- 6391746898825618320 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE datachannel
a=msid-semantic: WMS
m=application 16871 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 175.176.29.98
a=candidate:27599936 1 udp 2122260223 192.168.56.1 55980 typ host generation 0 network-id 2
a=candidate:312302872 1 udp 2122194687 192.168.43.7 55981 typ host generation 0 network-id 1 network-cost 10
a=candidate:329068350 1 udp 1685987071 175.176.29.98 16871 typ srflx raddr 192.168.43.7 rport 55981 generation 0 network-id 1 network-cost 10
a=candidate:4279234772 1 tcp 1518280447 192.168.56.1 9 typ host tcptype active generation 0 network-id 2
a=candidate:3963062668 1 tcp 1518214911 192.168.43.7 9 typ host tcptype active generation 0 network-id 1 network-cost 10
a=ice-ufrag:jIcW
a=ice-pwd:zxXNp/zCeJbh/0l5oCCD/2l7
a=ice-options:trickle
a=fingerprint:sha-256 08:EC:7D:43:91:82:72:84:AE:B3:06:59:B7:BF:5E:60:45:4C:B2:6A:AA:D0:A7:85:64:62:CF:DB:EF:E3:9B:A6
a=setup:active
a=mid:datachannel
a=sctp-port:5000
a=max-message-size:262144

?deviceId=esp32-30c6f704a954:406 {jsonrpc: '2.0', method: 'PEER_ANSWER', params: 'dj0wDQpvPS0gNjM5MTc0Njg5ODgyNTYxODMyMCAyIElOIElQNC…0OjUwMDANCmE9bWF4LW1lc3NhZ2Utc2l6ZToyNjIxNDQNCg==', id: 183}
rjjrbatarao commented 9 months ago

it looks like its connected now but no video, it says disconnected, i tested it with https://www.metered.ca/ and expressturn same behaviour

sepfy commented 9 months ago

I found the relay candidate is failed in your case a=candidate:1 1 UDP 255 0.0.0.0 0 typ relay raddr 0.0.0.0 rport 0 I tested the metered and expressturn. expressturn looks not works ( turn server no response), event I use the JavaScript to get candidate, it's still not working, so I skip it first. For metered, libpeer closes the socket too fast, so it cannot receive the relay information in time. I increase the socket timeout for waiting the response of turn server f67472f. it should fix the relay candidate issue.

rjjrbatarao commented 9 months ago

Hi i tried increasing the block to 7 seconds, still wont establish, im using metered stun relay

onicecandidate: v=0
o=- 1495799811084970 1495799811084970 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic: iot
a=group:BUNDLE datachannel
m=application 50712 UDP/DTLS/SCTP webrtc-datachannel
a=mid:datachannel
a=sctp-port:5000
c=IN IP4 0.0.0.0
a=max-message-size:262144
a=fingerprint:sha-256 37:4D:1D:13:A8:E0:4F:C9:0D:A3:5C:D0:22:5E:D9:74:64:3E:63:23:07:DE:7F:D1:D9:BF:33:E8:66:4D:19:EF
a=setup:actpass
a=ice-ufrag:6XUf
a=ice-pwd:6XUfw8Ks4IvvAui3JP2avu18
a=candidate:1 1 UDP 1693220863 124.105.97.61 60543 typ srflx raddr 0.0.0.0 rport 0
a=candidate:2 1 UDP 13703167 216.39.253.22 53527 typ relay raddr 0.0.0.0 rport 0
a=candidate:3 1 UDP 9791743 216.39.253.22 38248 typ relay raddr 0.0.0.0 rport 0
a=candidate:4 1 UDP 2129428479 10.0.0.229 60543 typ host
a=candidate:5 1 UDP 2127632383 10.0.0.229 53527 typ host
a=candidate:6 1 UDP 2123720959 10.0.0.229 38248 typ host
rjjrbatarao commented 7 months ago

here are some additional information: I (41545) webrtc: PeerConnectionState: 1 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_connection.c 281 ice_servers: stun:stun.relay.metered.ca:80

INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 163 resolved_addr.ipv4: 51.75.92.195 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 165 resolved_addr.port: 80 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x802b ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x802c ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x8022 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_connection.c 281 ice_servers: turn:a.relay.metered.ca:80

INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 163 resolved_addr.ipv4: 216.39.253.22 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 165 resolved_addr.port: 80 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x0009 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x8022 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x8022 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_connection.c 281 ice_servers: turn:a.relay.metered.ca:443 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 163 resolved_addr.ipv4: 216.39.253.22 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/agent.c 165 resolved_addr.port: 443 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x0009 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x8022 ERROR C:/Users/lan/Documents/Next-Fi/libpeer/src/stun.c 196 Unknown Attribute Type: 0x8022 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH I (84575) webrtc: PeerConnectionState: 2 INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH INFO C:/Users/lan/Documents/Next-Fi/libpeer/src/peer_signaling.c 271 MQTT_PACKET_TYPE_PUBLISH

areabow commented 7 months ago

Hi, I'm also experiencing a similar issue. My setup is laptop-->LTE/4G (ISP A) and ESPCAM-->WiFi/Router (ISP B) and using STUN. Log output on ESP is stuck on:

I (95656) webrtc: PeerConnectionState: 2

With debug output:

DEBUG   /libpeer/src/agent.c    396     send binding request to remote ip: 192.168.43.83, port: 53419
DEBUG   /libpeer/src/agent.c    434     timeout for nominate (pair 0)
DEBUG   /libpeer/src/udp.c      61      bind_addr: 0x3f81f5a8
DEBUG   /libpeer/src/agent.c    396     send binding request to remote ip: 41.116.137.120, port: 9762
DEBUG   /libpeer/src/agent.c    434     timeout for nominate (pair 1)
DEBUG  /libpeer/src/udp.c      61      bind_addr: 0x3f81f5a8
DEBUG   /libpeer/src/agent.c    396     send binding request to remote ip: 192.168.43.83, port: 53419
DEBUG   /libpeer/src/agent.c    434     timeout for nominate (pair 2)

In the browser console SDP is correct and receive answer ok is shown in console, then after above timeout, message disconnected is given.

I do not get this error when both peers are connected on the same network.

Any ideas on how to resolve this?

rjjrbatarao commented 6 months ago

I used eturnal and its been connecting since then and indeed its using my turn server, our isp is so restrictive then.