search

sipsorcery-org / sipsorcery

A WebRTC, SIP and VoIP library for C# and .NET. Designed for real-time communications apps.
https://sipsorcery-org.github.io/sipsorcery
Other
1.38k stars 423 forks source link

ice-mismatch when calling an UA based on PJSIP #827

Open maguoloalceo opened 1 year ago

maguoloalceo commented 1 year ago

Hi,

I'm trying to call a SIP client based on PJSIP, but the SDP negotiation fails because the PJSIP client answer with an attribute a=ice-mismatch.

I think I should modify the offer sent, but don't know what exactly is causing this problem. Maybe the c= line with the 0.0.0.0 or the m= line where the port specified is 9?

This is the SDP request

o=- 97624 0 IN IP4 127.0.0.1
s=sipsorcery
t=0 0
a=group:BUNDLE 0
m=audio 9 UDP/TLS/RTP/SAVP 0 8 9 18 101
c=IN IP4 0.0.0.0
a=ice-ufrag:WQRP
a=ice-pwd:SNOILPENSPUKVWCYBFAVDOJD
a=fingerprint:sha-256 91:F1:09:DC:8D:9A:AF:BD:C8:BE:D7:56:8B:4A:7B:AC:11:34:9B:F9:17:30:5D:AA:83:C3:80:4C:E6:0F:09:06
a=setup:actpass
a=candidate:4821 1 udp 8447 95.110.191.15 51868 typ relay raddr 0.0.0.0 rport 0 generation 0
a=candidate:4370 1 udp 1677730047 62.94.78.6 12446 typ srflx raddr 0.0.0.0 rport 0 generation 0
a=candidate:2436 1 udp 2113937663 192.168.1.47 59921 typ host generation 0
a=candidate:3088 1 udp 2113940223 2001:470:b46c:1:c0b7:6a2:a4c:8dc9 59921 typ host generation 0
a=candidate:2548 1 udp 2113940223 2001:470:b46c:1:6d52:d90d:b26a:273d 59921 typ host generation 0
a=candidate:2963 1 udp 2113940223 2001:470:b46c:1::135 59921 typ host generation 0
a=mid:0
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:9 G722/8000
a=rtpmap:18 G729/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ice-options:ice2,trickle
a=rtcp-mux
a=rtcp:9 IN IP4 0.0.0.0
a=sendrecv
a=ssrc:2081282689 cname:12fd34f7-bd61-47e5-a8d5-9adcb924ce64

and this is the SDP answer:

o=- 3870241133 3870241134 IN IP4 192.168.1.122
s=bcs
t=0 0
m=audio 56044 UDP/TLS/RTP/SAVP 0 101
c=IN IP4 95.110.191.15
b=TIAS:96000
a=rtcp:57094 IN IP4 95.110.191.15
a=sendrecv
a=rtpmap:0 PCMU/8000
a=ice-mismatch
a=setup:active
a=fingerprint:SHA-256 14:BF:01:9C:AA:9B:A9:4A:49:19:C0:6D:CF:40:49:AF:E6:9C:1A:53:17:E2:7C:35:93:39:79:6E:08:D8:F4:A6
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
Ephron-WL commented 1 year ago

from the ICE RFC:

"ice-mismatch" is a media-level
   attribute only, and when present in an answer, indicates that the
   offer arrived with a default destination for a media component that
   didn't have a corresponding candidate attribute.

[candidate attribute.] provides one of many possible candidate
      addresses for communication.  These addresses are validated with
      an end-to-end connectivity check using Session Traversal Utilities
      for NAT (STUN)).

[ice-mismatch] indicates that an agent is ICE capable,
      but did not proceed with ICE due to a mismatch of candidates with
      the default destination for media signaled in the SDP.

Default Destination/Candidate:  The default destination for a
      component of a media stream is the transport address that would be
      used by an agent that is not ICE aware.  For the RTP component,
      the default IP address is in the c line of the SDP, and the port
      is in the m line.  For the RTCP component, it is in the rtcp
      attribute when present, and when not present, the IP address is in
      the c line and 1 plus the port is in the m line.  A default
      candidate for a component is one whose transport address matches
      the default destination for that component.

 Once all of the media streams are completed, the controlling endpoint
   sends an updated offer if the candidates in the m and c lines for the
   media stream (called the DEFAULT CANDIDATES) don't match ICE's
   SELECTED CANDIDATES.

I'd say that your suspicions are correct. The c and m lines must be your default candidates. But, they are essentially undefined in your offer. IANA defines port 9 as "discard", probably like NULL.

maguoloalceo commented 1 year ago

Thank you for your answer, what I was trying to do is to make a call from Sipsorcery to another user agent using an RTCPeerConnection object instead of the "standard" VoIPMediaSession object to enable NAT traversal, but it seems something is wrong with the way I'm using the RTCPeerConnection object.

Do you have any idea of what might be causing this problem?

Ephron-WL commented 1 year ago

I'm having a hard time understanding the goal of your project. Are you trying to bridge SIP and WebRTC or are you trying to use SIP and need to use STUN with SIP (which is supported, you don't have to go to WebRTC).

I'm learning this too, so bear with me.

I learned that the c and the m lines may in fact be inconsequential. It would appear that the ICE and other servers will construct the candidates for us. Below I describe my implementation (learning proof of concept) today and I give you the websocket communication (i.e. signaling) between the two nodes as seen from Chrome, which might help you understanding how they are communicating.

I setup a dev environment:

System 1: WebRTCReceive example running under TLS (SSL) on the websocket. It is bound to the loopback interface. It is behind a NAT server. It has a conventional 192.168... IP. This example just listens for a video feed.

System 2: A Windows 10 system outside of previous LAN. A public IP has been assigned to the NIC. I had to run the capture.html file from IISExpress under SSL. If I don't do this, Chrome will create a mDNS name for my address, which is not supported in SIPSorcery (it can't be supported unless the server network has an mDNS server). Once my client is on IISExpress and then I use a valid SSL certificate for the websocket listener on the other machine, I can begin my connection process. The other issue with this system is the firewall. It is the standard Windows default settings and as such if it were behind a NAT service, the NAT service would hold open the socket connection for inbound RTP packets (I could be wrong about how the connection remains open). In this case, SIPSorcery attempts to check for a TURN server. If one (e.g. https://www.metered.ca/tools/openrelay/) is added to the list of IceServers, then it will be used to relay the media through it. So, in my case, having the TURN server defined was important to maintain the security on this, admittedly, unusual deployment.

When I opened the ports and STUN was used instead of TURN the following websocket messages were seen.

System 2 (client) connects to System 1 (server) and System 1 sends an offer to System 2:

v=0
o=- 74601 0 IN IP4 127.0.0.1
s=sipsorcery
t=0 0
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVP 0 101
c=IN IP4 0.0.0.0
a=ice-ufrag:ZXKP
a=ice-pwd:WNLIFZARFAKSCXJPASBBKHSF
a=fingerprint:sha-256 79:60:C2:E5:03:97:7E:F8:6C:9D:C4:0E:6C:4D:0D:50:E9:48:8D:6A:2C:FE:64:A2:F4:74:03:5F:AF:17:94:78
a=setup:actpass
a=candidate:2165547373 1 udp 2113937663 192.168.5.9 63281 typ host generation 0
a=ice-options:ice2,trickle
a=mid:0
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=rtcp-mux
a=rtcp:9 IN IP4 0.0.0.0
a=recvonly
a=ssrc:150200670 cname:781c4c44-76c5-4eef-a3bb-9a0c43c43544
m=video 9 UDP/TLS/RTP/SAVP 100
c=IN IP4 0.0.0.0
a=ice-ufrag:ZXKP
a=ice-pwd:WNLIFZARFAKSCXJPASBBKHSF
a=fingerprint:sha-256 79:60:C2:E5:03:97:7E:F8:6C:9D:C4:0E:6C:4D:0D:50:E9:48:8D:6A:2C:FE:64:A2:F4:74:03:5F:AF:17:94:78
a=setup:actpass
a=ice-options:ice2,trickle
a=mid:1
a=rtpmap:100 H264/90000
a=rtcp-mux
a=rtcp:9 IN IP4 0.0.0.0
a=recvonly
a=ssrc:1724587531 cname:feb70946-7c5c-47a5-9eaf-36a8509db4df

System 2 responds with the answer:

v=0
o=- 6281606354925479729 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS xd4b4eGW3IH6epQgim2ZELZmZh2Trbk5MtQu
m=audio 9 UDP/TLS/RTP/SAVP 0 101
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:SdgF
a=ice-pwd:RVyYRq0ZeQWpA/r2Wi7ncrF+
a=ice-options:trickle
a=fingerprint:sha-256 8C:79:3B:04:60:D3:22:25:24:0F:90:F5:32:27:9F:0A:F3:C9:6A:0D:14:91:1F:78:B7:29:18:8F:76:DE:4C:CF
a=setup:active
a=mid:0
a=sendonly
a=msid:xd4b4eGW3IH6epQgim2ZELZmZh2Trbk5MtQu ee03f74d-bd80-4221-bc4e-fe0999919900
a=rtcp-mux
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=ssrc:4148423679 cname:9nb4/QkQTndJqP2o
a=ssrc:4148423679 msid:xd4b4eGW3IH6epQgim2ZELZmZh2Trbk5MtQu ee03f74d-bd80-4221-bc4e-fe0999919900
m=video 9 UDP/TLS/RTP/SAVP 100
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:SdgF
a=ice-pwd:RVyYRq0ZeQWpA/r2Wi7ncrF+
a=ice-options:trickle
a=fingerprint:sha-256 8C:79:3B:04:60:D3:22:25:24:0F:90:F5:32:27:9F:0A:F3:C9:6A:0D:14:91:1F:78:B7:29:18:8F:76:DE:4C:CF
a=setup:active
a=mid:1
a=sendonly
a=msid:xd4b4eGW3IH6epQgim2ZELZmZh2Trbk5MtQu 9514453c-6a9b-4187-9eb5-58a62fabdf12
a=rtcp-mux
a=rtpmap:100 H264/90000
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=ssrc:3246671904 cname:9nb4/QkQTndJqP2o
a=ssrc:3246671904 msid:xd4b4eGW3IH6epQgim2ZELZmZh2Trbk5MtQu 9514453c-6a9b-4187-9eb5-58a62fabdf12

The lack of candidate in this SDP message may relate to the fact that the IP of the system is public, so it doesn't need to communicate any internal IP address.

System 2 also receives two candidates: candidate:2865850267 1 udp 2122260223 47.181.203.186 53442 typ host generation 0 ufrag SdgF network-id 1 then immediately after candidate:3830618987 1 tcp 1518280447 47.181.203.186 9 typ host tcptype active generation 0 ufrag SdgF network-id 1

System 2 responds with its candidate: candidate:2725182690 1 udp 1677730047 47.181.203.185 2456 typ srflx raddr 0.0.0.0 rport 0 generation 0

When the TURN server is an option and necessary for the RTP stream you will see System 2 send another candidate, the one of the TURN server, the type being relay.

See Udp Candidate Types

In every configuration it worked like a champ, a testament to the soundness of the SiPSorcery library.

maguoloalceo commented 1 year ago

I'm sorry to not be clear enough, what I'm trying to do is to use STUN (and eventually TURN to handle symmetric NAT scenarios) with SIP, and I thought using the WebRTC objects was the way to go.

Can you point me to an example of using STUN with SIP?

Ephron-WL commented 1 year ago

Have you reviewed the AsteriskIce example, or is that the one you were originally referring?

maguoloalceo commented 1 year ago

Yes, I was using the AsteriskIce example and that's why I was trying to use the RTCPeerConnection object.

Ephron-WL commented 1 year ago

The SoftPhone demo has a STUN client, but maybe you want to use ICE. For ICE, maybe if you setup two demo applications, each using a RTCPeerConnection, they could then negotiate a candidate. I'll play around with it and share my findings. I'm curious too.

Ephron-WL commented 1 year ago

Essentially by combining code from the AsteriskIce and WebRTCReceiver (mainly the latter), I was able to create a demo example that allows two nodes to negotiate candidates. The screenshot below is from the server side of the negotiation that is on 47.181.203.184. The RTCPeerConnection object correctly concluded the destination endpoint to be 47.181.203.186.

I can't speak to the behavior of PJSIP. I can just tell you that the RTCPeerConnection properly resolves the peer's media address and port number using ICE. I hope the demo helps.

RTCPeerConnectionDemo

image

maguoloalceo commented 1 year ago

I rewiewed your example but I didn't found any signficant differences from my code.

Can you tell me if the SDP offer on your demo has the m line with the port set to 9 (as in my test) or instead is set to the port of the default candidate?

Ephron-WL commented 1 year ago

Below are the SDPs for the local (server, begin the NAT server) and the remote (public IP):

{v=0 o=- 86496 0 IN IP4 127.0.0.1 s=sipsorcery t=0 0 a=group:BUNDLE 0 1 m=audio 9 UDP/TLS/RTP/SAVP 0 8 9 18 101 c=IN IP4 0.0.0.0 a=ice-ufrag:XQRJ a=ice-pwd:ZVHGTECWQZISHGALNUCEENCQ a=fingerprint:sha-256 BA:94:FA:9E:C2:01:75:48:34:52:40:51:D0:05:11:ED:13:46:FE:BA:82:2C:DB:E8:6A:E3:FB:29:69:67:4B:EF a=setup:actpass a=candidate:2746 1 udp 2113937663 192.168.5.9 56067 typ host generation 0 a=mid:0 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:9 G722/8000 a=rtpmap:18 G729/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-16 a=ice-options:ice2,trickle a=rtcp-mux a=rtcp:9 IN IP4 0.0.0.0 a=sendonly a=ssrc:759128949 cname:694f290e-7f7e-4b8f-88a9-4b50932e062d m=video 9 UDP/TLS/RTP/SAVP 96 100 c=IN IP4 0.0.0.0 a=ice-ufrag:XQRJ a=ice-pwd:ZVHGTECWQZISHGALNUCEENCQ a=fingerprint:sha-256 BA:94:FA:9E:C2:01:75:48:34:52:40:51:D0:05:11:ED:13:46:FE:BA:82:2C:DB:E8:6A:E3:FB:29:69:67:4B:EF a=setup:actpass a=mid:1 a=rtpmap:96 VP8/90000 a=rtpmap:100 H264/90000 a=fmtp:100 packetization-mode=1 a=ice-options:ice2,trickle a=rtcp-mux a=rtcp:9 IN IP4 0.0.0.0 a=sendonly a=ssrc:24957359 cname:b9e12b04-dcf8-4007-a658-a040f7b4ff9d }

{v=0 o=- 27537 0 IN IP4 127.0.0.1 s=sipsorcery t=0 0 a=group:BUNDLE 0 1 m=audio 9 UDP/TLS/RTP/SAVP 0 8 9 18 101 c=IN IP4 0.0.0.0 a=ice-ufrag:RKQS a=ice-pwd:XFWNHGWWFDKMYNQLHJEMMDOO a=fingerprint:sha-256 E6:C3:2F:18:CB:36:17:BE:9E:89:79:AB:EA:86:D3:09:58:F2:82:E5:20:73:D8:56:28:E8:6F:1D:E7:48:82:3B a=setup:active a=candidate:2617 1 udp 2113937663 47.181.203.186 51845 typ host generation 0 a=mid:0 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:9 G722/8000 a=rtpmap:18 G729/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-16 a=ice-options:ice2,trickle a=rtcp-mux a=rtcp:9 IN IP4 0.0.0.0 a=sendonly a=ssrc:1266212252 cname:783a685d-422c-492c-a55e-854396535915 m=video 9 UDP/TLS/RTP/SAVP 96 100 c=IN IP4 0.0.0.0 a=ice-ufrag:RKQS a=ice-pwd:XFWNHGWWFDKMYNQLHJEMMDOO a=fingerprint:sha-256 E6:C3:2F:18:CB:36:17:BE:9E:89:79:AB:EA:86:D3:09:58:F2:82:E5:20:73:D8:56:28:E8:6F:1D:E7:48:82:3B a=setup:active a=mid:1 a=rtpmap:96 VP8/90000 a=rtpmap:100 H264/90000 a=fmtp:100 packetization-mode=1 a=ice-options:ice2,trickle a=rtcp-mux a=rtcp:9 IN IP4 0.0.0.0 a=sendonly a=ssrc:322783954 cname:1314070f-ed80-4cf2-b52c-6ff4688f34f3 }

maguoloalceo commented 1 year ago

Looking at your SDP negotiation I see that the difference between your SDP and mine is that I'm trying to implement the "half-trickle" ICE negotiation https://www.rfc-editor.org/rfc/rfc8838.html#name-half-trickle because the pjsip client I'm trying to call doesn't currently support the "full" trickle negotiation, so I collect all candidates before sendig the SDP offer.

What i don't undestand reading the ICE RFCs is if setting the m parameter with port 9 is legit when using half trickle, or instead it must be set to the default candidate. In the first case the problem is on the pjsip side, otherwise the RCTPeerConnection object must set the default candidate in the m parameter where there are candidates available.

Ephron-WL commented 1 year ago

Have you attempted to set the c and m line to a value when generating the SDP instead of relying upon the default in which nothing is defined?

maguoloalceo commented 1 year ago

Forcing the c and m line to a valid value the negotiation completes succesfully.

By the way, I get the same ice-mismatch error even with Linphone https://new.linphone.org/technical-corner/linphone?qt-technical_corner=2#qt-technical_corner an open source softphone, so it seems the problem is not related to pjsip only.

Ephron-WL commented 1 year ago

I've used Linphone in the past. Why do you think this ice-mismatch occurs?

maguoloalceo commented 1 year ago

I think both pjsip and Linphone are expecting real values in the m and c parameters.

Looking at https://datatracker.ietf.org/doc/html/rfc8839#section-4.3.1

It is valid for an offer "m=" line to include no SDP "candidate" attributes and have the default destination set to the IP address values "0.0.0.0"/"::" and the port value to "9". This implies that the offering agent is only going to use peer-reflexive candidates or will provide additional candidates in subsequent signaling messages.

but I'm adding candidates to my offer, so maybe this is the source of the problem.

Ephron-WL commented 1 year ago

So, this would be part of the trickle ICE pattern and those clients don't support trickle ICE, as you mentioned earlier?

maguoloalceo commented 1 year ago

Yes, exactly.