search

EricssonResearch / openwebrtc

A cross-platform WebRTC client framework based on GStreamer
http://www.openwebrtc.org
BSD 2-Clause "Simplified" License
1.8k stars 537 forks source link

TURN support #575

Open xelven opened 8 years ago

xelven commented 8 years ago

One basic question, since I try to figure out what's my problem, ref: https://github.com/EricssonResearch/openwebrtc-ios-sdk/issues/59

but I found after I print all of the data from remote NICE candidate, there is NOT any TURN info,

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.1 NiceCandidateType: CANDIDATE_TYPE_HOST,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 192.168.1.207,base_addr: ,priority: 2122260223, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.2 NiceCandidateType: CANDIDATE_TYPE_HOST,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 192.168.18.132,base_addr: ,priority: 2122194687, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.3 NiceCandidateType: CANDIDATE_TYPE_SERVER_REFLEXIVE,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 220.130.33.227,base_addr: ,priority: 1686052607, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.4 NiceCandidateType: CANDIDATE_TYPE_SERVER_REFLEXIVE,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 116.226.127.167,base_addr: ,priority: 1685987071, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.5 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 41885439, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.6 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 41885439, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.7 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 41819903, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.8 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 41819903, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.9 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 25108223, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN

(<unknown>:3569): libnice-CRITICAL **: file address.c: line 280 (nice_address_to_string): should not be reached
nice_remote_candidate No.10 NiceCandidateType: CANDIDATE_TYPE_RELAY,NiceCandidateTransport: TRANSPORT_TYPE_UDP,addr: 54.178.56.14,base_addr: ,priority: 25108223, stream_id: 1, component_id: 1,username: mC3vpYCw/WK+/hWa,password:jXLq0MSHZLqzial7Ki9hM1jb, 
NO TURN
local_candidate No.1 candidate_type: CANDIDATE_TYPE_HOST,component_type: COMPONENT_TYPE_RTP,foundation: 2833646159,transport_type: TRANSPORT_TYPE_UDP,address: 192.168.1.207, port: 62390, priority: 2122260223,related_address: ,related_port:0
local_candidate No.2 candidate_type: CANDIDATE_TYPE_HOST,component_type: COMPONENT_TYPE_RTP,foundation: 1966334295,transport_type: TRANSPORT_TYPE_UDP,address: 192.168.18.132, port: 54597, priority: 2122194687,related_address: ,related_port:0
local_candidate No.3 candidate_type: CANDIDATE_TYPE_SERVER_REFLEXIVE,component_type: COMPONENT_TYPE_RTP,foundation: 699270395,transport_type: TRANSPORT_TYPE_UDP,address: 220.130.33.227, port: 62390, priority: 1686052607,related_address: ,related_port:0
local_candidate No.4 candidate_type: CANDIDATE_TYPE_SERVER_REFLEXIVE,component_type: COMPONENT_TYPE_RTP,foundation: 3116293828,transport_type: TRANSPORT_TYPE_UDP,address: 116.226.127.167, port: 54597, priority: 1685987071,related_address: ,related_port:0
local_candidate No.5 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 51638, priority: 41885439,related_address: ,related_port:0
local_candidate No.6 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 58119, priority: 41885439,related_address: ,related_port:0
local_candidate No.7 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 61101, priority: 41819903,related_address: ,related_port:0
local_candidate No.8 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 54455, priority: 41819903,related_address: ,related_port:0
local_candidate No.9 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 50767, priority: 25108223,related_address: ,related_port:0
local_candidate No.10 candidate_type: CANDIDATE_TYPE_RELAY,component_type: COMPONENT_TYPE_RTP,foundation: 841301765,transport_type: TRANSPORT_TYPE_UDP,address: 54.178.56.14, port: 61385, priority: 25108223,related_address: ,related_port:0

then I take look the code in owr_transport_agent.c and owr_candidate.c

NiceCandidate * _owr_candidate_to_nice_candidate(OwrCandidate *candidate)
{
    OwrCandidatePrivate *priv;
    NiceCandidate *nice_candidate;
    NiceCandidateType candidate_type;
    NiceComponentType component_type;
    NiceCandidateTransport transport;

    g_return_val_if_fail(candidate, NULL);

    priv = candidate->priv;

    switch (priv->type) {
    case OWR_CANDIDATE_TYPE_HOST:
        candidate_type = NICE_CANDIDATE_TYPE_HOST;
        break;

    case OWR_CANDIDATE_TYPE_SERVER_REFLEXIVE:
        candidate_type = NICE_CANDIDATE_TYPE_SERVER_REFLEXIVE;
        break;

    case OWR_CANDIDATE_TYPE_PEER_REFLEXIVE:
        candidate_type = NICE_CANDIDATE_TYPE_PEER_REFLEXIVE;
        break;

    case OWR_CANDIDATE_TYPE_RELAY:
        candidate_type = NICE_CANDIDATE_TYPE_RELAYED;
        break;

    default:
        g_return_val_if_reached(NULL);
    }

    switch (priv->component_type) {
    case OWR_COMPONENT_TYPE_RTP:
        component_type = NICE_COMPONENT_TYPE_RTP;
        break;

    case OWR_COMPONENT_TYPE_RTCP:
        component_type = NICE_COMPONENT_TYPE_RTCP;
        break;

    default:
        g_return_val_if_reached(NULL);
    }

    switch (priv->transport_type) {
    case OWR_TRANSPORT_TYPE_UDP:
        transport = NICE_CANDIDATE_TRANSPORT_UDP;
        break;

    case OWR_TRANSPORT_TYPE_TCP_ACTIVE:
        transport = NICE_CANDIDATE_TRANSPORT_TCP_ACTIVE;
        break;

    case OWR_TRANSPORT_TYPE_TCP_PASSIVE:
        transport = NICE_CANDIDATE_TRANSPORT_TCP_PASSIVE;
        break;

    case OWR_TRANSPORT_TYPE_TCP_SO:
        transport = NICE_CANDIDATE_TRANSPORT_TCP_SO;
        break;

    default:
        g_return_val_if_reached(NULL);
    }

    g_return_val_if_fail(priv->address && strlen(priv->address) > 0, NULL);
    g_return_val_if_fail(priv->port || transport == NICE_CANDIDATE_TRANSPORT_TCP_ACTIVE, NULL);

    nice_candidate = nice_candidate_new(candidate_type);
    nice_candidate->transport = NICE_CANDIDATE_TRANSPORT_UDP;
    nice_candidate->component_id = component_type;
    nice_candidate->transport = transport;

    nice_address_set_from_string(&nice_candidate->addr, priv->address);
    nice_address_set_port(&nice_candidate->addr, priv->port);

    if (priv->base_address && strlen(priv->base_address) > 0)
        nice_address_set_from_string(&nice_candidate->base_addr, priv->base_address);
    if (priv->base_port)
        nice_address_set_port(&nice_candidate->base_addr, priv->base_port);

    if (priv->foundation && strlen(priv->foundation) > 0) {
        g_strlcpy((gchar *)&nice_candidate->foundation, priv->foundation,
            MIN(NICE_CANDIDATE_MAX_FOUNDATION, 1 + strlen(priv->foundation)));
    }

    nice_candidate->priority = candidate->priv->priority;

    if (priv->ufrag && strlen(priv->ufrag) > 0)
        nice_candidate->username = g_strdup(priv->ufrag);
    if (priv->password && strlen(priv->password) > 0)
        nice_candidate->password = g_strdup(priv->password);

    return nice_candidate;
}

there is don't need to set any TURN server info set?

struct _TurnServer
{
  gint ref_count;

  NiceAddress server;
  gchar *username;
  gchar *password;
  NiceRelayType type;
};
stefanalund commented 8 years ago

We are investigating.

stefanalund commented 8 years ago

@alessandrod is looking in to this.

xelven commented 8 years ago

@stefanalund thanks, and may this is one of issues: https://phabricator.freedesktop.org/T7336

xelven commented 8 years ago

updated status here: https://phabricator.freedesktop.org/D786#15517

stefanalund commented 8 years ago

@alessandrod maybe you can comment since I know you have some info about this?

alessandrod commented 8 years ago

I'm looking into TURN bugs. I don't think it's useful to have a meta-bug for TURN so i'm going to report the issues individually.

alessandrod commented 8 years ago

Referencing some of the issues I've had so far:

https://github.com/EricssonResearch/openwebrtc/issues/589 https://github.com/EricssonResearch/openwebrtc/issues/590 https://github.com/EricssonResearch/openwebrtc/issues/591 https://github.com/EricssonResearch/openwebrtc-examples/issues/168

xelven commented 8 years ago

@alessandrod thanks for u man, and let me share what I know now. we was contact the developer of libnice, seems there have an bugs in STUN/TRUN in libnice 0.1.13 such as

libnice recently which fix bugs like these (connection problems with peer reflexive and TURN candidates), so you might want to try libnice master in a few weeks' time once all the changes have landed.

and at the same time, I build my own version libnice base on version: 1732c7d6a7a104438412309373818e493a2504c9) , and I am looking the RFC 3489, 5389, 5766 and 5245 now, also got few question now. such as the each Priority didnt follow the RFC yet.

xelven commented 8 years ago

I think I fixed my problem between 4G n Wifi the TURN relay issue, definitely is libnice problem, and I don't have time to wait, so I modified a lot for this in libnice. I still test on this but until now 100% connection and seems pretty fast now😉

urbaniak commented 8 years ago

@xelven can you share your patched libnice?

I'm happy to build and test that cause I'm also looking at some issues with TURN on iOS.

stefhak commented 8 years ago

@xelven nice work! I assume you're looking into upstreaming this to the libnice project?

xelven commented 8 years ago

after tested few days, my audio call got 100%, but unfortunately the video call base on 2 stream though NAT got problem in my version. I got take look into it. @urbaniak I used latest version of libnice it kind difference between 0.1.13. @stefhak yep, if I think I still need to take looking for while.

alessandrod commented 8 years ago

We've recently landed changes that improve TURN support. See https://github.com/EricssonResearch/cerbero/pull/51. It would be great if you could try rebuilding libnice with cerbero master.

With those changes and my own TURN server I get 100% audio/video working with Firefox and NativeDemo. There are still issues with public TURN servers which usually have very restrictive quotas (1-2 sessions max). When TURN requests fail because of quota limits, libnice (and so openwebrtc) fail silently and stuff just appears broken.

xelven commented 8 years ago

Hi, didn't update for a while, got really busy. let me update my latest status, I also using my own TURN server and there are no qoutas limit. For my video problem please take look this:https://github.com/EricssonResearch/openwebrtc-ios-sdk/issues/67 and Yes I fixed that.

I modify the latest ver of libnice are work for me, I won't used 0.1.13 cause it've other issues such as peer reflex, etc.

I was take look @alessandrod your patch for the Allocation Mismatch responses and allow REALM to be empty seems it didn't my case. my turn server's REALM name is not empty and didn't get error code 437 allocation mismatch. But still thanks for that complete the libnice.

pongponglau commented 8 years ago

Dear Alessandrod,

Thank you very much for your TURN patches. I have tried, and i still encounter the issue that i have reported before (https://github.com/EricssonResearch/bowser/issues/79).

My scenario has 2 endpoints, Bowser in iPad Pro and Win7 Chrome via TURN.

Case A: 1) Bowser join 2) Chrome join 3) Bowser call

Bowser can receive Chrome's video and audio through TURN. After a while, Chrome's video is frozen for sometimes, then move again, then frozen again, ....

Case B: 1) Bowser join 2) Chrome join 3) Chrome call

Bowser cannot receive any audio and video from Chrome, seems the TURN connection cannot be made successfully.

The libnice version is 0.1.13. I have enabled the environmental variable G_MESSAGE_DEBUG and NICE_DEBUG in xcode 7.3. However, i cannot find any useful hints.

I have also investigated the TURN server (Google coturn) runtime console log. However, i cannot find useful hints too.

May i ask what i can do in order to find out more useful information about the problems and resolve the issues?

Thank you very much.

PP

pongponglau commented 8 years ago

I have 1 more finding.

During the video is frozen, I can still hear the remote audio in Bowser clearly (as reported in issue #79 before). Besides, I use a network monitoring appl on iPad Pro to monitor the data outflow/inflow during the video is frozen. To my surprise, there is data inflow during the video is frozen (in a testing with ONLY video enabled and audio disabled) receiving at a similar speed during which the video is working normally.

Hope this information is useful for your consideration.

Thank you very much.

pongponglau commented 8 years ago

After more investigation, i find that the one-way video issue on iPad Pro should be the same/similar issue as reported in https://github.com/EricssonResearch/openwebrtc/issues/299.

May i seek for your advice/help on fixing the issue?

Many thanks