Video stream interlocutor remains in black screen

j20mc commented 4 years ago

Which platform(s) does your issue occur on?

Android

Please, provide the following version numbers that your issue occurs with:

Demo

Please, tell us how to recreate the issue in as much detail as possible.

Demo

Is there any code involved?

Demo and V2

Hi there, I currently have a video stream problem that remains black during a call. The sockets communicate well but there is no video stream. I had problems with my application and I just tested on your demo and I have the same problem.

Here is the console log of your demo =>

APPLICATION :

{
JS:   "_observers": {},
JS:   "dataChannels": {},
JS:   "remoteIceCandidates": [],
JS:   "inCall": false,
JS:   "isInitiator": false,
JS:   "connection": {
JS:     "_remoteDescription": null,
JS:     "_localDescription": null,
JS:     "android": {}
JS:   },
JS:   "me": "45f5f8f5-33ce-bd12-8bae-9f338a6769c8",
JS:   "socket": {
JS:     "socket": {}
JS:   }

JS: call:incoming to: 20d5c949-67a8-83d8-87e6-6a8519254d0a from: 45f5f8f5-33ce-bd12-8bae-9f338a6769c8
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:answered
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate
JS: call:iceCandidate

SERVER :

Connected: -> OIMOHepKBiNd35BHAAAF
User Connected: -> hKwbvt7KiKS-_ORUAAAG
call from e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:iceCandidate e96223c3-63d5-2375-a355-104c891a4e53
call:answered e433a775-203c-873e-b75f-64188ab94e2f
call:iceCandidate e433a775-203c-873e-b75f-64188ab94e2f
call:iceCandidate e433a775-203c-873e-b75f-64188ab94e2f
call:iceCandidate e433a775-203c-873e-b75f-64188ab94e2f
call:iceCandidate e433a775-203c-873e-b75f-64188ab94e2f
call:iceCandidate e433a775-203c-873e-b75f-64188ab94e2f
call from e433a775-203c-873e-b75f-64188ab94e2f

As you can see, there is no error but the video stream between the two smartphones remains in black. I specify that each smartphone has its own video stream but it does not have that of the interlocutor.

triniwiz commented 4 years ago

What type of connection are you using mobile or WiFi

j20mc commented 4 years ago

I use a 4G mobile connection for my mobile and my pc wifi

triniwiz commented 4 years ago

There are a couple things you can try

Try both on WiFi
Try calling the web demo
Try another stun server even turn if you have access to one

j20mc commented 4 years ago

Ok, I will try your advice and come back to you

j20mc commented 4 years ago

I did several tests with always the same problem.

Same Wifi for both smartphones and the pc.
Same 4G mobile connection for both smartphones and pc.
I put the server application on one of my servers.
I put the server application online via ngrok.
I did not use the web demo (I don't know how to launch it).

triniwiz commented 4 years ago

Since you only mentioned the video stream the audio works fine ?

j20mc commented 4 years ago

Sorry, video & audio doesn't work

triniwiz commented 4 years ago

Ok good to know also ensure you’re using the standard example not the basic (which should be the deprecated v1) another thing to verify is https://github.com/triniwiz/nativescript-webrtc/blob/master/demo/app/standard/standard-vm.ts this is firing to add the incoming candidates

j20mc commented 4 years ago

I am using your file well and I have the iceCandidate displayed on the console. I also use the standard and not the basic. I have been trying to find out why there is this problem for several days, but I decided to ask for help

triniwiz commented 4 years ago

Yes I’m seeing it logged but can you add some extra logging to verify the add method is being called ?

j20mc commented 4 years ago

addIceCandidate is called

j20mc commented 4 years ago

call:incoming to: 3e861fde-0f22-9241-9479-95f9f6ddf26e from: 7eb009d4-c608-1ff5-a1c3-4f6c96121eb2
JS: call:iceCandidate
JS: candidate candidate:4099077385 1 udp 2122260223 192.168.3.87 48023 typ host generation 0 ufrag jWQ5 network-id 3 network-cost 10
JS: call:iceCandidate
JS: candidate candidate:559267639 1 udp 2122202367 ::1 54376 typ host generation 0 ufrag jWQ5 network-id 2
JS: call:iceCandidate
JS: candidate candidate:1510613869 1 udp 2122129151 127.0.0.1 58425 typ host generation 0 ufrag jWQ5 network-id 1
JS: call:iceCandidate
JS: candidate candidate:4099077385 1 udp 2122260223 192.168.3.87 55760 typ host generation 0 ufrag jWQ5 network-id 3 network-cost 10
JS: call:iceCandidate
JS: candidate candidate:559267639 1 udp 2122202367 ::1 37523 typ host generation 0 ufrag jWQ5 network-id 2
JS: call:iceCandidate
JS: candidate candidate:1510613869 1 udp 2122129151 127.0.0.1 56640 typ host generation 0 ufrag jWQ5 network-id 1
JS: call:iceCandidate
JS: candidate candidate:1876313031 1 tcp 1518222591 ::1 44656 typ host tcptype passive generation 0 ufrag jWQ5 network-id 2
JS: call:iceCandidate
JS: candidate candidate:344579997 1 tcp 1518149375 127.0.0.1 47123 typ host tcptype passive generation 0 ufrag jWQ5 network-id 1
JS: call:iceCandidate
JS: candidate candidate:1876313031 1 tcp 1518222591 ::1 50444 typ host tcptype passive generation 0 ufrag jWQ5 network-id 2
JS: call:iceCandidate
JS: candidate candidate:344579997 1 tcp 1518149375 127.0.0.1 58483 typ host tcptype passive generation 0 ufrag jWQ5 network-id 1
JS: call:answered
JS: call:iceCandidate
JS: candidate candidate:2787615239 1 udp 2122260223 10.250.96.80 44201 typ host generation 0 ufrag oXIG network-id 3 network-cost 900
JS: call:iceCandidate
JS: candidate candidate:559267639 1 udp 2122202367 ::1 37909 typ host generation 0 ufrag oXIG network-id 2
JS: call:iceCandidate
JS: candidate candidate:1510613869 1 udp 2122129151 127.0.0.1 45265 typ host generation 0 ufrag oXIG network-id 1
JS: call:iceCandidate
JS: candidate candidate:1876313031 1 tcp 1518222591 ::1 41755 typ host tcptype passive generation 0 ufrag oXIG network-id 2
JS: call:iceCandidate
JS: candidate candidate:344579997 1 tcp 1518149375 127.0.0.1 40995 typ host tcptype passive generation 0 ufrag oXIG network-id 1

triniwiz commented 4 years ago

Can you bind to the onConnectionStateChange event also use connectionState (really iceConnectionState will be renamed in the future) on the peerconnection it should tell you a bit more what's happening https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/iceConnectionState

j20mc commented 4 years ago

Ok, I will try your advice and come back to you

j20mc commented 4 years ago

I cannot link the onConnectionStateChange event to use connectionState. I think I lack a practice on the subject. But The function onConnectionStateChange is called only once per smartphone per call

This function blocks communication without making an error =>

this.connection.onConnectionStateChange(() => { console.log("EVENT LISTENER STATE CHANGE", this.connection.connectionState)})

abhayastudios commented 4 years ago

@j20mc the function blocks on Android right?

@triniwiz this is the same behaviour I described in #44 (I think)

triniwiz commented 4 years ago

Oh interesting I’ll need some time to look into it also that callback needs to be renamed you can check out here

j20mc commented 4 years ago

@abhayastudios Yes indeed, you seem to understand why it doesn't work, can you tell me more?

abhayastudios commented 4 years ago

@j20mc actually for me it is working with branch alpha22 if both devices are on the same (Wifi) network, or if one of the devices has all UDP traffic forwarded to the device private IP. My challenge is that I can't seem to make it work when the devices are on symmetrical NAT in which case a TURN server is need (if I understand correctly).

j20mc commented 4 years ago

Thank you for the explanations. Based on my research, I too came across a TURN server solution. Little question about a 4G mobile network at my place it does not work and you?

j20mc commented 4 years ago

To see if the problem can be solved via a TURN server, I add my server url instead of the STUN google servers. I could see that there is an error in file TNSRTCConfiguration.ts on the line:

if (Array.isArray(options)) { instead of if (Array.isArray(options.iceServers)) {

Otherwise it will never run.

However I can't call the TURN server using this function : this.connection = new TNSRTCPeerConnection(config);

abhayastudios commented 4 years ago

@j20mc first off: nice catch :) Once I fix that line in the alpha22 branch then I can actually see that it starts adding the IceServers from Twilio that I add on Android (on iOS it is already working seems).

However, like for you, it is still not working from me if I am on different networks. Apparently there is something else going on. Will need to investigate some more...

abhayastudios commented 3 years ago

@j20mc and anyone else still struggling with this can you please watch my comment on #44 and see whether this solves your issue?

triniwiz / nativescript-webrtc