ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.22k stars 618 forks source link

Android SDK publish stops after few seconds (~50 seconds) enterperise edition #2362

Closed nagendraK6 closed 4 years ago

nagendraK6 commented 4 years ago

Short description

Brief description of what happened I tried webrtc android sample app with enterprise edition of the server. After couple of seconds I see that publishing is getting stopped. On further investigation found that ice candidate receives the disconnect. On close analysis I found that stun server returned stun server binding request timed out and then entire ice connection disconnects. I tried with new stun server as wll (after hosting on aws) but faced the same problem

On the same android device if I try with webview js (in browser) then no issue and there is no disconnect from server.

Environment

Steps to reproduce

  1. install enterprise server
  2. change server url and stream key in sample mode, change the mode to publishing mode
  3. run on physical android device. After enabling permissions publishing starts but then 30-40 seconda later it stops.

Expected behavior

Put as much detail here as possible Ice connection shouldn't disconnect.

Actual behavior

Put as much detail here as possible Publishing should continue as long as user has not stopped it

Logs

Place logs on pastebin or elsewhere and put links here

2020-07-21 18:25:31.530 12013-12055/io.antmedia.webrtc_android_sample_app E/stun_port.cc: (line 96): Binding request timed out from [0:0:0:x:x:x:x:x]:56421 (wlan0) 2020-07-21 18:25:31.638 12013-12055/io.antmedia.webrtc_android_sample_app E/stun_port.cc: (line 96): Binding request timed out from [0:0:0:x:x:x:x:x]:58209 (radio0) 2020-07-21 18:25:31.819 12013-12055/io.antmedia.webrtc_android_sample_app E/stun_port.cc: (line 96): Binding request timed out from [0:0:0:x:x:x:x:x]:39676 (lo) 2020-07-21 18:25:31.819 12013-12055/io.antmedia.webrtc_android_sample_app E/stun_port.cc: (line 96): Binding request timed out from 127.0.0.x:52523 (lo)

Join Google Group and be a part of Ant Media Server Community.

nagendraK6 commented 4 years ago

I debugged it further and tried with different websocket client (okhttp). Finally found that ICEConnection state is changing to disconnected and sometimes closed. On the server side, I found the error websocket enterprise error. No further details in logs.

Maybe somehow the server is getting a bad packet and then peer connection from the server is getting disconnected and that is what changes to ICEConnection disconnected

This issue is reported earlier as well. Any idea what could go wrong here? I and my cofounder are evaluating red5pro vs antmedia enterprise and the android client publishing is critical for us. Wondering if anyone can unblock on this.It will help us to evaluate the experience end to end.

cc: @mekya

mekya commented 4 years ago

Hey @nagendraK6 , Are you using our sample application for that?

In normal cases, we don't encounter this case.
I think I can make a test on my end with my android device and one of our test server in the cloud and then I'll update you.

Thank you for reporting.

nagendraK6 commented 4 years ago

Yes. It is with the sample application provided.

mekya commented 4 years ago

Hey @nagendraK6 , I made a quick test on my end. I've used real android device(Samsung S5, Android 6). I've used a remote server(not a local one). After 5 minutes of non-stop publishing, I stop the streaming.

I've just added a screenshot below.

Screen Shot 2020-07-23 at 20 14 52

Basically problem may be something below.

nagendraK6 commented 4 years ago

I have a few more observations. My video server is behind AWS load balancer. By default, the AWS application load balancer creates two instances of LB. So let's say URL is video.abc.com then ping can return different ip addresses in round robin (IP of LB). I believe in this case when call starts maybe after some time different IP is detected as a result ICE connection state is updated and the callback is received. In my setup I have one ant media video server behind aws lb but due to lb when I ping then I receive two different IPs in round robin.

I tried without load balancer by directly mapping URL to video server IP and things work as expected.

Do you have any recommendation on how to setup aws load balancer for ant media servers?

nagendraK6 commented 4 years ago

Issue resolve. Root cause id is default AWS idle timeout. Once I modified it everything worked

mekya commented 4 years ago

Hi @nagendraK6 ,

Thank you for the feedback. There is also ping/pong messages in the SDK that handles this load balancer timeout.

Could you please search in the SDK that has a method with startPingPongTimer? It should be in WebSocketHandler.java

nagendraK6 commented 4 years ago

@mekya You are right. Ping/Pong is supposed to handle this. There is a pingPongTimoutCount that tracks any timeout and it is reset to 0 when we receive pong command. Looking at logs I can see that when aws LB used then the server is not returning pong message

2020-07-24 01:47:38.775 12985-12985/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong timer is started 2020-07-24 01:47:41.776 12985-13150/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong timer is executed 2020-07-24 01:47:41.777 12985-13150/io.antmedia.webrtc_android_sample_app W/WSChannelRTCClient: Ping/Pong message sent 2020-07-24 01:47:43.777 12985-13150/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong timer is executed 2020-07-24 01:47:43.778 12985-13150/io.antmedia.webrtc_android_sample_app W/WSChannelRTCClient: Ping/Pong message sent 2020-07-24 01:47:45.776 12985-13150/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong timer is executed 2020-07-24 01:47:45.777 12985-13150/io.antmedia.webrtc_android_sample_app W/WSChannelRTCClient: Ping/Pong message sent 2020-07-24 01:47:45.777 12985-13150/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong websocket response not received for 4 seconds 2020-07-24 01:47:45.777 12985-13150/io.antmedia.webrtc_android_sample_app D/WSChannelRTCClient: Ping Pong timer stop called

I will check tomorrow if any port issue on my LB, which may be causing any issue. All other messages to exchange ICE is also going through the same websocket so should be fine.

nagendraK6 commented 4 years ago

@mekya I think there is a small bug that may be on the server side. The server is not sending the stream id in pong messages. As a result, the code hits exception and logic to validate that pong message is not executed and client stops the timer. Once the timer stops there is no ping message from the client and then a 60 seconds timer of AWS LB expires and the socket is disconnected.

pong payload 2020-07-24 02:04:56.823 14910-14910/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:04:58.827 14910-14910/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:00.818 14910-14910/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:24.489 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:26.465 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:28.461 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:30.464 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:32.464 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:05:34.471 15248-15248/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"} 2020-07-24 02:08:08.395 15766-15766/io.antmedia.webrtc_android_sample_app E/WSChannelRTCClient: onTextMessage: {"command":"pong"}

mekya commented 4 years ago

You're great. Thank you very much. I found the problem in the SDK. I'm sending a fix for the SDK right now. It'll be available to download later today.

Generally server is not supposed to send stream id for the pong message. The problem is in the SDK side.

mekya commented 4 years ago

I've sent the fix and it'll available to download later today.

meet-zimozi commented 3 years ago

@mekya I am facing the same issue when ( android os >=10) please give the solution .

mekya commented 3 years ago

Hi @meet-zimozi ,

Are you sure that you have opened the server ports and you can publish from your web browser?

If you cannot publish from your desktop web browser, please make sure that ports are open.

https://github.com/ant-media/Ant-Media-Server/wiki/Installation#server-ports

meet-zimozi commented 3 years ago

yes i am able to stream using browser even if i am using android 9 then can able to stream the problem i am facing is only for android 10 and 11.


From: Ahmet Oğuz Mermerkaya @.> Sent: Friday, August 27, 2021 3:13 PM To: ant-media/Ant-Media-Server @.> Cc: Meet Chandra @.>; Mention @.> Subject: Re: [ant-media/Ant-Media-Server] Android SDK publish stops after few seconds (~50 seconds) enterperise edition (#2362)

Hi @meet-zimozihttps://github.com/meet-zimozi ,

Are you sure that you have opened the server ports and you can publish from your web browser?

If you cannot publish from your desktop web browser, please make sure that ports are open.

https://github.com/ant-media/Ant-Media-Server/wiki/Installation#server-ports

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ant-media/Ant-Media-Server/issues/2362#issuecomment-907072561, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AURQPOQTMRG36FGSXHOMHBLT65M53ANCNFSM4PEGSGPA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.