search

ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.28k stars 631 forks source link

Audio issues on AntMedia server 2.10.0 (WebRTC Multitrack) #6520

Open dalkofjac opened 3 months ago

dalkofjac commented 3 months ago

Short description

As we've deployed the AntMedia server version "Enterprise Edition 2.10.0" to production servers few days ago, the users are starting to use it in real-life scenarios and unfortunately we've stumbled across an issue with audio. There was a streaming session few days ago where users reported the audio publishing wasn't working at all for the audio-only publishers, meaning no one could hear them speaking. The logs show that all of the clients successfully played their audio stream, but still, publishers couldn't be heard. FYI, for this audio streaming we're using the AntMedia's multitrack stream functionality with "maxAudioTrackCount=3".

I'm adding all the AntMedia server logs for you to have a look. The audio stream (with multiple audio tracks) I'm referencing here is the following: "lQ4L0LcwNMbo5205471030727401".

Upon analyzing the logs myself I've seen the following error pop up all the time:

ERROR i.a.enterprise.webrtc.WebRTCClient - java.lang.NullPointerException: Cannot invoke "org.webrtc.PeerConnectionFactory.createAudioSource(org.webrtc.MediaConstraints)" because "this.peerConnectionFactory" is null       at io.antmedia.enterprise.webrtc.WebRTCClient.startNewWebRTCAudioTracks(WebRTCClient.java:1136)       at io.antmedia.enterprise.webrtc.WebRTCClient.createNewAudioTrack(WebRTCClient.java:545)       at io.antmedia.enterprise.webrtc.WebRTCClient.newSubtrackAdded(WebRTCClient.java:487)       at io.antmedia.enterprise.webrtc.WebRTCClient.lambda$addTrackOnTheFly$2(WebRTCClient.java:454)       at io.antmedia.enterprise.webrtc.WebRTCClient.lambda$executeOnSignallingThread$24(WebRTCClient.java:1566)       at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:279)       at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:261)       at io.vertx.core.impl.ContextInternal.lambda$runOnContext$0(ContextInternal.java:59)       at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)       at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)       at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)       at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)       at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)       at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)       at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)       at java.base/java.lang.Thread.run(Thread.java:840)

Environment

Steps to reproduce

So far not reproducible.

Expected behavior

The multitrack stream processes the audio tracks normally so every player hears the publishers talking.

Actual behavior

Nobody heard the audio publishers talking.

Logs

dalkofjac commented 3 months ago

@burak-58 @mekya @amarantmeida @Mohit-3196 Referencing you here to have a look as we've usually worked with you guys on solving the issues. Please let us know as soon as you have some feedback as we're looking into downgrading the AntMedia server if we experience this issue again (seems as serious one). Thanks!

burak-58 commented 3 months ago

Hi @dalkofjac, Today we checked the issue with @Mohit-3196 and I requested him to create the issue. It seems that it is a bug we need to fix. I will assign the issue myself and hope that provide a solution next week.

burak-58 commented 3 months ago

Hi @dalkofjac, I have tried to reproduce the issue in my local enviroment with the following scenario. I could hear the audio for the audio only participants. Let me share the scenario and please tell me if I do something wrong.

  1. I have added audio only option into conference.html page here and use it.
  2. I limit the audio tracks with 1.
  3. I add video+audio participants and audio only participants in the same room. 4 I check if the participants can here the audio only participants.

Please verify this scenario if it is similar to your real scenario.

dalkofjac commented 3 months ago

Hi @burak-58,

As far as I see, the main difference between your local scenario and our production scenario is that we limit the audio tracks to "3" instead of "1", meaning up to 3 attendees can speak at the same time. In the live case where the issue was experienced, at least 2 people had voice privileges (meaning their audio track was published to the audio-only stream). Also, it's important to mention that we use our own AntMedia server implementation for web app, very similar to the one you linked but not the same.

Also, it's important to note that we aren't able to reproduce this issue, it was reported just this one time. Do the logs reveal anything useful (e.g. the NullPointerException)?

We can also go through all of these details on a live meeting, let me know (as discussed over email).