ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.31k stars 637 forks source link

io.vertx.core.impl.BlockedThreadChecker #2558

Open salsaweb opened 4 years ago

salsaweb commented 4 years ago

Hello, have a fresh server version, also I'm newbie.

Got this error sometime when broadcasting from mobile browser, however from desktop all fine...

`WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 2348 ms, time limit is 2000

Oct 02, 2020 5:21:28 AM io.vertx.core.impl.BlockedThreadChecker WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 3348 ms, time limit is 2000

Oct 02, 2020 5:21:29 AM io.vertx.core.impl.BlockedThreadChecker WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 4348 ms, time limit is 2000

Oct 02, 2020 5:21:30 AM io.vertx.core.impl.BlockedThreadChecker WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 5348 ms, time limit is 2000 io.vertx.core.VertxException: Thread blocked at java.lang.Thread.sleep(Native Method) at io.antmedia.muxer.MuxAdaptor$1.call(MuxAdaptor.java:235) at org.bytedeco.javacpp.avformat.avformat_find_stream_info(Native Method) at io.antmedia.muxer.MuxAdaptor.prepare(MuxAdaptor.java:487) at io.antmedia.muxer.MuxAdaptor.lambda$start$6(MuxAdaptor.java:977) at io.antmedia.muxer.MuxAdaptor$$Lambda$204/1553596889.handle(Unknown Source) at io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:885) at io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:844) at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:339) at io.vertx.core.impl.ContextImpl$$Lambda$31/1623156057.run(Unknown Source) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748)`

Is it possible to increase time limit from 2000 to higher in configs? as I can see from all exist variables this one has same value rtmp.max_handling_time=2000

Thank you.

mekya commented 4 years ago

Hey @salsaweb ,

This is actually a warning that lets you know that some thread is taking long time.

I think you can make sure that you're streaming with good health to the server and make sure your server has enough CPU source. Because generally this is not supposed to happen.

pixxelspotter commented 4 years ago

We have the same Bug when sending 30 different Streams to the Server. Is this because of CPU overload? The Server has 64 Cores and a Tesla T4 GPU

mekya commented 4 years ago

This problem is something more related the way how RTMP streams are ingested. We see this bottleneck and we've improved the RTMP ingestion. It's much more simpler for now.

The improvement I've mentioned is available in 2.3.0-SNAPSHOT. It means it will be available in next release