Open opg7371 opened 4 years ago
j1elo
commented
4 years ago It worries me that the whole provided backtrace is totally contained in the GStreamer library code and there is nothing that can lead us to some point in the code of Kurento.
There were some concurrency issues that got solved with Kurento 6.14, could you test that? Upgrade instructions are here: Local Upgrade
opg7371
commented
4 years ago Thanks for such a quick reply. The same thing that you mentioned also worried me. I have upgraded my release to 6.14 and will soon do a quick load testing.
In the meantime, can I ask you a favor?
we wanted to test something rather complex with the kurento media server.
Description - Subscriber A calls subscriber B and their call goes through a sip server. Call between subscriber A and subscriber B is established and RTP flows in between the endpoint points of A and B. Now, A decides to call the third subscriber C and as soon as the call is picked up, we want the RTP of all the 3 subscribers should flow through the KMS.
We ran the Group call tutorial in kurento to find out the JSON request and responses sent and received by the server respectively, To our knowledge, we found out the following -
1- As soon as 1 subscriber Joins the room 1 Create RTPENDPOINT 1 Process Offer
2- As soon as 2nd Subscriber joins the room 3 CREATE RTPENDPOINT 3 Connect RTPENDPOINT 3 Process Offer
3- As soon as 3rd Subscriber joins the room 5 CREATE RTPENDPOINT 9 Connect RTPENDPOINT 5 Process Offer
For each subscriber, kurento will create N RTPENDPOINTs (N total number of members in the room) out of which 1 RTPENDPOINT will be sendonly and others remaining will be recvonly. Is this observation correct?
Here is something we have thought will work, Assuming the RTPENDPOINT for subscriber A are denoted in an array A[N] and similarly B[N] and C[N] for subscriber A and C respectively.
Assuming for keeping things simple, let say N=3
1- We will GenerateOffer for A[0], B[1] and C[2].
2- We will send these SDP in the Re-INVITE message to A, B, and C.
3- For the SDP recieved in the response of 200OK from A, we will do ProcessOffer(B[0]), ProcessOffer(C[0]), ProcessAnswer(A[0])
4- For the SDP recieved in the response of 200OK from B, we will do ProcessOffer(A[1]), ProcessOffer(C[1]), ProcessAnswer(B[1])
5- For the SDP recieved in the response of 200OK from C, we will do ProcessOffer(A[2]), ProcessOffer(B[2]), ProcessAnswer(C[2])
We believe that after all these processing, the RTP for subscribers A, B, and C will flow through kurento. Are we missing something or is this correct?
We implemented the above-mentioned implementation and found out that - When A calls B and A calls C to create a conference group, 1- A can hear the voice of Both B and C. 2- B can hear the voice of only A and not C. 3- C cannot hear the voice of anyone.
Please tell me what are we missing? Our application code is in C and we are using WebSocket to send request in JSON format.
opg7371
commented
4 years ago The following core was observed after running it on Release 6.14
#0 0x00007f999df92428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007f999df9402a in __GI_abort () at abort.c:89
#2 0x000055bd1a73d769 in Debug::DeathHandler::SignalHandler(int, void*, void*) ()
#3 0x00007f99a06b0390 in <signal handler called> () at /lib/x86_64-linux-gnu/libpthread.so.0
#4 0x00007f999f5edb15 in g_mutex_lock (mutex=0x18) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gthread-posix.c:1335
#5 0x00007f999fb02d1b in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#6 0x00007f999f5a61e6 in g_source_unref_internal (source=0x7f980ca41350, context=0x55bd1b8a5d10, have_lock=1)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:2048
#7 0x00007f999f5a644e in g_source_iter_next (iter=iter@entry=0x7ffc90742c90, source=source@entry=0x7ffc90742c88)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:953
#8 0x00007f999f5a8cea in g_main_context_check (context=context@entry=0x55bd1b8a5d10, max_priority=2147483647, fds=fds@entry=0x55bd1c5fd800, n_fds=n_fds@entry=481) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3661
#9 0x00007f999f5a9330 in g_main_context_iterate (context=0x55bd1b8a5d10, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3837
#10 0x00007f999f5a9712 in g_main_loop_run (loop=0x55bd1b8a5e50) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:4034
#11 0x000055bd1a6f474c in main ()
Please help
opg7371
commented
4 years ago Here is what is observed in errors.log
2020-06-29T09:05:46+00:00 -- New execution
Segmentation fault (thread 140297814491712, pid 23217)
Stack trace:
[g_mutex_lock]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gthread-posix.c:1335
[gst_buffer_pool_set_flushing]
/usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0:0x3FD1B
[g_source_unref_internal]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:2050
[g_source_iter_next]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:953 (discriminator 2)
[g_main_context_check]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3661
[g_main_context_iterate]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3837
[g_main_loop_run]
/build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:4033
[??]
??:0
0x55BD1A6F474C at /usr/bin/kurento-media-server
[__libc_start_main]
csu/libc-start.c:325
[??]
??:0
0x55BD1A6F6639 at /usr/bin/kurento-media-server
Can you provide (here or privately) a core dump file so I can analyze it? Also could you provide a bit more info about the conditions when this crash happened?
j1elo
commented
4 years ago Also I'll reply to your question not here but in the forum thread: https://groups.google.com/g/kurento/c/EmPhwkH5KAw/m/EXyZrk-VBQAJ
but in short, yes, that's a good architecture.
j1elo
commented
4 years ago Oops, I closed by mistake. Reopening now.
opg7371
commented
4 years ago Also I'll reply to your question not here but in the forum thread: https://groups.google.com/g/kurento/c/EmPhwkH5KAw/m/EXyZrk-VBQAJ
but in short, yes, that's a good architecture.
Pardon me for asking here. I do know that this section is particularly for bugs and issues. Thank you for your reply. Much much much gratitude from our side
opg7371
commented
4 years ago Can you provide (here or privately) a core dump file so I can analyze it?
I would love to provide the core dump file to you but the size of the core-dump is nearly 57GB. I don't have the bandwidth for such and nor do I have any shared cloud storage on which I can host it. If you have a way of file sharing, I am happy to follow it. Being said that, I would be happy to print out any stack or thread that you want me to do.
Also could you provide a bit more info about the conditions when this crash happened?
Of course, We are using the kurento media server (release 6.14) on Ubuntu 16.04. We have 15 Cores, 32 GB RAM, and nearly 500 GB HDD. When one call is established (RTP starts flowing), then we initiate a sequence of nearly 9 messages (involving Create Mediapipeline, Create RTPendpoint, Connecting them, processOffer, GenerateOffer, and ProcessAnswer). When the call ends, we do the release of the pipeline.
Being said that, we are running a load of nearly 15 cps, Meaning at any given second, kurento will process nearly 150 requests.
When the concurrent call reaches somewhat around 700-800 calls, Kurento crashes, but it is completely random. The hold time of the call is 110 seconds and we are using sipp to generate a load of media.
- Was the CPU or RAM saturated? (100% resource usage)
Indeed, I don't think RAM was saturated(don't have the logs at the time of crash), but the kurento CPU usage was nearly 1200% out of 1500% for a very small node. This has indeed made me suspicious but after reading several issues, you have mentioned that it is normal in bulk load as kurento will adjust the media packets which are displaced due to some reason. (pardon my grammar).
- How many connections there were in the room?
We are not using kurento room. What do you mean by room? We are just doing a simple load testing. We feel that there are nearly 800 * (Number of RTP port connections needed in-between EP-A and EP B). EP: Endpoint
- Was it just audio or also video streams?
It was just audio stream. As of now, we do not have any use of video streams. Transcoding is also disabled. I have verified through the logs.
- Was it all RtpEndpoints or there was something else, such as PlayerEndpoint or RecorderEndpoints in the Kurento Pipeline?
There are all RTPEndpoints and for each call, we allocate 2 RTP endpoint. There is nothing else when we are doing load testing.
- Did you find a more or less reliable way to reproduce the crash voluntarily, or is it random?
If I pump sufficient load for nearly 10 minutes, then I can reproduce the crash easily. But due to some constraints, we are now pumping a limited load of nearly 1000 calls with rate of 15cps, so in this case, it is random.
- Did you notice any behavioral patterns such as crash happens when some user disconnects (what device is that user using?), or crash happens when the session is closed, etc.
No, I have not noticed any behavioral patterns as mentioned above. But, I do have observed that our session just closes randomly and we are debugging the same.
May you please refer us to some previous release that is stable enough so we can do some load testing with our application?
Also, Initially, I have enabled print during the sent and received of message through WebSocket and after some time of 1-2 minutes, I can see there is a lag of nearly 2-3 seconds in between the message sent and received. Can kurento be the reason for delayed processing?
I would love to understand the code of kurento. If it is possible, can you redirect me to a bug which may be of beginner level?
Also, if we are running kurento on Docker and using apt-get to install kurento instead of a docker image due to some reason, will it cause a problem?
Please let me know if there is any other way i can help you?
opg7371
commented
4 years ago Some important log I feel :
bt
(gdb) bt
#0 0x00007f999df92428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007f999df9402a in __GI_abort () at abort.c:89
#2 0x000055bd1a73d769 in Debug::DeathHandler::SignalHandler(int, void*, void*) ()
#3 0x00007f99a06b0390 in <signal handler called> () at /lib/x86_64-linux-gnu/libpthread.so.0
#4 0x00007f999f5edb15 in g_mutex_lock (mutex=0x18) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gthread-posix.c:1335
#5 0x00007f999fb02d1b in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#6 0x00007f999f5a61e6 in g_source_unref_internal (source=0x7f980ca41350, context=0x55bd1b8a5d10, have_lock=1)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:2048
#7 0x00007f999f5a644e in g_source_iter_next (iter=iter@entry=0x7ffc90742c90, source=source@entry=0x7ffc90742c88)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:953
#8 0x00007f999f5a8cea in g_main_context_check (context=context@entry=0x55bd1b8a5d10, max_priority=2147483647, fds=fds@entry=0x55bd1c5fd800, n_fds=n_fds@entry=481)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3661
#9 0x00007f999f5a9330 in g_main_context_iterate (context=0x55bd1b8a5d10, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>)
at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:3837
#10 0x00007f999f5a9712 in g_main_loop_run (loop=0x55bd1b8a5e50) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gmain.c:4034
#11 0x000055bd1a6f474c in main ()
bt of thread 2
(gdb) bt of thread 1
#0 0x00007f99a06a798d in pthread_join (threadid=140236176922368, thread_return=thread_return@entry=0x0) at pthread_join.c:90
#1 0x00007f999f5edb77 in g_system_thread_wait (thread=0x7f979922b1e0) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gthread-posix.c:1211
#2 0x00007f999f5d007a in g_thread_join (thread=0x7f979922b1e0) at /build/glib2.0-xkQkqE/glib2.0-2.48.2/./glib/gthread.c:948
#3 0x00007f995050edaf in () at /usr/lib/x86_64-linux-gnu/gstreamer-1.5/libgstrtpmanager.so
#4 0x00007f999fb1c24e in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#5 0x00007f999fb1c9c7 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#6 0x00007f999fafb515 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#7 0x00007f999fb1c24e in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#8 0x00007f999fb1c9c7 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#9 0x00007f999fafb515 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#10 0x00007f999fb1c24e in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#11 0x00007f999fb1c2a5 in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#12 0x00007f999fb1c9c7 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.5.so.0
#13 0x00007f999f282e4a in kurento::MediaElementImpl::~MediaElementImpl() () at /usr/lib/x86_64-linux-gnu/libkmscoreimpl.so.6
#14 0x00007f999a5dc538 in kurento::RtpEndpointImpl::~RtpEndpointImpl() () at /usr/lib/x86_64-linux-gnu/libkmselementsimpl.so.6
#15 0x00007f999a5dc5d9 in kurento::RtpEndpointImpl::~RtpEndpointImpl() () at /usr/lib/x86_64-linux-gnu/libkmselementsimpl.so.6
#16 0x00007f999f2276b8 in std::_Function_handler<void (), std::_Bind<void (*(kurento::MediaObjectImpl*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >))(kurento::MediaObjectImpl*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)> >::_M_invoke(std::_Any_data const&) ()
at /usr/lib/x86_64-linux-gnu/libkmscoreimpl.so.6
#17 0x00007f999f21f972 in boost::asio::detail::completion_handler<std::function<void ()> >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) () at /usr/lib/x86_64-linux-gnu/libkmscoreimpl.so.6
#18 0x00007f999f237555 in boost::asio::io_service::run() () at /usr/lib/x86_64-linux-gnu/libkmscoreimpl.so.6
#19 0x00007f99a0e245d5 in () at /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
#20 0x00007f99a06a66ba in start_thread (arg=0x7f99929ba700) at pthread_create.c:333
#21 0x00007f999e06441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Total 6414 threads can be seen in the gdb of the core. I can paste them if you like. Please let me know.
j1elo
commented
4 years ago Those backtraces are not very useful because debug symbols are not installed. Please do as suggested in the Troubleshooting Media Server Crashes doc section, and Install debug symbols so we can see the file names and line numbers where the crash happens.
Also if it is possible it would help that you run with the AddressSanitizer flavor of the nightly Docker image: https://hub.docker.com/r/kurento/kurento-media-server-dev
The complete Docker image name to use is kurento/kurento-media-server-dev:latest-asan:
$ docker pull kurento/kurento-media-server-dev:latest-asan
$ docker run [...] kurento/kurento-media-server-dev:latest-asan
$ docker logs --follow kmsHey @j1elo, I feel that I may have resolved the error that I mentioned in this very first comment.
It appears that the maximum limit on the number of threads in my server, i.e, the stack size of the thread was 8192. For a 15cps load with a hold time of 110 seconds, total calls (one call = media flowing between 2 RTP endpoints) were nearly 1650calls. I don't know the number of threads kurento opens up for 1 call, but this limit was reached and it was behaving as a bottleneck due to which this error was occurring.
For nearly 1700 calls, the total thread count reached nearly 9150 threads. When the kurento started, the total thread count was 115.
This was resolved by just setting the
ulimit -s unlimitedThe same can be done by making changes in
/etc/security/limits.confPlease correct me if I am wrong. But ever since, I have done that, the crash was not observed.
But a few other problems were observed in media pipeline implementation. I will keep you posted and have opened the necessary debugging to track these issues. I will let you know regarding these issues shortly
j1elo
commented
4 years ago Note that with ulimit -s you are setting the stack size limit for the current shell and its child processes, not the thread limit:
https://manpages.ubuntu.com/manpages/bionic/en/man1/sh.1.html
The thread size limit is, by default, set to half of the absolute system limit. You can change that in the file /etc/default/kurento-media-server, here
But if your issue was actually caused by KMS reaching the default stack size, maybe we should consider increasing that too. We'll have to do some more testing to be sure that this is actually a problem.
opg7371
commented
4 years ago Hey, i observed the following error, can you help
0:02:28.611209630 25547 0x14f1e00e51e0 ERROR rtpsynchronizer kmsrtpsynchronizer.c:415:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14f1941392d0> Invalid SSRC (3390112438), not matching with 3390111768
0:02:28.628488640 25547 0x55c78f984a00 ERROR KurentoMediaPipelineImpl MediaPipelineImpl.cpp:69:processBusMessage: Error code 1: 'Internal data flow error.', element: udpsrc1386, parent: pipeline347
0:02:28.628564786 25547 0x55c78f984a00 ERROR KurentoMediaPipelineImpl MediaPipelineImpl.cpp:72:processBusMessage: Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline347/KmsRtpEndpoint:kmsrtpendpoint693/KmsRtpSession:kmsrtpsession693/GstUDPSrc:udpsrc1386:
streaming task paused, reason not-linked (-1)
0:02:28.628630894 25547 0x55c78f984a00 ERROR KurentoMediaElementImpl MediaElementImpl.cpp:266:processBusMessage: Error code 1: 'Internal data flow error.', element: udpsrc1386, parent: kmsrtpendpoint693
0:02:28.628651210 25547 0x55c78f984a00 ERROR KurentoMediaElementImpl MediaElementImpl.cpp:269:processBusMessage: Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline347/KmsRtpEndpoint:kmsrtpendpoint693/KmsRtpSession:kmsrtpsession693/GstUDPSrc:udpsrc1386:
streaming task paused, reason not-linked (-1)
I will enable the debug log and will let you know more
opg7371
commented
4 years ago We were doing the transcoding in between AMR-WB and PCMA. Following logs were observed.
0:05:21.845295757 10265 0x7f2130008e30 INFO KurentoWebSocketTransport WebSocketTransport.cpp:550:openHandler: Client connected from http://192.168.105.203
0:05:21.845335037 10265 0x7f2130008e30 ERROR KurentoWebSocketTransport WebSocketTransport.cpp:558:openHandler: Invalid path "/kurento", closing connection
0:06:07.298324330 10265 0x7f2134001e90 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.298385908 10265 0x7f2134001e90 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"describe","params":{"object":"manager_ServerManager"},"id":"cbb39de2ebd33a11cce263d16f196d6711"}
0:06:07.298401870 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.298459922 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:07.298505763 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:376:describe: inside describe
0:06:07.298606757 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.298620676 10265 0x7f2134001e90 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.298643833 10265 0x7f2134001e90 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.298687182 10265 0x7f2134001e90 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.298704816 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.298743418 10265 0x7f2134001e90 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6711","jsonrpc":"2.0","result":{"hierarchy":["kurento.MediaObject"],"qualifiedType":"kurento.ServerManager","sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","type":"ServerManager"}}
0:06:07.298757214 10265 0x7f2134001e90 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.298805855 10265 0x7f2134001e90 INFO KurentoServerMethods ServerMethods.cpp:303:keepAliveSession: inside keepAliveSession
0:06:07.299550456 10265 0x7f212c008430 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.299571923 10265 0x7f212c008430 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"create","params":{"type":"MediaPipeline","properties":{},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6712"}
0:06:07.299590324 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.299612199 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.299640551 10265 0x7f212c008430 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.299671344 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:675:create: LOGS to see flow
0:06:07.299691257 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:683:create: LOGS to see flow
0:06:07.299710052 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:175:checkResources: LOGS to see flow
0:06:07.299718009 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:72:getMaxThreads: LOGS to see flow
0:06:07.299726844 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:88:checkThreads: LOGS to see flow
0:06:07.299806105 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:61:getNumberOfThreads: LOGS to see flow
0:06:07.299842715 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:40:get_int: LOGS to see flow
0:06:07.299854059 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:116:getMaxOpenFiles: LOGS to see flow
0:06:07.299862335 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:150:checkOpenFiles: LOGS to see flow
0:06:07.299887772 10265 0x7f212c008430 INFO KurentoResourceManager ResourceManager.cpp:136:getNumberOfOpenFiles: LOGS to see flow
0:06:07.300186596 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.300203986 10265 0x7f212c008430 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.300231844 10265 0x7f212c008430 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.300253190 10265 0x7f212c008430 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.300267275 10265 0x7f212c008430 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.300317965 10265 0x7f212c008430 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6712","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline"}}
0:06:07.300332389 10265 0x7f212c008430 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.300988483 10265 0x7f212001ef00 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.301003914 10265 0x7f212001ef00 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"create","params":{"type":"RtpEndpoint","constructorParams":{"mediaPipeline":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline"},"properties":{},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6713"}
0:06:07.301021608 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.301043461 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.301070208 10265 0x7f212001ef00 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.301097330 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:675:create: LOGS to see flow
0:06:07.301108309 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:683:create: LOGS to see flow
0:06:07.301123292 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:175:checkResources: LOGS to see flow
0:06:07.301131211 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:72:getMaxThreads: LOGS to see flow
0:06:07.301139942 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:88:checkThreads: LOGS to see flow
0:06:07.301171139 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:61:getNumberOfThreads: LOGS to see flow
0:06:07.301208545 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:40:get_int: LOGS to see flow
0:06:07.301220399 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:116:getMaxOpenFiles: LOGS to see flow
0:06:07.301228930 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:150:checkOpenFiles: LOGS to see flow
0:06:07.301246087 10265 0x7f212001ef00 INFO KurentoResourceManager ResourceManager.cpp:136:getNumberOfOpenFiles: LOGS to see flow
0:06:07.301484614 10265 0x7f212001ef00 INFO rtpsynchronizer kmsrtpsynchronizer.c:141:kms_rtp_synchronizer_init_stats_file:<KmsRtpSynchronizer@0x7f21300071f0> No path for stats; enable with env variable: 'KMS_RTP_SYNC_STATS_PATH'
0:06:07.301500207 10265 0x7f212001ef00 INFO rtpsynchronizer kmsrtpsynchronizer.c:141:kms_rtp_synchronizer_init_stats_file:<KmsRtpSynchronizer@0x7f21300072a0> No path for stats; enable with env variable: 'KMS_RTP_SYNC_STATS_PATH'
0:06:07.302745693 10265 0x7f212001ef00 WARN structure gststructure.c:195:gst_structure_validate_name: Invalid character ' ' at offset 4 in structure name: H264 SVC
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_new_empty: assertion 'gst_structure_validate_name (name)' failed
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_free: assertion 'structure != NULL' failed
0:06:07.303200933 10265 0x7f212001ef00 INFO KurentoBaseRtpEndpointImpl BaseRtpEndpointImpl.cpp:114:BaseRtpEndpointImpl: No predefined RTP MTU found in config; using default
0:06:07.303709034 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.303736029 10265 0x7f212001ef00 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.303779736 10265 0x7f212001ef00 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.303807307 10265 0x7f212001ef00 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.303825774 10265 0x7f212001ef00 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.303866521 10265 0x7f212001ef00 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6713","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/117b3459-3e9f-4d04-8ddd-dfec2ec41248_kurento.RtpEndpoint"}}
0:06:07.303883629 10265 0x7f212001ef00 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.304737577 10265 0x7f2118002380 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.304755337 10265 0x7f2118002380 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"create","params":{"type":"RtpEndpoint","constructorParams":{"mediaPipeline":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline"},"properties":{},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6714"}
0:06:07.304776135 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.304817560 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.304851555 10265 0x7f2118002380 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.304884063 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:675:create: LOGS to see flow
0:06:07.304898633 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:683:create: LOGS to see flow
0:06:07.304915961 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:175:checkResources: LOGS to see flow
0:06:07.304926317 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:72:getMaxThreads: LOGS to see flow
0:06:07.304937874 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:88:checkThreads: LOGS to see flow
0:06:07.304988338 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:61:getNumberOfThreads: LOGS to see flow
0:06:07.305020219 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:40:get_int: LOGS to see flow
0:06:07.305033763 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:116:getMaxOpenFiles: LOGS to see flow
0:06:07.305046058 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:150:checkOpenFiles: LOGS to see flow
0:06:07.305086737 10265 0x7f2118002380 INFO KurentoResourceManager ResourceManager.cpp:136:getNumberOfOpenFiles: LOGS to see flow
0:06:07.305280338 10265 0x7f2118002380 INFO rtpsynchronizer kmsrtpsynchronizer.c:141:kms_rtp_synchronizer_init_stats_file:<KmsRtpSynchronizer@0x7f21300074b0> No path for stats; enable with env variable: 'KMS_RTP_SYNC_STATS_PATH'
0:06:07.305310311 10265 0x7f2118002380 INFO rtpsynchronizer kmsrtpsynchronizer.c:141:kms_rtp_synchronizer_init_stats_file:<KmsRtpSynchronizer@0x7f2130007400> No path for stats; enable with env variable: 'KMS_RTP_SYNC_STATS_PATH'
0:06:07.306589371 10265 0x7f2118002380 WARN structure gststructure.c:195:gst_structure_validate_name: Invalid character ' ' at offset 4 in structure name: H264 SVC
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_new_empty: assertion 'gst_structure_validate_name (name)' failed
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_free: assertion 'structure != NULL' failed
0:06:07.306975547 10265 0x7f2118002380 INFO KurentoBaseRtpEndpointImpl BaseRtpEndpointImpl.cpp:114:BaseRtpEndpointImpl: No predefined RTP MTU found in config; using default
0:06:07.307363250 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.307386485 10265 0x7f2118002380 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.307421102 10265 0x7f2118002380 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.307448027 10265 0x7f2118002380 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.307466614 10265 0x7f2118002380 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.307506565 10265 0x7f2118002380 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6714","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/c85d1bf4-92a8-44c9-9dcd-82b4039d790b_kurento.RtpEndpoint"}}
0:06:07.307541424 10265 0x7f2118002380 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.308352751 10265 0x7f211c002120 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.308370546 10265 0x7f211c002120 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"invoke","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/c85d1bf4-92a8-44c9-9dcd-82b4039d790b_kurento.RtpEndpoint","operation":"connect","operationParams":{"sink":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/117b3459-3e9f-4d04-8ddd-dfec2ec41248_kurento.RtpEndpoint"},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6715"}
0:06:07.308393482 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.308429551 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.308462842 10265 0x7f211c002120 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.308497995 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:602:invoke: LOGS to see flow
0:06:07.308511593 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:07.308864958 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.308887390 10265 0x7f211c002120 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.308920946 10265 0x7f211c002120 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.308947963 10265 0x7f211c002120 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.308965602 10265 0x7f211c002120 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.309003623 10265 0x7f211c002120 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6715","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":null}}
0:06:07.309035936 10265 0x7f211c002120 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.309675122 10265 0x7f2124013590 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.309692595 10265 0x7f2124013590 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"invoke","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/117b3459-3e9f-4d04-8ddd-dfec2ec41248_kurento.RtpEndpoint","operation":"connect","operationParams":{"sink":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/c85d1bf4-92a8-44c9-9dcd-82b4039d790b_kurento.RtpEndpoint"},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6716"}
0:06:07.309714432 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.309741620 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.309774942 10265 0x7f2124013590 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.309809729 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:602:invoke: LOGS to see flow
0:06:07.309822966 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:07.310142719 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.310164728 10265 0x7f2124013590 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.310197907 10265 0x7f2124013590 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.310224947 10265 0x7f2124013590 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.310243128 10265 0x7f2124013590 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.310280779 10265 0x7f2124013590 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6716","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":null}}
0:06:07.310312130 10265 0x7f2124013590 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.311015876 10265 0x7f2128002190 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.311033457 10265 0x7f2128002190 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"invoke","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/117b3459-3e9f-4d04-8ddd-dfec2ec41248_kurento.RtpEndpoint","operation":"processOffer","operationParams":{"offer":"v=0
o=sipp 53655765 1 IN IP4 192.168.137.43
s=-
c=IN IP4 192.168.137.43
t=0 0
m=audio 6000 RTP/AVP 8 101
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=ptime:20
"},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6718"}
0:06:07.311066928 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.311093688 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.311122632 10265 0x7f2128002190 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.311149965 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:602:invoke: LOGS to see flow
0:06:07.311160321 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:07.311272163 10265 0x7f2128002190 INFO kmssdpsession kmssdpsession.c:95:kms_sdp_session_process_offer:<kmsrtpsession6> Process SDP Offer:
v=0
o=sipp 53655765 1 IN IP4 192.168.137.43
s=-
c=IN IP4 192.168.137.43
t=0 0
m=audio 6000 RTP/AVP 8 101
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=ptime:20
0:06:07.311360619 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: PCMA/8000
0:06:07.311383452 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: PCMU/8000
0:06:07.311395616 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: speex/8000
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_get_name: assertion 'structure != NULL' failed
0:06:07.311444806 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: (null)
0:06:07.311457715 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311468452 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311477968 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311487036 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311496255 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311505529 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311514626 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311528639 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311542472 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311568316 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311584176 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311597233 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311611221 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311624283 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311637506 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311689555 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311703051 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311712667 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311721981 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311731486 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311740820 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311749841 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311759010 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311767985 10265 0x7f2128002190 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.311785751 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: H265
0:06:07.311799779 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: theora
0:06:07.311812184 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: iLBC
0:06:07.311824253 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: PCMA-WB
0:06:07.311836756 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: PCMU-WB
0:06:07.311849525 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G718
0:06:07.311861502 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G719
0:06:07.311873932 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G7221
0:06:07.311892539 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G726-16
0:06:07.311904548 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G726-24
0:06:07.311917180 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G726-32
0:06:07.311929938 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G726-40
0:06:07.311942309 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G729D
0:06:07.311959260 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G729E
0:06:07.311971942 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: G7291
0:06:07.311997267 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: GSM-EFR
0:06:07.312010887 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: GSM-HR-08
0:06:07.312023135 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: speex/16000
0:06:07.312035889 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: mpa-robust
0:06:07.312048984 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: MP4A-LATM
0:06:07.312061321 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: MP4V-ES
0:06:07.312073314 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: mpeg4-generic
0:06:07.312096338 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: VP8
0:06:07.312110128 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: AMR/16000
0:06:07.312122372 10265 0x7f2128002190 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x560fe0000ea0> Add format support, media: audio, codec: opus/48000/2
0:06:07.312185524 10265 0x7f2128002190 ERROR glib GLib:0:: g_ascii_strcasecmp: assertion 's1 != NULL' failed
0:06:07.312215352 10265 0x7f2128002190 WARN sdpmidext kmssdpmidext.c:116:kms_sdp_mid_ext_add_answer_attributes:<KmsSdpMidExt@0x7f20fc001610> Remote agent does not support groups
0:06:07.313009916 10265 0x7f2128002190 INFO kmssdpsession kmssdpsession.c:140:kms_sdp_session_process_offer:<kmsrtpsession6> Generated SDP Answer:
v=0
o=- 3803193307 3803193307 IN IP4 192.168.105.203
s=Kurento Media Server
c=IN IP4 192.168.105.203
t=0 0
m=audio 43358 RTP/AVP 8 101
a=sendrecv
a=rtcp:43359
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=ptime:20
a=ssrc:410716057 cname:user74647214@host-2a7554a0
0:06:07.313690657 10265 0x7f2128002190 INFO rtpendpoint kmsrtpendpoint.c:982:kms_rtp_endpoint_start_transport_send:<kmsrtpendpoint6> COMEDIA: Media 'audio' doesn't use COMEDIA
0:06:07.313709589 10265 0x7f2128002190 INFO kmsrtpconnection kmsrtpconnection.c:91:kms_rtp_connection_set_remote_info:<KmsRtpConnection@0x7f211c0092c0> Set remote host: 192.168.137.43, RTP: 6000, RTCP: 6001
0:06:07.314416077 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.314440097 10265 0x7f2128002190 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.314479645 10265 0x7f2128002190 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.314508266 10265 0x7f2128002190 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.314526741 10265 0x7f2128002190 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.314570833 10265 0x7f2128002190 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6718","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"v=0\r\no=- 3803193307 3803193307 IN IP4 192.168.105.203\r\ns=Kurento Media Server\r\nc=IN IP4 192.168.105.203\r\nt=0 0\r\nm=audio 43358 RTP/AVP 8 101\r\na=sendrecv\r\na=rtcp:43359\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101 telephone-event/8000\r\na=fmtp:101 0-15\r\na=ptime:20\r\na=ssrc:410716057 cname:user74647214@host-2a7554a0\r\n"}}
0:06:07.314589496 10265 0x7f2128002190 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:07.315360298 10265 0x7f210c002330 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:07.315385750 10265 0x7f210c002330 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"invoke","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/c85d1bf4-92a8-44c9-9dcd-82b4039d790b_kurento.RtpEndpoint","operation":"generateOffer","sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6719"}
0:06:07.315406408 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:07.315431342 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:07.315451728 10265 0x7f210c002330 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.315473978 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:602:invoke: LOGS to see flow
0:06:07.315492731 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:07.315565949 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: PCMA/8000
0:06:07.315579416 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: PCMU/8000
0:06:07.315590182 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: speex/8000
(kurento-media-server:10265): GStreamer-CRITICAL **: gst_structure_get_name: assertion 'structure != NULL' failed
0:06:07.315630463 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: (null)
0:06:07.315650482 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315659631 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315669021 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315683207 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315699344 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315708068 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315716299 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315731986 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315744682 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315752948 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315760786 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315768679 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315777396 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315785548 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315794587 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315802550 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315810949 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315819492 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315827581 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315836448 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315853841 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315862890 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315871091 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315879819 10265 0x7f210c002330 ERROR glib GLib:0:: g_str_has_prefix: assertion 'str != NULL' failed
0:06:07.315890301 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: H265
0:06:07.315902221 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: theora
0:06:07.315912556 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: iLBC
0:06:07.315922665 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: PCMA-WB
0:06:07.315932836 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: PCMU-WB
0:06:07.315952041 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G718
0:06:07.315963238 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G719
0:06:07.315974214 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G7221
0:06:07.315985021 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G726-16
0:06:07.315995141 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G726-24
0:06:07.316005675 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G726-32
0:06:07.316016419 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G726-40
0:06:07.316026599 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G729D
0:06:07.316036813 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G729E
0:06:07.316055209 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: G7291
0:06:07.316066389 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: GSM-EFR
0:06:07.316077520 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: GSM-HR-08
0:06:07.316088170 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: speex/16000
0:06:07.316098713 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: mpa-robust
0:06:07.316109280 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: MP4A-LATM
0:06:07.316119353 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: MP4V-ES
0:06:07.316129787 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: mpeg4-generic
0:06:07.316140621 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: VP8
0:06:07.316152232 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: AMR/16000
0:06:07.316162358 10265 0x7f210c002330 INFO sdprtpavpmediahandler kmssdprtpavpmediahandler.c:1249:kms_sdp_rtp_avp_media_handler_add_codec:<KmsSdpRtpAvpfMediaHandler@0x7f2130007f50> Add format support, media: audio, codec: opus/48000/2
0:06:07.316781823 10265 0x7f210c002330 INFO kmssdpsession kmssdpsession.c:71:kms_sdp_session_generate_offer:<kmsrtpsession7> Generated SDP Offer:
v=0
o=- 3803193307 3803193307 IN IP4 192.168.105.203
s=Kurento Media Server
c=IN IP4 192.168.105.203
t=0 0
m=audio 64878 RTP/AVPF 8 0 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122
a=setup:actpass
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=rtpmap:96 speex/8000
a=rtpmap:97 (null)
a=rtpmap:98 H265
a=rtpmap:99 theora
a=rtpmap:100 iLBC
a=rtpmap:101 PCMA-WB
a=rtpmap:102 PCMU-WB
a=rtpmap:103 G718
a=rtpmap:104 G719
a=rtpmap:105 G7221
a=rtpmap:106 G726-16
a=rtpmap:107 G726-24
a=rtpmap:108 G726-32
a=rtpmap:109 G726-40
a=rtpmap:110 G729D
a=rtpmap:111 G729E
a=rtpmap:112 G7291
a=rtpmap:113 GSM-EFR
a=rtpmap:114 GSM-HR-08
a=rtpmap:115 speex/16000
a=rtpmap:116 mpa-robust
a=rtpmap:117 MP4A-LATM
a=rtpmap:118 MP4V-ES
a=rtpmap:119 mpeg4-generic
a=rtpmap:120 VP8
a=rtpmap:121 AMR/16000
a=rtpmap:122 opus/48000/2
a=rtcp:64879
a=sendrecv
a=mid:audio0
a=ssrc:1751747321 cname:user1494734174@host-cbb2c6b
0:06:07.316897026 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:07.316908650 10265 0x7f210c002330 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:07.316936322 10265 0x7f210c002330 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:07.316967923 10265 0x7f210c002330 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:07.316981685 10265 0x7f210c002330 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:07.317049466 10265 0x7f210c002330 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6719","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"v=0\r\no=- 3803193307 3803193307 IN IP4 192.168.105.203\r\ns=Kurento Media Server\r\nc=IN IP4 192.168.105.203\r\nt=0 0\r\nm=audio 64878 RTP/AVPF 8 0 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122\r\na=setup:actpass\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=rtpmap:96 speex/8000\r\na=rtpmap:97 (null)\r\na=rtpmap:98 H265\r\na=rtpmap:99 theora\r\na=rtpmap:100 iLBC\r\na=rtpmap:101 PCMA-WB\r\na=rtpmap:102 PCMU-WB\r\na=rtpmap:103 G718\r\na=rtpmap:104 G719\r\na=rtpmap:105 G7221\r\na=rtpmap:106 G726-16\r\na=rtpmap:107 G726-24\r\na=rtpmap:108 G726-32\r\na=rtpmap:109 G726-40\r\na=rtpmap:110 G729D\r\na=rtpmap:111 G729E\r\na=rtpmap:112 G7291\r\na=rtpmap:113 GSM-EFR\r\na=rtpmap:114 GSM-HR-08\r\na=rtpmap:115 speex/16000\r\na=rtpmap:116 mpa-robust\r\na=rtpmap:117 MP4A-LATM\r\na=rtpmap:118 MP4V-ES\r\na=rtpmap:119 mpeg4-generic\r\na=rtpmap:120 VP8\r\na=rtpmap:121 AMR/16000\r\na=rtpmap:122 opus/48000/2\r\na=rtcp:64879\r\na=sendrecv\r\na=mid:audio0\r\na=ssrc:1751747321 cname:user1494734174@host-cbb2c6b\r\n"}}
0:06:07.317065037 10265 0x7f210c002330 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:08.442352679 10265 0x7f2110044350 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:08.442395942 10265 0x7f2110044350 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"invoke","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline/c85d1bf4-92a8-44c9-9dcd-82b4039d790b_kurento.RtpEndpoint","operation":"processAnswer","operationParams":{"answer":"v=0
o=sipp 53655765 1 IN IP4 192.168.137.42
s=-
c=IN IP4 192.168.137.42
t=0 0
m=audio 6000 RTP/AVP 121 101
a=rtpmap:121 AMR-WB/16000
a=rtpmap:101 telephone-event/8000
a=fmtp:121 mode-change-capability=2;octet_align=1
a=fmtp:101 0-15
a=ptime:20
"},"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d6721"}
0:06:08.442425965 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:08.442465788 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:08.442518112 10265 0x7f2110044350 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:08.442574826 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:602:invoke: LOGS to see flow
0:06:08.442589521 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:08.442826275 10265 0x7f2110044350 INFO kmssdpsession kmssdpsession.c:164:kms_sdp_session_process_answer:<kmsrtpsession7> Process SDP Answer:
v=0
o=sipp 53655765 1 IN IP4 192.168.137.42
s=-
c=IN IP4 192.168.137.42
t=0 0
m=audio 6000 RTP/AVP 121 101
a=rtpmap:121 AMR-WB/16000
a=rtpmap:101 telephone-event/8000
a=fmtp:121 mode-change-capability=2;octet_align=1
a=fmtp:101 0-15
a=ptime:20
0:06:08.443978704 10265 0x7f2110044350 INFO rtpendpoint kmsrtpendpoint.c:982:kms_rtp_endpoint_start_transport_send:<kmsrtpendpoint7> COMEDIA: Media 'audio' doesn't use COMEDIA
0:06:08.444003498 10265 0x7f2110044350 INFO kmsrtpconnection kmsrtpconnection.c:91:kms_rtp_connection_set_remote_info:<KmsRtpConnection@0x7f2130006ce0> Set remote host: 192.168.137.42, RTP: 6000, RTCP: 6001
0:06:08.445146087 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:08.445173424 10265 0x7f2110044350 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:08.445214169 10265 0x7f2110044350 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:08.445248286 10265 0x7f2110044350 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:08.445267001 10265 0x7f2110044350 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:08.445334396 10265 0x7f2110044350 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d6721","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5","value":"v=0\r\no=- 3803193307 3803193307 IN IP4 192.168.105.203\r\ns=Kurento Media Server\r\nc=IN IP4 192.168.105.203\r\nt=0 0\r\nm=audio 64878 RTP/AVPF 8 0 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122\r\na=setup:actpass\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=rtpmap:96 speex/8000\r\na=rtpmap:97 (null)\r\na=rtpmap:98 H265\r\na=rtpmap:99 theora\r\na=rtpmap:100 iLBC\r\na=rtpmap:101 PCMA-WB\r\na=rtpmap:102 PCMU-WB\r\na=rtpmap:103 G718\r\na=rtpmap:104 G719\r\na=rtpmap:105 G7221\r\na=rtpmap:106 G726-16\r\na=rtpmap:107 G726-24\r\na=rtpmap:108 G726-32\r\na=rtpmap:109 G726-40\r\na=rtpmap:110 G729D\r\na=rtpmap:111 G729E\r\na=rtpmap:112 G7291\r\na=rtpmap:113 GSM-EFR\r\na=rtpmap:114 GSM-HR-08\r\na=rtpmap:115 speex/16000\r\na=rtpmap:116 mpa-robust\r\na=rtpmap:117 MP4A-LATM\r\na=rtpmap:118 MP4V-ES\r\na=rtpmap:119 mpeg4-generic\r\na=rtpmap:120 VP8\r\na=rtpmap:121 AMR/16000\r\na=rtpmap:122 opus/48000/2\r\na=rtcp:64879\r\na=sendrecv\r\na=mid:audio0\r\na=ssrc:1751747321 cname:user1494734174@host-cbb2c6b\r\n"}}
0:06:08.445367136 10265 0x7f2110044350 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:06:08.446254547 10265 0x7f212400bc00 INFO basertpendpoint kmsbasertpendpoint.c:1914:kms_base_rtp_endpoint_jitterbuffer_set_latency:<rtpjitterbuffer6> Add probe: Set jitterbuffer latency
0:06:08.446289135 10265 0x7f212400bc00 INFO basertpendpoint kmsbasertpendpoint.c:1963:kms_base_rtp_endpoint_jitterbuffer_monitor_rtp_out:<rtpjitterbuffer6> Add probe: Adjust jitterbuffer PTS out
0:06:08.446571620 10265 0x7f212400bc00 INFO basertpendpoint kmsbasertpendpoint.c:2008:kms_base_rtp_endpoint_jitterbuffer_monitor_rtcp_in:<rtpjitterbuffer6> Add probe: Get jitterbuffer RTCP SR timing
0:06:08.446748148 10265 0x7f20ec002230 INFO basertpendpoint kmsbasertpendpoint.c:1899:kms_base_rtp_endpoint_jitterbuffer_set_latency_probe:<rtpjitterbuffer6> Setting latency to: 100
0:06:08.446825761 10265 0x7f20ec002230 INFO basertpendpoint kmsbasertpendpoint.c:1902:kms_base_rtp_endpoint_jitterbuffer_set_latency_probe:<rtpjitterbuffer6> Jitterbuffer latency set; remove probe
0:06:08.447185817 10265 0x7f20ec002230 INFO kmsutils kmsutils.c:494:kms_utils_pad_monitor_gaps:<'':sink> Add probe: Detect stream gaps
0:06:08.447956629 10265 0x7f20ec002230 INFO kmsutils kmsutils.c:1481:kms_utils_depayloader_monitor_pts_out:<rtpamrdepay0> Add probe: Adjust depayloader PTS out
0:06:08.486209651 10265 0x7f212400bc00 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc14> error: Internal data flow error.
0:06:08.486243884 10265 0x7f212400bc00 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc14> error: streaming task paused, reason not-negotiated (-4)
0:06:08.486405798 10265 0x560fe01e0200 ERROR KurentoMediaPipelineImpl MediaPipelineImpl.cpp:69:processBusMessage: Error code 1: 'Internal data flow error.', element: udpsrc14, parent: pipeline3
0:06:08.486439687 10265 0x560fe01e0200 ERROR KurentoMediaPipelineImpl MediaPipelineImpl.cpp:72:processBusMessage: Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline3/KmsRtpEndpoint:kmsrtpendpoint7/KmsRtpSession:kmsrtpsession7/GstUDPSrc:udpsrc14:
streaming task paused, reason not-negotiated (-4)
0:06:08.486511430 10265 0x560fe01e0200 ERROR KurentoMediaElementImpl MediaElementImpl.cpp:266:processBusMessage: Error code 1: 'Internal data flow error.', element: udpsrc14, parent: kmsrtpendpoint7
0:06:08.486530223 10265 0x560fe01e0200 ERROR KurentoMediaElementImpl MediaElementImpl.cpp:269:processBusMessage: Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline3/KmsRtpEndpoint:kmsrtpendpoint7/KmsRtpSession:kmsrtpsession7/GstUDPSrc:udpsrc14:
streaming task paused, reason not-negotiated (-4)
0:06:08.514717084 10265 0x7f211c006a80 INFO basertpendpoint kmsbasertpendpoint.c:1914:kms_base_rtp_endpoint_jitterbuffer_set_latency:<rtpjitterbuffer7> Add probe: Set jitterbuffer latency
0:06:08.514752782 10265 0x7f211c006a80 INFO basertpendpoint kmsbasertpendpoint.c:1963:kms_base_rtp_endpoint_jitterbuffer_monitor_rtp_out:<rtpjitterbuffer7> Add probe: Adjust jitterbuffer PTS out
0:06:08.515108591 10265 0x7f211c006a80 INFO basertpendpoint kmsbasertpendpoint.c:2008:kms_base_rtp_endpoint_jitterbuffer_monitor_rtcp_in:<rtpjitterbuffer7> Add probe: Get jitterbuffer RTCP SR timing
0:06:08.515272933 10265 0x7f2108002230 INFO basertpendpoint kmsbasertpendpoint.c:1899:kms_base_rtp_endpoint_jitterbuffer_set_latency_probe:<rtpjitterbuffer7> Setting latency to: 100
0:06:08.515348946 10265 0x7f2108002230 INFO basertpendpoint kmsbasertpendpoint.c:1902:kms_base_rtp_endpoint_jitterbuffer_set_latency_probe:<rtpjitterbuffer7> Jitterbuffer latency set; remove probe
0:06:08.515674115 10265 0x7f2108002230 INFO kmsutils kmsutils.c:494:kms_utils_pad_monitor_gaps:<'':sink> Add probe: Detect stream gaps
0:06:08.516399494 10265 0x7f2108002230 INFO kmsutils kmsutils.c:1481:kms_utils_depayloader_monitor_pts_out:<rtppcmadepay3> Add probe: Adjust depayloader PTS out
0:06:08.517420553 10265 0x7f2108002230 INFO agnosticbin kmsagnosticbin.c:899:input_bin_src_caps_probe:<kmsagnosticbin2-6> Set input caps: audio/x-alaw, channels=(int)1, rate=(int)8000
0:06:08.517468804 10265 0x7f2108002230 INFO agnosticbin kmsagnosticbin.c:774:kms_agnostic_bin2_link_pad:<kmsagnosticbin2-6> Upstream provided caps: ANY
0:06:08.517503087 10265 0x7f2108002230 INFO agnosticbin kmsagnosticbin.c:783:kms_agnostic_bin2_link_pad:<kmsagnosticbin2-6> Downstream wanted caps: audio/AMR, channels=(int)1, rate=(int)8000
0:06:08.549893631 10265 0x7f2108002230 INFO agnosticbin kmsagnosticbin.c:729:kms_agnostic_bin2_find_or_create_bin_for_caps:<kmsagnosticbin2-6> TRANSCODING ACTIVE for audio
0:06:27.316958881 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317011039 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317026885 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317046828 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317062756 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317077290 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317099718 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:27.317130177 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:28.446330516 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
0:06:58.455479325 10265 0x7f2130008e30 INFO KurentoWebSocketTransport WebSocketTransport.cpp:521:processMessage: LOGS to see flow
0:06:58.455520601 10265 0x7f2130008e30 INFO KurentoWebSocketTransport WebSocketTransport.cpp:528:processMessage: Message: {"jsonrpc":"2.0","method":"release","params":{"object":"2ec26ac1-7cc8-4294-849e-1dac6dd809b4_kurento.MediaPipeline","sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"},"id":"cbb39de2ebd33a11cce263d16f196d67333661404"}
0:06:58.455543991 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:271:process: inside ServerMethods::process
0:06:58.455574839 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:243:injectSessionId: inside getSessionId
0:06:58.455625279 10265 0x7f2130008e30 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:58.455695856 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:208:getOrCreateSessionId: inside getOrCreateSessionId
0:06:58.455712721 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:432:release: inside release
0:06:58.456082518 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:340:postProcess: inside postProcess
0:06:58.456116798 10265 0x7f2130008e30 INFO KurentoRequestCache RequestCache.cpp:77:getCachedResponse: LOGS to see flow
0:06:58.456183258 10265 0x7f2130008e30 INFO KurentoCacheEntry CacheEntry.cpp:57:CacheEntry: WebSocket Ping/Pong
0:06:58.456247463 10265 0x7f2130008e30 INFO KurentoRequestCache RequestCache.cpp:49:addResponse: LOGS to see flow
0:06:58.456290697 10265 0x7f2130008e30 INFO KurentoServerMethods ServerMethods.cpp:223:getSessionId: inside getSessionId
0:06:58.456404514 10265 0x7f2130008e30 INFO KurentoWebSocketTransport WebSocketTransport.cpp:530:processMessage: Response: {"id":"cbb39de2ebd33a11cce263d16f196d67333661404","jsonrpc":"2.0","result":{"sessionId":"efdde39e-4f79-4f0e-b10a-8145767546e5"}}
0:06:58.456429155 10265 0x7f2130008e30 INFO KurentoWebSocketTransport WebSocketTransport.cpp:443:storeConnection: LOGS to see flow
0:07:00.172849492 10265 0x560fe01e1700 INFO KurentoWebSocketTransport WebSocketTransport.cpp:352:keepAliveSessions: Keep alive efdde39e-4f79-4f0e-b10a-8145767546e5
0:07:00.172895331 10265 0x560fe01e1700 INFO KurentoServerMethods ServerMethods.cpp:303:keepAliveSession: inside keepAliveSession
0:07:18.475418101 10265 0x560fe01e0200 INFO KurentoCacheEntry CacheEntry.cpp:78:~CacheEntry: WebSocket Ping/Pong
Internal Data flow error was observed. Please help
j1elo
commented
4 years ago The error
streaming task paused, reason not-negotiated
comes from GStreamer and it is very common judging by the amount of results that come from a search. As far as I know, it happens because the GStreamer Pipeline (inside the Kurento Pipeline) is NOT prepared to receive the data format that it receives. For example: it would happen if the pipeline configures itself for H.264, but then the UDP socket receives VP8 video.
To solve the GStreamer-CRITICAL errors:
Please first of all, make sure to install debug symbols.
Then, follow the troubleshooting instructions about Run and debug with GDB to run KMS with a debugger and obtain a backtrace (with the backtrace command).
The GDB backtrace will let you know exactly where this problem is happening:
Invalid character ' ' at offset 4 in structure name: H264 SVCwhich is clearly caused by an invalid codec name (as it seems, codec names cannot contain spaces).
opg7371
commented
4 years ago I have installed debug symbols in the kurento media server. Please find the bt mentioned below.
0:00:16.767080902 14864 0x155508003720 INFO basertpendpoint kmsbasertpendpoint.c:1902:kms_base_rtp_endpoint_jitterbuffer_set_latency_probe:<rtpjitterbuffer1> Jitterbuffer latency set; remove probe
0:00:16.767551740 14864 0x155508003720 INFO kmsutils kmsutils.c:494:kms_utils_pad_monitor_gaps:<'':sink> Add probe: Detect stream gaps
0:00:16.768743585 14864 0x155508003720 INFO kmsutils kmsutils.c:1481:kms_utils_depayloader_monitor_pts_out:<rtppcmadepay0> Add probe: Adjust depayloader PTS out
0:00:16.776151675 14864 0x15552000d9e0 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc2> error: Internal data flow error.
0:00:16.776205860 14864 0x15552000d9e0 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc2> error: streaming task paused, reason not-negotiated (-4)
0:00:16.776465779 14864 0x5555560a4350 ERROR KurentoMediaPipelineImpl MediaPipelineImpl.cpp:69:processBusMessage: Error code 1: 'Internal data flow error.', element: udpsrc2, parent: pipeline0
Thread 1 "kurento-media-s" hit Breakpoint 1, kurento::MediaPipelineImpl::processBusMessage (this=0x155514002020,
msg=0x1555140090a0) at /root/PIYUSH/kms-omni-build/kms-core/src/server/implementation/objects/MediaPipelineImpl.cpp:71
71 GST_CAT_LEVEL_LOG (GST_CAT_DEFAULT, log_level, NULL, "Debugging info: %s",
Breakpoint observed was applied at : MediaPipelineImpl.cpp:69
bt at this breakpoint is
#0 0x0000155553fe28ff in kurento::MediaPipelineImpl::processBusMessage(_GstMessage*) (this=0x155514002020, msg=0x1555140090a0) at /root/PIYUSH/kms-omni-build/kms-core/src/server/implementation/objects/MediaPipelineImpl.cpp:71
#1 0x0000155553fe5486 in std::__invoke_impl<void, void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*>(std::__invoke_memfun_deref, void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*&&) (__f=
@0x15551400da90: (void (kurento::MediaPipelineImpl::*)(kurento::MediaPipelineImpl * const, _GstMessage *)) 0x155553fe272a <kurento::MediaPipelineImpl::processBusMessage(_GstMessage*)>, __t=@0x15551400daa0: 0x155514002020)
at /usr/include/c++/7/bits/invoke.h:73
#2 0x0000155553fe5372 in std::__invoke<void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*>(void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*&&) (__fn=
@0x15551400da90: (void (kurento::MediaPipelineImpl::*)(kurento::MediaPipelineImpl * const, _GstMessage *)) 0x155553fe272a <kurento::MediaPipelineImpl::processBusMessage(_GstMessage*)>) at /usr/include/c++/7/bits/invoke.h:95
#3 0x0000155553fe5246 in std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)>::__call<void, _GstBus*&&, _GstMessage*&&, 0ul, 1ul>(std::tuple<_GstBus*&&, _GstMessage*&&>&&, std::_Index_tuple<0ul, 1ul>) (this=0x15551400da90, __args=...) at /usr/include/c++/7/functional:467
#4 0x0000155553fe4fc3 in std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)>::operator()<_GstBus*, _GstMessage*, void>(_GstBus*&&, _GstMessage*&&) (this=0x15551400da90)
at /usr/include/c++/7/functional:551
#5 0x0000155553fe48c8 in std::_Function_handler<void (_GstBus*, _GstMessage*), std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)> >::_M_invoke(std::_Any_data const&, _GstBus*&&, _GstMessage*&&) (__functor=..., __args#0=@0x7fffffffd490: 0x555555fd9f10 [GstBus], __args#1=@0x7fffffffd488: 0x1555140090a0)
at /usr/include/c++/7/bits/std_function.h:316
#6 0x0000155553fe5147 in std::function<void (_GstBus*, _GstMessage*)>::operator()(_GstBus*, _GstMessage*) const (this=0x1555140081c0, __args#0=0x555555fd9f10 [GstBus], __args#1=0x1555140090a0) at /usr/include/c++/7/bits/std_function.h:706
#7 0x0000155553fe3f61 in kurento::signal_handler_adaptor<void, _GstBus*, _GstMessage*>(gpointer) (data=0x1555140081c0)
at /root/PIYUSH/kms-omni-build/kms-core/src/server/implementation/SignalHandler.hpp:54
#8 0x00001555508c2dae in ffi_call_unix64 () at /usr/lib/x86_64-linux-gnu/libffi.so.6
#9 0x00001555508c271f in ffi_call () at /usr/lib/x86_64-linux-gnu/libffi.so.6
---Type <return> to continue, or q <return> to quit---
#14 0x000015555283512f in <emit signal message:error on instance 0x555555fd9f10 [GstBus]> (instance=instance@entry=0x555555fd9f10, signal_id=<optimized out>, detail=<optimized out>) at ../../../../gobject/gsignal.c:3447
#10 0x0000155552819901 in g_cclosure_marshal_generic (closure=0x15551400dad0, return_gvalue=0x0, n_param_values=<optimized out>, param_values=<optimized out>, invocation_hint=<optimized out>, marshal_data=<optimized out>)
at ../../../../gobject/gclosure.c:1490
#11 0x000015555281910d in g_closure_invoke (closure=0x15551400dad0, return_value=0x0, n_param_values=2, param_values=0x7fffffffd950, invocation_hint=0x7fffffffd8d0) at ../../../../gobject/gclosure.c:804
#12 0x000015555282c05e in signal_emit_unlocked_R (node=node@entry=0x555555e9c980, detail=detail@entry=279, instance=instance@entry=0x555555fd9f10, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7fffffffd950) at ../../../../gobject/gsignal.c:3635
#13 0x0000155552834715 in g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>, var_args=var_args@entry=0x7fffffffdb20) at ../../../../gobject/gsignal.c:3391
#15 0x0000155553151f24 in gst_bus_async_signal_func (bus=0x555555fd9f10 [GstBus], message=0x1555140090a0, data=<optimized out>) at gstbus.c:1197
#16 0x0000155553152ccd in gst_bus_source_dispatch (source=source@entry=0x15551400d9d0, callback=0x155553151ed0 <gst_bus_async_signal_func>, user_data=0x0) at gstbus.c:791
#17 0x000015555253e285 in g_main_dispatch (context=0x555555e690a0) at ../../../../glib/gmain.c:3176
#18 0x000015555253e285 in g_main_context_dispatch (context=context@entry=0x555555e690a0) at ../../../../glib/gmain.c:3829
#19 0x000015555253e650 in g_main_context_iterate (context=0x555555e690a0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../../../../glib/gmain.c:3902
#20 0x000015555253e962 in g_main_loop_run (loop=0x555555e691e0) at ../../../../glib/gmain.c:4098
#21 0x00005555559e1910 in main(int, char**) (argc=4, argv=0x7fffffffe298)
at /root/PIYUSH/kms-omni-build/kurento-media-server/server/main.cpp:261
We didn't found any useful thread. So, pasting them also
Id Target Id Frame
* 1 Thread 0x155555514280 (LWP 14864) "kurento-media-s" 0x0000155553fe5486 in std::__invoke_impl<void, void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*> (__f=
@0x15551400da90: (void (kurento::MediaPipelineImpl::*)(kurento::MediaPipelineImpl * const, _GstMessage *)) 0x155553fe272a <kurento::MediaPipelineImpl::processBusMessage(_GstMessage*)>, __t=@0x15551400daa0: 0x155514002020)
at /usr/include/c++/7/bits/invoke.h:73
2 Thread 0x155547ffe700 (LWP 14865) "KmsPool#0" 0x0000155551c81d67 in epoll_wait (epfd=5, events=0x155547ffcb10,
maxevents=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
3 Thread 0x155536d6f700 (LWP 14866) "KmsPool#1" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51984) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
4 Thread 0x155536b6e700 (LWP 14867) "KmsPool#2" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51984) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
5 Thread 0x15553696d700 (LWP 14868) "KmsPool#3" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
6 Thread 0x15553676c700 (LWP 14869) "KmsPool#4" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
7 Thread 0x15553656b700 (LWP 14870) "KmsPool#5" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
8 Thread 0x15553636a700 (LWP 14871) "KmsPool#6" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
9 Thread 0x155536169700 (LWP 14872) "KmsPool#7" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
10 Thread 0x155535f68700 (LWP 14873) "KmsPool#8" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
11 Thread 0x155535d67700 (LWP 14874) "KmsPool#9" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
12 Thread 0x155535b66700 (LWP 14875) "KmsPool#10" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
---Type <return> to continue, or q <return> to quit---
13 Thread 0x155535965700 (LWP 14876) "KmsPool#11" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
14 Thread 0x155535764700 (LWP 14877) "KmsPool#12" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
15 Thread 0x155535563700 (LWP 14878) "KmsPool#13" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
16 Thread 0x155535362700 (LWP 14879) "KmsPool#14" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
17 Thread 0x155535161700 (LWP 14880) "KmsPool#15" 0x00001555545789f3 in futex_wait_cancelable (private=<optimized out>,
expected=0, futex_word=0x555555e51980) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
18 Thread 0x155534f60700 (LWP 14881) "kurento-media-s" 0x0000155554578f85 in futex_abstimed_wait_cancelable (
private=<optimized out>, abstime=0x155534f5f120, expected=0, futex_word=0x555555fb2cf8)
at ../sysdeps/unix/sysv/linux/futex-internal.h:205
19 Thread 0x155534d5f700 (LWP 14882) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
20 Thread 0x155534b5e700 (LWP 14883) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
21 Thread 0x15553495d700 (LWP 14884) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
22 Thread 0x15553475c700 (LWP 14885) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
23 Thread 0x15553455b700 (LWP 14886) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
24 Thread 0x15553435a700 (LWP 14887) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf84) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
25 Thread 0x155534159700 (LWP 14888) "kurento-media-s" 0x0000155551c81d67 in epoll_wait (epfd=8, events=0x1555341579e0,
maxevents=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
26 Thread 0x155533f58700 (LWP 14889) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
---Type <return> to continue, or q <return> to quit---
private=<optimized out>, expected=0, futex_word=0x555555e9bf84) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
27 Thread 0x155533d57700 (LWP 14890) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf80) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
28 Thread 0x155533b56700 (LWP 14891) "kurento-media-s" 0x00001555545789f3 in futex_wait_cancelable (
private=<optimized out>, expected=0, futex_word=0x555555e9bf84) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
29 Thread 0x155533955700 (LWP 14892) "kurento-media-s" 0x0000155554578f85 in futex_abstimed_wait_cancelable (
private=<optimized out>, abstime=0x155533953fb0, expected=0, futex_word=0x555556643560)
at ../sysdeps/unix/sysv/linux/futex-internal.h:205
30 Thread 0x155533546700 (LWP 14893) "KmsLoop" 0x0000155551c74cf9 in __GI___poll (fds=0x15550c0039e0, nfds=1, timeout=801)
at ../sysdeps/unix/sysv/linux/poll.c:29
31 Thread 0x155532cc9700 (LWP 14894) "kurento-media-s" 0x0000155551c74df6 in __GI_ppoll (fds=0x15552802f910, nfds=1,
timeout=<optimized out>, timeout@entry=0x155532cc80b0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:39
32 Thread 0x155532ac8700 (LWP 14895) "udpsrc0:src" 0x0000155551c74cf9 in __GI___poll (fds=fds@entry=0x155532ac7020,
nfds=nfds@entry=2, timeout=timeout@entry=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
33 Thread 0x1555328c7700 (LWP 14896) "udpsrc1:src" 0x0000155551c74cf9 in __GI___poll (fds=fds@entry=0x1555328c6020,
nfds=nfds@entry=2, timeout=timeout@entry=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
34 Thread 0x155532447700 (LWP 14897) "kurento-media-s" 0x0000155551c74df6 in __GI_ppoll (fds=0x15552802f910, nfds=1,
timeout=<optimized out>, timeout@entry=0x1555324460b0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:39
35 Thread 0x155532246700 (LWP 14898) "kurento-media-s" 0x0000155551c74df6 in __GI_ppoll (fds=0x15552802f910, nfds=1,
timeout=<optimized out>, timeout@entry=0x1555322450b0, sigmask=sigmask@entry=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:39
36 Thread 0x155532045700 (LWP 14899) "udpsrc2:src" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
37 Thread 0x155531e44700 (LWP 14900) "udpsrc3:src" 0x0000155551c74cf9 in __GI___poll (fds=fds@entry=0x155531e43020,
nfds=nfds@entry=2, timeout=timeout@entry=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
38 Thread 0x155531c43700 (LWP 14901) "timer" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
39 Thread 0x155531a42700 (LWP 14902) "rtpjitterbuffer" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
40 Thread 0x1555313c7700 (LWP 14903) "timer" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
41 Thread 0x1555311c6700 (LWP 14904) "rtpjitterbuffer" 0x000015555534146d in dl_open_worker (a=a@entry=0x1555311c2df0)
at dl-open.c:515
bt full was
(gdb) bt full
#0 0x0000155553fe28ff in kurento::MediaPipelineImpl::processBusMessage(_GstMessage*) (this=0x155514002020, msg=0x1555140090a0) at /root/PIYUSH/kms-omni-build/kms-core/src/server/implementation/objects/MediaPipelineImpl.cpp:71
log_level = GST_LEVEL_ERROR
err = 0x555555e9c010
dbg_info = 0x55555610e700 "gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/KmsRtpEndpoint:kmsrtpendpoint1/KmsRtpSession:kmsrtpsession1/GstUDPSrc:udpsrc2:\nstreaming task paused, reason not-negotiated (-4)"
parent = 0x155514006070 [GstPipeline]
err_code = 1
err_msg = 0x155514001ca0 "Internal data flow error."
__FUNCTION__ = "processBusMessage"
errorMessage = "\270\000\000\000\000\000\000\000\000\247\216\200#\347L\271", '\000' <repeats 16 times>, "\002\000\000\000\000\000\000\000P\331\377\377\377\177\000\000\320\330\377\377\377\177\000\000\000\227\201RU\025\000\000\320\332\000\024U\025\000\000\r\221\201RU\025", '\000' <repeats 18 times>, "\002\000\002@UU\000\000\324\311RRU\025\000\000\320\327\377\377\377\177\000\000\020\237\375UUU\000\000\000\000\000\000\000\000\000\000\200\206\nVUU\000\000\200\311\351UUU\000\000\000\333\000\024U\025\000\000\200\206\nVUU\000\000\320\330\377\377\377\177\000\000\200\206\nVUU\000\000^\300\202RU\025\000\000 \330\377\377\377\177\000\000"...
#1 0x0000155553fe5486 in std::__invoke_impl<void, void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*>(std::__invoke_memfun_deref, void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*&&) (__f=
@0x15551400da90: (void (kurento::MediaPipelineImpl::*)(kurento::MediaPipelineImpl * const, _GstMessage *)) 0x155553fe272a <kurento::MediaPipelineImpl::processBusMessage(_GstMessage*)>, __t=@0x15551400daa0: 0x155514002020)
at /usr/include/c++/7/bits/invoke.h:73
#2 0x0000155553fe5372 in std::__invoke<void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*>(void (kurento::MediaPipelineImpl::*&)(_GstMessage*), kurento::MediaPipelineImpl*&, _GstMessage*&&) (__fn=
@0x15551400da90: (void (kurento::MediaPipelineImpl::*)(kurento::MediaPipelineImpl * const, _GstMessage *)) 0x155553fe272a <kurento::MediaPipelineImpl::processBusMessage(_GstMessage*)>) at /usr/include/c++/7/bits/invoke.h:95
#3 0x0000155553fe5246 in std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)>::__call<void, _GstBus*&&, _GstMessage*&&, 0ul, 1ul>(std::tuple<_GstBus*&&, _GstMessage*&&>&&, std::_Index_tuple<0u---Type <return> to continue, or q <return> to quit---
l, 1ul>) (this=0x15551400da90, __args=...) at /usr/include/c++/7/functional:467
#4 0x0000155553fe4fc3 in std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)>::operator()<_GstBus*, _GstMessage*, void>(_GstBus*&&, _GstMessage*&&) (this=0x15551400da90)
at /usr/include/c++/7/functional:551
#5 0x0000155553fe48c8 in std::_Function_handler<void (_GstBus*, _GstMessage*), std::_Bind<void (kurento::MediaPipelineImpl::*(kurento::MediaPipelineImpl*, std::_Placeholder<2>))(_GstMessage*)> >::_M_invoke(std::_Any_data const&, _GstBus*&&, _GstMessage*&&) (__functor=..., __args#0=@0x7fffffffd490: 0x555555fd9f10 [GstBus], __args#1=@0x7fffffffd488: 0x1555140090a0)
at /usr/include/c++/7/bits/std_function.h:316
#6 0x0000155553fe5147 in std::function<void (_GstBus*, _GstMessage*)>::operator()(_GstBus*, _GstMessage*) const (this=0x1555140081c0, __args#0=0x555555fd9f10 [GstBus], __args#1=0x1555140090a0) at /usr/include/c++/7/bits/std_function.h:706
#7 0x0000155553fe3f61 in kurento::signal_handler_adaptor<void, _GstBus*, _GstMessage*>(gpointer) (data=0x1555140081c0)
at /root/PIYUSH/kms-omni-build/kms-core/src/server/implementation/SignalHandler.hpp:54
adaptor = 0x1555140081c0
aux = warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<kurento::MediaObjectImpl*, kurento::MediaSet::ref(kurento::MediaObjectImpl*)::{lambda(kurento::MediaObjectImpl*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<kurento::MediaObjectImpl*, kurento::MediaSet::ref(kurento::MediaObjectImpl*)::{lambda(kurento::MediaObjectImpl*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
std::shared_ptr<kurento::MediaObjectImpl> (use count 4, weak count 3) = {get() = 0x155514002020}
#8 0x00001555508c2dae in ffi_call_unix64 () at /usr/lib/x86_64-linux-gnu/libffi.so.6
#9 0x00001555508c271f in ffi_call () at /usr/lib/x86_64-linux-gnu/libffi.so.6
#14 0x000015555283512f in <emit signal message:error on instance 0x555555fd9f10 [GstBus]> (instance=instance@entry=0x555555fd9f10, signal_id=<optimized out>, detail=<optimized out>) at ../../../../gobject/gsignal.c:3447
var_args = {{gp_offset = 32, fp_offset = 48, overflow_arg_area = 0x7fffffffdc00, reg_save_area = 0x7fffffffdb40}}
#10 0x0000155552819901 in g_cclosure_marshal_generic (closure=0x15551400dad0, return_gvalue=0x0, n_param_values=<optimized out>, param_values=<optimized out>, invocation_hint=<optimized out>, marshal_data=<optimized out>)
at ../../../../gobject/gclosure.c:1490
rtype = <optimized out>
rvalue = 0x7fffffffd690
n_args = 3
atypes = <optimized out>
i = <optimized out>
cif =
---Type <return> to continue, or q <return> to quit---
{abi = FFI_UNIX64, nargs = 3, arg_types = 0x7fffffffd660, rtype = 0x1555508c31a0 <ffi_type_void>, bytes = 0, flags = 0}
cc = 0x15551400dad0
enum_tmpval = <optimized out>
tmpval_used = 0
#11 0x000015555281910d in g_closure_invoke (closure=0x15551400dad0, return_value=0x0, n_param_values=2, param_values=0x7fffffffd950, invocation_hint=0x7fffffffd8d0) at ../../../../gobject/gclosure.c:804
marshal = 0x155552819700 <g_cclosure_marshal_generic>
marshal_data = 0x0
in_marshal = 0
real_closure = 0x15551400dab0
__func__ = "g_closure_invoke"
#12 0x000015555282c05e in signal_emit_unlocked_R (node=node@entry=0x555555e9c980, detail=detail@entry=279, instance=instance@entry=0x555555fd9f10, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7fffffffd950) at ../../../../gobject/gsignal.c:3635
tmp = <optimized out>
handler = 0x5555560a8680
accumulator = 0x0
emission =
{next = 0x0, instance = 0x555555fd9f10, ihint = {signal_id = 14, detail = 279, run_type = G_SIGNAL_RUN_FIRST}, state = EMISSION_RUN, chain_type = 4}
handler_list = 0x5555560a8680
return_accu = 0x0
accu =
{g_type = 0, data = {{v_int = 0, v_uint = 0, v_long = 0, v_ulong = 0, v_int64 = 0, v_uint64 = 0, v_float = 0, v_double = 0, v_pointer = 0x0}, {v_int = 0, v_uint = 0, v_long = 0, v_ulong = 0, v_int64 = 0, v_uint64 = 0, v_float = 0, v_double = 0, v_pointer = 0x0}}}
signal_id = 14
---Type <return> to continue, or q <return> to quit---
max_sequential_handler_number = 165
return_value_altered = 0
#13 0x0000155552834715 in g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>, var_args=var_args@entry=0x7fffffffdb20) at ../../../../gobject/gsignal.c:3391
instance_and_params = 0x7fffffffd950
signal_return_type = <optimized out>
param_values = 0x7fffffffd968
i = <optimized out>
n_params = <optimized out>
__func__ = "g_signal_emit_valist"
#15 0x0000155553151f24 in gst_bus_async_signal_func (bus=0x555555fd9f10 [GstBus], message=0x1555140090a0, data=<optimized out>) at gstbus.c:1197
detail = <optimized out>
#16 0x0000155553152ccd in gst_bus_source_dispatch (source=source@entry=0x15551400d9d0, callback=0x155553151ed0 <gst_bus_async_signal_func>, user_data=0x0) at gstbus.c:791
handler = 0x155553151ed0 <gst_bus_async_signal_func>
bsource = 0x15551400d9d0
message = 0x1555140090a0
keep = <optimized out>
bus = 0x555555fd9f10 [GstBus]
__func__ = "gst_bus_source_dispatch"
__PRETTY_FUNCTION__ = "gst_bus_source_dispatch"
#17 0x000015555253e285 in g_main_dispatch (context=0x555555e690a0) at ../../../../glib/gmain.c:3176
dispatch = 0x155553152c00 <gst_bus_source_dispatch>
prev_source = 0x0
was_in_call = 0
user_data = 0x0
callback = 0x155553151ed0 <gst_bus_async_signal_func>
---Type <return> to continue, or q <return> to quit---
cb_funcs = 0x155552807280 <g_source_callback_funcs>
cb_data = 0x15551400da40
need_destroy = <optimized out>
source = 0x15551400d9d0
current = 0x555555e9c050
i = 0
#18 0x000015555253e285 in g_main_context_dispatch (context=context@entry=0x555555e690a0) at ../../../../glib/gmain.c:3829
#19 0x000015555253e650 in g_main_context_iterate (context=0x555555e690a0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../../../../glib/gmain.c:3902
max_priority = 2147483647
timeout = 18666
some_ready = 1
nfds = <optimized out>
allocated_nfds = 2
fds = 0x555556105ca0
#20 0x000015555253e962 in g_main_loop_run (loop=0x555555e691e0) at ../../../../glib/gmain.c:4098
__func__ = "g_main_loop_run"
#21 0x00005555559e1910 in main(int, char**) (argc=4, argv=0x7fffffffe298)
at /root/PIYUSH/kms-omni-build/kurento-media-server/server/main.cpp:261
signalAction =
{__sigaction_handler = {sa_handler = 0x5555559e0962 <signal_handler(int)>, sa_sigaction = 0x5555559e0962 <signal_handler(int)>}, sa_mask = {__val = {0 <repeats 16 times>}}, sa_flags = 0, sa_restorer = 0x0}
transport = std::shared_ptr<kurento::Transport> (use count 1, weak count 1) = {get() = 0x555556642e00}
config = {m_data = "", m_children = 0x555555e89280}
confFile = "/root/PIYUSH/kms-omni-build/build-Debug/config/kurento.conf.json"
modulesPath = "/root/PIYUSH/kms-omni-build/build-Debug:/usr/lib/x86_64-linux-gnu/kurento/modules"
logsPath = ""
modulesConfigPath = "/root/PIYUSH/kms-omni-build/build-Debug/config"
---Type <return> to continue, or q <return> to quit---
fileSize = 100
fileNumber = 10
dh =
{static kNeededMemory = 12288, static generate_core_dump_ = true, static cleanup_ = true, static quick_exit_ = false, static frames_count_ = 16, static cut_common_path_root_ = true, static cut_relative_paths_ = true, static append_pid_ = false, static color_output_ = true, static thread_safe_ = true, static memory_ = 0x555555e892b0 ""}
__FUNCTION__ = "main"
killResourceLimit =
{<boost::optional_detail::optional_base<float>> = {<boost::optional_detail::optional_tag> = {<No data fields>}, m_initialized = false, m_storage = {dummy_ = {data = "UU\000", aligner_ = {<No data fields>}}}}, <No data fields>}
(gdb)
I still feel that debug symbols are not properly installed. Is that the case?
info frame 0
(gdb) info frame
Stack level 0, frame at 0x7fffffffd2e0:
rip = 0x155553fe28ff in kurento::MediaPipelineImpl::processBusMessage
(kms-omni-build/kms-core/src/server/implementation/objects/MediaPipelineImpl.cpp:71);
saved rip = 0x155553fe5486
called by frame at 0x7fffffffd320
source language c++.
Arglist at 0x7fffffffd2d0, args: this=0x155514002020, msg=0x1555140090a0
Locals at 0x7fffffffd2d0, Previous frame's sp is 0x7fffffffd2e0
Saved registers:
rbx at 0x7fffffffd2c0, rbp at 0x7fffffffd2d0, r12 at 0x7fffffffd2c8, rip at 0x7fffffffd2d8
Please let us know if there is any other logs we can provide it to you.
j1elo
commented
4 years ago The debug symbols seem to be correctly installed, as I can see Kurento source file names and line numbers.
But the purpose of running with GDB was to let it run and catch the GStreamer-CRITICAL errors, not the Internal data flow error.
The Internal data flow error first happens inside GStreamer, and then it is notified to the application (KMS) via an asynchronous message bus. Notice the method name, processBusMessage? This means that by applying a breakpoint at MediaPipelineImpl.cpp:69, you are just getting info from the bus message is obtained, not from where the error is happening.
I told you to use GDB because maybe the initial GStreamer-CRITICAL error is the actual case of other subsequent errors that happen later.
There is no specific tool that GStreamer provides to solve the Internal data flow error. You just need to study the environment where it happens and deduce a reason for it to happen.
Regarding this from the logs:
Add format support, media: audio, codec: H265
Add format support, media: audio, codec: theora
Add format support, media: audio, codec: iLBC
Add format support, media: audio, codec: PCMA-WB
Add format support, media: audio, codec: PCMU-WB
Add format support, media: audio, codec: G718
Add format support, media: audio, codec: G719
Add format support, media: audio, codec: G7221
Add format support, media: audio, codec: G726-16
Add format support, media: audio, codec: G726-24
Add format support, media: audio, codec: G726-32
Add format support, media: audio, codec: G726-40
Add format support, media: audio, codec: G729D
Add format support, media: audio, codec: G729E
Add format support, media: audio, codec: G7291
Add format support, media: audio, codec: GSM-EFR
Add format support, media: audio, codec: GSM-HR-08
Add format support, media: audio, codec: speex/16000
Add format support, media: audio, codec: mpa-robust
Add format support, media: audio, codec: MP4A-LATM
Add format support, media: audio, codec: MP4V-ES
Add format support, media: audio, codec: mpeg4-generic
Add format support, media: audio, codec: VP8
Add format support, media: audio, codec: AMR/16000
Add format support, media: audio, codec: opus/48000/2
[... and more]The official Kurento builds only enable these codecs for WebRtcEndpoint: OPUS, PCMU, VP8, H264, and for RtpEndpoint: OPUS, PCMU, AMR, VP8, H264. Because you're building from sources a customized version, where many more codecs are enabled, you now have to deal with making them work, otherwise you will get Internal data flow error messages. But that's something I cannot help with. I'd suggest requesting Commercial Support in case you cannot work it out and want the team to have a look into it.
Here are some ideas to help you going on:
Set your Debug Logging to the default sugested level, and then add levels for Transcoding and RtpEndpoint:
export GST_DEBUG="3,Kurento*:4,kms*:4,sdp*:4,webrtc*:4,*rtpendpoint:4,rtp*handler:4,rtpsynchronizer:4,agnosticbin:4"
export GST_DEBUG="$GST_DEBUG,Kurento*:5,agnosticbin*:5"
export GST_DEBUG="$GST_DEBUG,KurentoMediaElementImpl:5"
export GST_DEBUG="$GST_DEBUG,kmssdpsession:5,sdp*:5"
export GST_DEBUG="$GST_DEBUG,webrtcendpoint:5,kmswebrtcsession:5,kmsiceniceagent:5"You will then see the transcoding messages, which can help understanding why there is misunderstanding about codec capabilities in the GStreamer pipeline.
Review which codecs you are actually sending to Kurento. Are you using H.265? Are you using H.264 Scalable Video Coding (SVC)? Kurento doesn't support SVC. If your answer is "No", then think about why these are appearing in the debug and error logs.
Install the package gstreamer1.5-tools, and use the tool gst-inspect-1.5 to get information about the codecs that are supported by GStreamer (note: "gstreamer1.0" is from the system and not in use; KMS uses "gstreamer1.5" distributed from the Kurento repositories).
opg7371
commented
4 years ago I want to know whether kurento can handle non octect align (Bandwidth Efficient) AMR-WB audio packets during transcoding?
opg7371
commented
4 years ago These codecs were added not because we required them but inorder to solve a issue that i was experiencing.
A -----------------> Kurento ------------------->B Payload recieved from A was 8 PCMA/8000. Offer Generated by kurento for B included 8,0 and 96 (PCMA, PCMU and AMR-WB/16000) In response of B we recieved the payload 121.(AMR-WB/16000) (This was because the payload in the media file of AMR-WB played by sipp(compiled with media) was 121). AMR-WB is a codec with a dynamic payload and in this case, B is the one with AMR-WB codec.
So, proper negotitation among payload was not possible and hence, we inserted these much random codecs, just to increase the payload number of AMR-WB to 121 as generated by the kurento.
But, after doing such a thing, transcoding was not possible. the packets of AMR-WB were not transcoded into PCMA but the packets of PCMA were transcoded into AMR-WB. Hence, the transcoding failed.
The error occured for this is the same as mentioned above. Also, the print of TRANSCODING IS ACTIVE was displayed.
Hence, you can see all these codecs but in truth, these were of no use to us.
> Add format support, media: audio, codec: H265
> Add format support, media: audio, codec: theora
> Add format support, media: audio, codec: iLBC
> Add format support, media: audio, codec: PCMA-WB
> Add format support, media: audio, codec: PCMU-WB
> Add format support, media: audio, codec: G718
> Add format support, media: audio, codec: G719
> Add format support, media: audio, codec: G7221
> Add format support, media: audio, codec: G726-16
> Add format support, media: audio, codec: G726-24
> Add format support, media: audio, codec: G726-32
> Add format support, media: audio, codec: G726-40
> Add format support, media: audio, codec: G729D
> Add format support, media: audio, codec: G729E
> Add format support, media: audio, codecThe only thing that is useful for us is the transcoding in between AMR-WB/16000 and PCMA/8000 and load of nearly 50cps with a call hold time of 110 seconds.
We have a legacy code and we need to be 100 percent sure that we can intergrate kurento into our legacy code without any flaws. Only after that, we can consider the commercial support as mentioned by you in the previous comment.
opg7371
commented
4 years ago Consider another scenario where both phones are using codecs with dynamic payload (AMR-WB).
A -----------------> Kurento ----------------> B
In the SdpEndPoint.conf.json, we have done the following
{
"name": "AMR-WB/16000/1",
"properties": {
"maxcodedaudiobandwidth": "16000",
"maxaveragebitrate": "23850",
"stereo": "1",
"useinbandfec": "1",
"usedtx": "0"
}
}The payload value for AMR-WB when the GenerateOffer will be called will be 96.
Now, here is the scenario - -> A sends the payload of 105 (AMR-WB/16000) to kurento. -> Kurento generates the payload of 96, 8 and 0 (AMR-WB/16000 , PCMA/8000, PCMU/8000) -> In response of B, we will recieve 96 (AMR-WB/16000)
Now, one side payload value is 105 and on another it is 96 but the payload type for both is AMR-WB.
Will transcoding be active in the above scenario? If yes, then why would you do it?
In my view, there is no transcoding required in between the the endpoints where the codec at both end is same. ,ie AMR-WB.
In my view, I haven't done this testcase and will soon perform it but, transcoding should not be done and if transcoding occurs, I feel it as a bug. What do you think?
If it is a bug, should i raise it as a seperate issue?
opg7371
commented
4 years ago Now, to answer your question
The debug symbols seem to be correctly installed, as I can see Kurento source file names and line numbers.
cool
I told you to use GDB because maybe the initial GStreamer-CRITICAL error is the actual case of other subsequent errors that happen later.
There is no specific tool that GStreamer provides to solve the Internal data flow error. You just need to study the environment where it happens and deduce a reason for it to happen.
I did the same as mentioned in this Debugging Link.
I did the
export G_DEBUG=fatal_warnings
source /etc/default/kurento-media-server with all the logs that were enabled by you.
then i ran the gdb command and typed run.
Then after i performed my testcase, the problem was still there, but there was no exception occured in the GDB. When i did ctrl+C, and then
info threads and bt for each threads,
I was not able to find any useful kurento_file or gstreamer error that might effect the transcoding.
Overall,1000 threads were already created and when a new call was initiated, then 10 threads were created. As soon as the release came, those 10 threads were destroyed. I can send you the bt but i didn't find anything useful.
Is there anything else, we can do to identify the source apart from the things mentioned above. PS - I will continue to work on debugging with the GDB and will let you know if anything productive occurs.
opg7371
commented
4 years ago Here, this was our main problem and our doubt,
A ----------------> Kurento ---------------> B
Payload sent by A to Kurento -104 (AMR-WB/16000/1) Offer Generated by Kurento to B - 96, 8, 0 (AMR-WB/16000, PCMA/8000, PCMU/8000) In response we recieved - 8 (PCMA/8000) In response, A recieved from Kurento - 104 (AMR-WB/16000) Case - 1 : As per flow, the signalling is correct. A is sending data of the payload 104 with bitrate 23.85kbps and A is recieving with bitrate 23.05kbps
Question - Why is there a difference in this bitrate and how to correct this difference. We believe that due to this difference, we are not getting voice at all.
Case -2 : A is sending non-octect aligned packets (Band Width Efficient) with payload of 105 (AMR-WB/16000) and Kurento is sending the octect aligned packets back to A with the payload of 105 (AMR-WB/16000)
Question - Why is there a difference regarding octect align and non octect align and how to resolve this difference.
opg7371
commented
3 years ago Hey While attempting a transcoding in between AMR-WB and PCMU, everything is done correctly. Call is established and Transcoding is ACTIVE. For one model of phone,we are getting clear voice after transcoding but for the second phone, we are not getting proper conversion of AMR-WB to PCMU.
We are getting a ton of warning for the latter test case.
0:02:36.377867601 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 217, rtp_ts: 15848716, ext_ts: 15848716, last: 0:00:08.709533715, current: 0:00:05.849533715, fixed = last: 0:00:08.709533715^M
0:02:36.397808124 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 218, rtp_ts: 15848876, ext_ts: 15848876, last: 0:00:08.709533715, current: 0:00:05.869533715, fixed = last: 0:00:08.709533715^M
0:02:36.417816286 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 219, rtp_ts: 15849036, ext_ts: 15849036, last: 0:00:08.709533715, current: 0:00:05.889533715, fixed = last: 0:00:08.709533715^M
0:02:36.437845184 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 220, rtp_ts: 15849196, ext_ts: 15849196, last: 0:00:08.709533715, current: 0:00:05.909533715, fixed = last: 0:00:08.709533715^M
0:02:36.457887422 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 221, rtp_ts: 15849356, ext_ts: 15849356, last: 0:00:08.709533715, current: 0:00:05.929533715, fixed = last: 0:00:08.709533715^M
0:02:36.477826945 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 222, rtp_ts: 15849516, ext_ts: 15849516, last: 0:00:08.709533715, current: 0:00:05.949533715, fixed = last: 0:00:08.709533715^M
0:02:36.497873275 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 223, rtp_ts: 15849676, ext_ts: 15849676, last: 0:00:08.709533715, current: 0:00:05.969533715, fixed = last: 0:00:08.709533715^M
0:02:36.517747474 ^[[334m19935^[[00m 0x152d48004cf0 ^[[33;01mWARN ^[[00m ^[[00m rtpsynchronizer kmsrtpsynchronizer.c:539:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x152d68023d80>^[[00m [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3795252812, seq: 224, rtp_ts: 15849836, ext_ts: 15849836, last: 0:00:08.709533715, current: 0:00:05.989533715, fixed = last: 0:00:08.709533715^M
Encoders negotitated : mulanenc0 : voamrwbenc0
I know that the above warning comes due to duplicate packets in the network but for the latter case, we are recieving nearly thousands of these messages in 20-30 seconds whereas for the first one case, we are not recieving any warning.
Can anyone help?
Issue description
Kurento was running in release mode with zero debug. Kurento was installed using apt-get on Ubuntu 16.04. A total load of 15 cps with nearly 110 hold time is pumped through kurento media server and as soon as nearly 800 concurrent calls are reached, kurento crashes with this debug. There is no useful information in /var/crash crash file.
LOGS :
Context
We want to verify whether we can process (RTP of ) nearly 50cps calls through kurento or not with a total hold time of 110 seconds. This crash hinders us to achieve that particular task.
How to reproduce?
Just pump a load through sipp (compiled with media) of 15cps with a total hold time of 110 seconds.
The flow of message used by us is provided in this link by Anthony alba [https://groups.google.com/forum/#!searchin/kurento/piyush%7Csort:date/kurento/EmPhwkH5KAw/pJ_xeXhdBwAJ]
Expected & current behavior
Expected is it should not have crashed under any circumstances. The current behavior is it crashes.
(Optional) Possible solution
As of now, we do not have any idea of possible solutions.
INFO about Kurento Media Server
INFO about your Application Server
INFO about end-user clients
INFO about your environment
Commands output