RidgeRun / gstd-1.x

GStreamer Daemon is a GStreamer framework for controlling audio and video streaming using TCP messages. This version is based on GStreamer 1.x
https://developer.ridgerun.com/wiki/index.php?title=Gstd-1.0
GNU Lesser General Public License v2.1
172 stars 55 forks source link

gstd becoming unresponsive #302

Closed clogwog closed 2 years ago

clogwog commented 2 years ago

i'm running :

#gstd --version
GstD version 0.14.0
Copyright (C) 2015-2021 RidgeRun (https://www.ridgerun.com)

and i regularly check if the pipeline that i started is still going, before i check i first check if there is still a connection using

g_logging->info("is_connected trying ping");
GstcStatus ret = gstc_client_ping(m_client);
g_logging->info("ping result {}", ret);

setup the client using:

        const char *address = "127.0.0.1";
        const unsigned int port = 5000;
        const long wait_time_ms = 10000;
        const int keep_open = 1;
        GstcStatus ret = gstc_client_new(address, port, wait_time_ms, keep_open, &m_client);

and everything goes well for a long time, then suddenly after about 4 hours:

2022-06-22T07:25:00.921Z ping result 0
2022-06-22T07:25:00.922Z is_connected trying ping
2022-06-22T07:25:00.923Z ping result 0
2022-06-22T07:25:00.924Z is_connected trying ping
2022-06-22T07:25:00.925Z ping result 0
2022-06-22T07:25:00.983Z healthcheck: camera usb-046d_0825_2821F020-video-index0 is not connected, removing pipeline to try to restart it   elapsed_seconds.count() 87.688172837 > g_server_config->health_check_seconds : 60
2022-06-22T07:25:00.983Z is_connected trying ping
2022-06-22T07:25:00.984Z ping result 0
2022-06-22T07:25:00.984Z is_connected trying ping
2022-06-22T07:25:00.985Z ping result 0
2022-06-22T07:25:05.987Z is_connected trying ping
2022-06-22T07:25:10.991Z ping result -13

even if i start gstd-client-1.0 and try a list_pipelines it just hangs.

Tasks:   5 total,   1 running,   4 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.7 us,  5.4 sy,  0.0 ni, 85.6 id,  0.3 wa,  2.8 hi,  0.3 si,  0.0 st
KiB Mem :  4059240 total,    80412 free,  2376308 used,  1602520 buff/cache
KiB Swap:  2029616 total,  1668436 free,   361180 used.  1476376 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
   17 root      20   0 14.650g 1.072g 286168 S  56.2 27.7 140:45.31 gstd
    1 root      20   0  539868  44000   8388 S   0.0  1.1  20:03.63 streamcontrolle
    7 root      20   0    1912     68      0 S   0.0  0.0   0:00.00 rungstd.sh
  143 root      20   0    3784   2304   1840 S   0.0  0.1   0:00.06 bash
  508 root      20   0    5652   2740   2224 R   0.0  0.1   0:00.00 top

if i do an

strace gst-client-1.0 list_pipelines : https://gist.github.com/clogwog/8f19f0bf177a62253295b092d11dad62

i wanted to look at the logfiles that i specify root 17 33.9 27.7 15600100 1127380 ? Sl 01:36 142:24 /usr/local/bin/gstd -l /data/logs/gst.log -d /data/logs/gstd.log -f /tmp/ --gst-debug-level=2

but there are no logfiles created (i can't remember this ever working)

what would the next thing be that i can do to investigate this ?

clogwog commented 2 years ago

bueller ?