ossrs / srs

SRS is a simple, high-efficiency, real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181.
https://ossrs.io
MIT License
25.65k stars 5.38k forks source link

Cluster pull stream and break stream. #3535

Closed cat1555 closed 1 year ago

cat1555 commented 1 year ago

Note: Please read FAQ before file an issue, see #2716

Description

Cluster deployment, one origin node and one edge node. When I push the stream to the origin node and then pull the stream from the edge node, the stream will be interrupted first and then trigger the stream interruption callback.

Callback information: The stream is interrupted first, then pulled, and then interrupted again. 微信图片_20230509093643

SRS Version: 5.0.33

SRS Log:

origin node logs

[2023-05-09 09:16:48.250][Trace][26425][1ff1403z] XCORE-SRS/5.0.33(Leo) [2023-05-09 09:16:48.250][Trace][26425][1ff1403z] config parse complete [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] write log to console [2023-05-09 09:16:48.251][Trace][26425][1ff1403z][MAIN] SRS/5.0.33(Leo), MIT [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] authors: Winlinwinlin@vip.126.com ZhaoWenjiezhaowenjie@tal.com ShiWeishiwei05@kuaishou.com XiaoZhihonghondaxiao@tencent.com WuPengqiangpengqiang.wpq@alibaba-inc.com XiaLixinxialixin@kanzhun.com LiPengmozhan.lp@alibaba-inc.com ChenGuanghuajinxue.cgh@alibaba-inc.com ChenHaibonmgchenhaibo@foxmail.com Johzzyhellojinqiang@gmail.com Zhouxiaojun2008279686030@qq.com Pengfei.mapengfei.ma.chn@outlook.com ZhangJunqinzhangjunqin@jd.com and https://github.com/ossrs/srs/blob/develop/trunk/AUTHORS.md#contributors [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] cwd=/home/git/srs/trunk, work_dir=./, build: 2023-05-04 14:24:23, configure: --x86-x64, uname: Linux 47f5f5aa33bb 3.10.0-1160.88.1.el7.x86_64 #1 SMP Tue Mar 7 15:41:52 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux, osx: 0, pkg: , region: , source: [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] configure detail: --prefix=/usr/local/srs --config=conf/srs.conf --hls=on --hds=off --dvr=on --ssl=on --https=on --ssl-1-0=off --ssl-local=off --sys-ssl=off --transcode=on --ingest=on --stat=on --http-callback=on --http-server=on --stream-caster=on --http-api=on --utest=off --cherrypy=off --srt=on --rtc=on --simulator=off --cxx11=on --cxx14=off --ffmpeg-fit=on --nasm=on --srtp-nasm=on --clean=on --gperf=off --gmc=off --gmd=off --gmp=off --gcp=off --gprof=off --static=off --shared-st=off --shared-srt=off --shared-ffmpeg=off --log-verbose=off --log-info=off --log-trace=on --gcov=off --debug=off --debug-stats=off --cross-build=off --cc=gcc --cxx=g++ --ar=ar --ld=ld --randlib=randlib [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] srs checking config... [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] ips, iface[0] eth0 ipv4 0x11043 172.172.0.13 [2023-05-09 09:16:48.251][Trace][26425][1ff1403z] devices, internet eth0 172.172.0.13 [2023-05-09 09:16:48.251][Warn][26425][1ff1403z][22] stats network use index=0, ip=172.172.0.13, ifname=eth0 [2023-05-09 09:16:48.251][Warn][26425][1ff1403z][22] stats disk not configed, disk iops disabled. [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] write log to console [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] features, rch:on, dash:on, hls:on, hds:off, srt:on, hc:on, ha:on, hs:on, hp:on, dvr:on, trans:on, inge:on, stat:on, sc:on [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] SRS on amd64 x86_64, conf:conf/origin.conf, limit:1000, writev:1024, encoding:little-endian, HZ:100 [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] mw sleep:350ms. mr enabled:on, default:0, sleep:350ms [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] gc:on, pq:30000ms, cscc:[0,16), csa:on, tn:on(may hurts performance), ss:auto(guess by merged write) [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] system default latency(ms): mw(0-350) + mr(0-350) + play-queue(0-30000) [2023-05-09 09:16:48.252][Warn][26425][1ff1403z][22] SRS/5.0.33 is not stable [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] write pid=26425 to ./objs/origin.pid success! [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] Thread #1(primordial): init name=srs-master-1, interval=5000ms [2023-05-09 09:16:48.252][Trace][26425][1ff1403z] Pool: Start threads primordial=1, hybrids=1 ok [2023-05-09 09:16:48.253][Trace][26425][2y504l0c] Thread #2: run with tid=26426, entry=0x1251a40, label=hybrid, name=srs-hybrid-2 [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] fingerprint=8E:B5:5A:2A:79:3F:2D:02:31:29:19:22:3B:F2:5F:DC:0A:7D:4A:B2:A9:1D:2B:02:CE:F6:FB:F2:53:12:FA:24 [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] CircuitBreaker: enabled=1, high=2x90, critical=1x95, dying=5x99 [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] http flv live stream, vhost=defaultVhost, mount=[vhost]/[app]/[stream].flv [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] http: root mount to ./objs/nginx/html [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] server main cid=2y504l0c, pid=26425, ppid=25900, asprocess=0 [2023-05-09 09:16:48.266][Trace][26425][2y504l0c] RTMP listen at tcp://0.0.0.0:1935, fd=9 [2023-05-09 09:16:48.267][Trace][26425][2y504l0c] HTTP-API listen at tcp://0.0.0.0:1985, fd=10 [2023-05-09 09:16:48.267][Trace][26425][2y504l0c] HTTP-Server listen at tcp://0.0.0.0:8080, fd=11 [2023-05-09 09:16:48.267][Trace][26425][2y504l0c] signal installed, reload=1, reopen=10, fast_quit=15, grace_quit=3 [2023-05-09 09:16:48.267][Trace][26425][2y504l0c] http: api mount /console to ./objs/nginx/html/console [2023-05-09 09:16:48.268][Trace][26425][df94bw80] Hybrid cpu=0.00%,10MB [2023-05-09 09:16:48.268][Warn][26425][l41j1cq3][22] use public address as ip: 172.172.0.13, ifname=eth0 [2023-05-09 09:16:48.268][Trace][26425][l41j1cq3] Startup query id=2ba376daee0711edb7fc0904f800fe32, eip=172.172.0.13, wait=566s [2023-05-09 09:16:48.268][Trace][26425][0k4u990d] TCP: connection manager run, conns=0 [2023-05-09 09:16:48.268][Warn][26425][2y504l0c][11] enable auto reload for docker [2023-05-09 09:16:48.268][Trace][26425][2y504l0c] auto reload watching fd=12, watch=1, file=conf [2023-05-09 09:16:48.269][Trace][26425][t64qn455] SRT: connection manager run, conns=0 [2023-05-09 09:16:48.269][Trace][26425][t4057g3y] RTC: connection manager run, conns=0 [2023-05-09 09:16:53.253][Trace][26425][df94bw80] Hybrid cpu=0.00%,11MB [2023-05-09 09:16:53.258][Trace][26425][1ff1403z] Process: cpu=0.00%,11MB, threads=2 [2023-05-09 09:16:53.713][Trace][26425][b9210lb1] RTMP client ip=172.172.0.1:59077, fd=13 [2023-05-09 09:16:53.717][Trace][26425][b9210lb1] simple handshake success. [2023-05-09 09:16:53.718][Trace][26425][b9210lb1] connect app, tcUrl=rtmp://172.16.160.251:1935/activity, pageUrl=, swfUrl=rtmp://172.16.160.251:1935/activity, schema=rtmp, vhost=172.16.160.251, port=1935, app=activity, args=null [2023-05-09 09:16:53.718][Trace][26425][b9210lb1] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=4096, out.chunk=128 [2023-05-09 09:16:53.771][Trace][26425][b9210lb1] client identified, type=fmle-publish, vhost=172.16.160.251, app=activity, stream=5fcb99c0491b8e8c9dbd62c1093c2adf, param=, duration=0ms [2023-05-09 09:16:54.070][Trace][26425][f1035h60] HTTP #0 172.172.0.1:54180 GET http://172.16.160.251:1985/api/v1/streams/, content-length=-1 [2023-05-09 09:16:54.071][Trace][26425][f1035h60] TCP: before dispose resource(HttpConn)(0x7f96000336e0), conns=2, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:16:54.071][Warn][26425][f1035h60][104] client disconnect peer. ret=1007 [2023-05-09 09:16:54.071][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=2, removing=0, unsubs=0 [2023-05-09 09:16:54.071][Trace][26425][f1035h60] TCP: disposing #0 resource(HttpConn)(0x7f96000336e0), conns=2, disposing=1, zombies=0 [2023-05-09 09:16:54.073][Trace][26425][b9210lb1] http: on_connect ok, client_id=b9210lb1, url=http://172.16.160.251:1120/api/v2/intranet/push_success, request={"server_id":"vid-93i46x6","action":"on_connect","client_id":"b9210lb1","ip":"172.172.0.1","vhost":"defaultVhost","app":"activity","stream":"5fcb99c0491b8e8c9dbd62c1093c2adf","param":"","tcUrl":"rtmp://172.16.160.251:1935/activity","pageUrl":""}, response={"code":0,"message":"Callback successful","data":true} [2023-05-09 09:16:54.073][Trace][26425][b9210lb1] connected stream, tcUrl=rtmp://172.16.160.251:1935/activity, pageUrl=, swfUrl=rtmp://172.16.160.251:1935/activity, schema=rtmp, vhost=defaultVhost, port=1935, app=activity, stream=5fcb99c0491b8e8c9dbd62c1093c2adf, param=, args=null [2023-05-09 09:16:54.073][Trace][26425][b9210lb1] new live source, stream_url=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf [2023-05-09 09:16:54.073][Trace][26425][b9210lb1] source url=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, ip=172.172.0.1, cache=1, is_edge=0, source_id=/ [2023-05-09 09:16:54.404][Trace][26425][9b182r71] HTTP #0 172.172.0.1:54218 GET http://172.16.160.251:1985/api/v1/streams/, content-length=-1 [2023-05-09 09:16:54.405][Trace][26425][9b182r71] TCP: before dispose resource(HttpConn)(0x7f9600036540), conns=2, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:16:54.405][Warn][26425][9b182r71][104] client disconnect peer. ret=1007 [2023-05-09 09:16:54.405][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=2, removing=0, unsubs=0 [2023-05-09 09:16:54.405][Trace][26425][9b182r71] TCP: disposing #0 resource(HttpConn)(0x7f9600036540), conns=2, disposing=1, zombies=0 [2023-05-09 09:16:54.409][Trace][26425][b9210lb1] http: on_publish ok, client_id=b9210lb1, url=http://172.16.160.251:1120/api/v2/intranet/push_success, request={"server_id":"vid-93i46x6","action":"on_publish","client_id":"b9210lb1","ip":"172.172.0.1","vhost":"defaultVhost","app":"activity","tcUrl":"rtmp://172.16.160.251:1935/activity","stream":"5fcb99c0491b8e8c9dbd62c1093c2adf","param":""}, response={"code":0,"message":"Callback successful","data":true} [2023-05-09 09:16:54.410][Trace][26425][b9210lb1] hls: win=86400000ms, frag=5000ms, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream]/[stream].m3u8, ts=[app]/[stream]/[2006][01][02]/[15][04][05].ts, aof=2.00, floor=0, clean=0, waitk=1, dispose=86400000ms, dts_directly=1 [2023-05-09 09:16:54.410][Trace][26425][b9210lb1] dvr stream 5fcb99c0491b8e8c9dbd62c1093c2adf to file ./objs/nginx/html/activity-transcoding/5fcb99c0491b8e8c9dbd62c1093c2adf.1683595014410.mp4 [2023-05-09 09:16:54.410][Trace][26425][b9210lb1] ignore disabled exec for vhost=defaultVhost [2023-05-09 09:16:54.411][Trace][26425][b9210lb1] http: mount flv stream for sid=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, mount=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf.flv [2023-05-09 09:16:54.411][Trace][26425][b9210lb1] start publish mr=0/350, p1stpt=600000, pnt=600000, tcp_nodelay=0 [2023-05-09 09:16:54.411][Trace][26425][b9210lb1] got metadata, width=1920, height=1080, vcodec=7, acodec=10 [2023-05-09 09:16:54.414][Trace][26425][m0348551] RTMP client ip=172.172.0.117:54050, fd=16 [2023-05-09 09:16:54.417][Trace][26425][m0348551] complex handshake success [2023-05-09 09:16:54.418][Trace][26425][m0348551] connect app, tcUrl=rtmp://172.172.0.13/activity, pageUrl=, swfUrl=, schema=rtmp, vhost=172.172.0.13, port=1935, app=activity, args=(obj) [2023-05-09 09:16:54.418][Trace][26425][m0348551] edge-srs ip=172.172.0.117, version=5.0.33, pid=23152, id=0 [2023-05-09 09:16:54.418][Trace][26425][m0348551] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128 [2023-05-09 09:16:54.458][Trace][26425][m0348551] client identified, type=rtmp-play, vhost=172.172.0.13, app=activity, stream=5fcb99c0491b8e8c9dbd62c1093c2adf, param=, duration=-1ms [2023-05-09 09:16:54.679][Trace][26425][b9210lb1] 7B audio sh, codec(10, profile=LC, 2channels, 0kbps, 48000HZ), flv(16bits, 2channels, 44100HZ) [2023-05-09 09:16:54.679][Trace][26425][b9210lb1] 58B video sh, codec(7, profile=High, level=4, 1920x1080, 0kbps, 0.0fps, 0.0s) [2023-05-09 09:16:54.748][Trace][26425][1013il40] HTTP #0 172.172.0.1:54258 GET http://172.16.160.251:1985/api/v1/streams/, content-length=-1 [2023-05-09 09:16:54.748][Trace][26425][1013il40] TCP: before dispose resource(HttpConn)(0x7f9600059970), conns=3, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:16:54.749][Warn][26425][1013il40][104] client disconnect peer. ret=1007 [2023-05-09 09:16:54.749][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0 [2023-05-09 09:16:54.749][Trace][26425][1013il40] TCP: disposing #0 resource(HttpConn)(0x7f9600059970), conns=3, disposing=1, zombies=0 [2023-05-09 09:16:54.750][Trace][26425][m0348551] http: on_connect ok, client_id=m0348551, url=http://172.16.160.251:1120/api/v2/intranet/push_success, request={"server_id":"vid-93i46x6","action":"on_connect","client_id":"m0348551","ip":"172.172.0.117","vhost":"defaultVhost","app":"activity","stream":"5fcb99c0491b8e8c9dbd62c1093c2adf","param":"","tcUrl":"rtmp://172.172.0.13/activity","pageUrl":""}, response={"code":0,"message":"Callback successful","data":true} [2023-05-09 09:16:54.751][Trace][26425][m0348551] connected stream, tcUrl=rtmp://172.172.0.13/activity, pageUrl=, swfUrl=, schema=rtmp, vhost=defaultVhost, port=1935, app=activity, stream=5fcb99c0491b8e8c9dbd62c1093c2adf, param=, args=(obj) [2023-05-09 09:16:54.751][Trace][26425][m0348551] source url=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, ip=172.172.0.117, cache=1, is_edge=0, source_id=b9210lb1/b9210lb1 [2023-05-09 09:16:54.751][Trace][26425][m0348551] dispatch cached gop success. count=2, duration=40 [2023-05-09 09:16:54.751][Trace][26425][m0348551] create consumer, active=1, queue_size=0.00, jitter=30000000 [2023-05-09 09:16:54.751][Trace][26425][m0348551] set fd=16, SO_SNDBUF=87040=>175000, buffer=350ms [2023-05-09 09:16:54.751][Trace][26425][m0348551] start play smi=0ms, mw_sleep=350, mw_msgs=8, realtime=0, tcp_nodelay=0 [2023-05-09 09:16:58.254][Trace][26425][df94bw80] Hybrid cpu=1.00%,12MB [2023-05-09 09:16:58.263][Trace][26425][1ff1403z] Process: cpu=3.00%,13MB, threads=2 [2023-05-09 09:17:03.255][Trace][26425][df94bw80] Hybrid cpu=3.00%,14MB, cid=6,3, timer=62,0,0, clock=1,44,4,0,0,0,0,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:206,oth:0,buf:0) [2023-05-09 09:17:03.268][Trace][26425][1ff1403z] Process: cpu=2.00%,14MB, threads=2 [2023-05-09 09:17:04.084][Trace][26425][b9210lb1] -> HLS time=10006326ms, sno=1, ts=20230509/091654.ts, dur=9360ms, dva=0p [2023-05-09 09:17:05.235][Trace][26425][m0348551] -> PLA time=10130300, msgs=25, okbps=1975,0,0, ikbps=2,0,0, mw=350/8 [2023-05-09 09:17:06.003][Trace][26425][b9210lb1] cleanup when unpublish [2023-05-09 09:17:06.107][Trace][26425][2y504l0c] http hook on_dvr success. client_id=b9210lb1, url=http://172.16.160.251:1120/api/v2/intranet/record_video_complete_video, request={"server_id":"vid-93i46x6","action":"on_dvr","client_id":"b9210lb1","ip":"172.172.0.1","vhost":"defaultVhost","app":"activity","stream":"5fcb99c0491b8e8c9dbd62c1093c2adf","param":"","cwd":"/home/git/srs/trunk","file":"./objs/nginx/html/activity-transcoding/5fcb99c0491b8e8c9dbd62c1093c2adf.1683595014410.mp4"}, response={"code":0,"message":"Callback successful","data":true} [2023-05-09 09:17:06.284][Trace][26425][u7cus83q] HTTP #0 172.172.0.1:54828 GET http://172.16.160.251:1985/api/v1/streams/, content-length=-1 [2023-05-09 09:17:06.284][Trace][26425][u7cus83q] TCP: before dispose resource(HttpConn)(0x7f9600063b50), conns=3, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:17:06.285][Warn][26425][u7cus83q][104] client disconnect peer. ret=1007 [2023-05-09 09:17:06.285][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0 [2023-05-09 09:17:06.285][Trace][26425][u7cus83q] TCP: disposing #0 resource(HttpConn)(0x7f9600063b50), conns=3, disposing=1, zombies=0 [2023-05-09 09:17:06.286][Trace][26425][b9210lb1] http: on_unpublish ok, client_id=b9210lb1, url=http://172.16.160.251:1120/api/v2/intranet/push_success, request={"server_id":"vid-93i46x6","action":"on_unpublish","client_id":"b9210lb1","ip":"172.172.0.1","vhost":"defaultVhost","app":"activity","stream":"5fcb99c0491b8e8c9dbd62c1093c2adf","param":""}, response={"code":0,"message":"Callback successful","data":true} [2023-05-09 09:17:06.286][Trace][26425][b9210lb1] TCP: before dispose resource(RtmpConn)(0x7f9600028b00), conns=2, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:17:06.286][Warn][26425][b9210lb1][32] client disconnect peer. ret=1009 [2023-05-09 09:17:06.286][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=2, removing=0, unsubs=0 [2023-05-09 09:17:06.286][Trace][26425][b9210lb1] TCP: disposing #0 resource(RtmpConn)(0x7f9600028b00), conns=2, disposing=1, zombies=0 [2023-05-09 09:17:08.255][Trace][26425][df94bw80] Hybrid cpu=0.00%,14MB, cid=6,3, timer=62,0,0, clock=1,44,4,0,0,0,0,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:206,oth:0,buf:0) [2023-05-09 09:17:08.273][Trace][26425][1ff1403z] Process: cpu=2.00%,14MB, threads=2 [2023-05-09 09:17:09.569][Trace][26425][m0348551] TCP: before dispose resource(RtmpConn)(0x7f96000591b0), conns=1, zombies=0, ign=0, inz=0, ind=0 [2023-05-09 09:17:09.569][Warn][26425][m0348551][104] client disconnect peer. ret=1007 [2023-05-09 09:17:09.569][Trace][26425][0k4u990d] TCP: clear zombies=1 resources, conns=1, removing=0, unsubs=0 [2023-05-09 09:17:09.569][Trace][26425][m0348551] TCP: disposing #0 resource(RtmpConn)(0x7f96000591b0), conns=1, disposing=1, zombies=0 ^C[2023-05-09 09:17:10.053][Trace][26425][2y504l0c] sig=2, user terminate program, fast quit [2023-05-09 09:17:10.263][Trace][26425][2y504l0c] cleanup for quit signal fast=1, grace=0 [2023-05-09 09:17:10.274][Trace][26425][1ff1403z] quit for thread #2(hybrid) finished

Edge Node Log

[2023-05-09 09:16:37.532][Trace][23152][038co826] XCORE-SRS/5.0.33(Leo) [2023-05-09 09:16:37.532][Trace][23152][038co826] config parse complete [2023-05-09 09:16:37.532][Warn][23152][038co826][22] transform: vhost.mode to vhost.cluster.mode of vhost [2023-05-09 09:16:37.532][Trace][23152][038co826] write log to console [2023-05-09 09:16:37.532][Trace][23152][038co826][MAIN] SRS/5.0.33(Leo), MIT [2023-05-09 09:16:37.532][Trace][23152][038co826] authors: Winlinwinlin@vip.126.com ZhaoWenjiezhaowenjie@tal.com ShiWeishiwei05@kuaishou.com XiaoZhihonghondaxiao@tencent.com WuPengqiangpengqiang.wpq@alibaba-inc.com XiaLixinxialixin@kanzhun.com LiPengmozhan.lp@alibaba-inc.com ChenGuanghuajinxue.cgh@alibaba-inc.com ChenHaibonmgchenhaibo@foxmail.com Johzzyhellojinqiang@gmail.com Zhouxiaojun2008279686030@qq.com Pengfei.mapengfei.ma.chn@outlook.com ZhangJunqinzhangjunqin@jd.com and https://github.com/ossrs/srs/blob/develop/trunk/AUTHORS.md#contributors [2023-05-09 09:16:37.532][Trace][23152][038co826] cwd=/home/git/srs/trunk, work_dir=./, build: 2023-04-14 10:41:01, configure: --x86-x64 --ffmpeg-tool=on, uname: Linux 47f5f5aa33bb 3.10.0-1160.15.2.el7.x86_64 #1 SMP Wed Feb 3 15:06:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux, osx: 0, pkg: , region: , source: [2023-05-09 09:16:37.532][Trace][23152][038co826] configure detail: --prefix=/usr/local/srs --config=conf/srs.conf --hls=on --hds=off --dvr=on --ssl=on --https=on --ssl-1-0=off --ssl-local=off --sys-ssl=off --transcode=on --ingest=on --stat=on --http-callback=on --http-server=on --stream-caster=on --http-api=on --utest=off --cherrypy=off --srt=on --rtc=on --simulator=off --cxx11=on --cxx14=off --ffmpeg-fit=on --nasm=on --srtp-nasm=on --clean=on --gperf=off --gmc=off --gmd=off --gmp=off --gcp=off --gprof=off --static=off --shared-st=off --shared-srt=off --shared-ffmpeg=off --log-verbose=off --log-info=off --log-trace=on --gcov=off --debug=off --debug-stats=off --cross-build=off --cc=gcc --cxx=g++ --ar=ar --ld=ld --randlib=randlib [2023-05-09 09:16:37.532][Trace][23152][038co826] srs checking config... [2023-05-09 09:16:37.533][Trace][23152][038co826] ips, iface[0] eth0 ipv4 0x11043 172.172.0.117 [2023-05-09 09:16:37.533][Trace][23152][038co826] devices, internet eth0 172.172.0.117 [2023-05-09 09:16:37.533][Warn][23152][038co826][22] stats network use index=0, ip=172.172.0.117, ifname=eth0 [2023-05-09 09:16:37.533][Warn][23152][038co826][22] stats disk not configed, disk iops disabled. [2023-05-09 09:16:37.533][Trace][23152][038co826] write log to console [2023-05-09 09:16:37.533][Trace][23152][038co826] features, rch:on, dash:on, hls:on, hds:off, srt:on, hc:on, ha:on, hs:on, hp:on, dvr:on, trans:on, inge:on, stat:on, sc:on [2023-05-09 09:16:37.533][Trace][23152][038co826] SRS on amd64 x86_64, conf:conf/edge.conf, limit:1000, writev:1024, encoding:little-endian, HZ:100 [2023-05-09 09:16:37.533][Trace][23152][038co826] mw sleep:350ms. mr enabled:on, default:0, sleep:350ms [2023-05-09 09:16:37.533][Trace][23152][038co826] gc:on, pq:30000ms, cscc:[0,16), csa:on, tn:on(may hurts performance), ss:auto(guess by merged write) [2023-05-09 09:16:37.533][Trace][23152][038co826] system default latency(ms): mw(0-350) + mr(0-350) + play-queue(0-30000) [2023-05-09 09:16:37.533][Warn][23152][038co826][22] SRS/5.0.33 is not stable [2023-05-09 09:16:37.534][Trace][23152][038co826] write pid=23152 to objs/edge.pid success! [2023-05-09 09:16:37.534][Trace][23152][038co826] Thread #1(primordial): init name=srs-master-1, interval=5000ms [2023-05-09 09:16:37.534][Trace][23152][038co826] Pool: Start threads primordial=1, hybrids=1 ok [2023-05-09 09:16:37.534][Trace][23152][e938j5e4] Thread #2: run with tid=23153, entry=0x1769490, label=hybrid, name=srs-hybrid-2 [2023-05-09 09:16:37.546][Trace][23152][e938j5e4] fingerprint=C5:70:B9:0E:71:72:BD:42:06:1C:D9:4E:4C:59:28:5C:16:EC:CE:70:0A:C2:01:8F:46:58:CE:9F:46:8E:F4:B1 [2023-05-09 09:16:37.546][Trace][23152][e938j5e4] CircuitBreaker: enabled=1, high=2x90, critical=1x95, dying=5x99 [2023-05-09 09:16:37.546][Trace][23152][e938j5e4] http flv live stream, vhost=defaultVhost, mount=[vhost]/[app]/[stream].flv [2023-05-09 09:16:37.546][Trace][23152][e938j5e4] http: root mount to ./objs/nginx/html [2023-05-09 09:16:37.546][Trace][23152][e938j5e4] server main cid=e938j5e4, pid=23152, ppid=22780, asprocess=0 [2023-05-09 09:16:37.547][Trace][23152][e938j5e4] RTMP listen at tcp://0.0.0.0:1935, fd=9 [2023-05-09 09:16:37.547][Trace][23152][e938j5e4] HTTP-Server listen at tcp://0.0.0.0:8080, fd=10 [2023-05-09 09:16:37.547][Trace][23152][e938j5e4] signal installed, reload=1, reopen=10, fast_quit=15, grace_quit=3 [2023-05-09 09:16:37.547][Trace][23152][e938j5e4] http: api mount /console to ./objs/nginx/html/console [2023-05-09 09:16:37.547][Trace][23152][93417j86] Hybrid cpu=0.00%,10MB [2023-05-09 09:16:37.548][Warn][23152][8362776k][22] use public address as ip: 172.172.0.117, ifname=eth0 [2023-05-09 09:16:37.548][Trace][23152][8362776k] Startup query id=253fb81cee0711ed86f07b9809f5bf4b, eip=172.172.0.117, wait=300s [2023-05-09 09:16:37.548][Trace][23152][3584c4i7] TCP: connection manager run, conns=0 [2023-05-09 09:16:37.548][Warn][23152][e938j5e4][11] enable auto reload for docker [2023-05-09 09:16:37.548][Trace][23152][e938j5e4] auto reload watching fd=11, watch=1, file=conf [2023-05-09 09:16:37.549][Trace][23152][9338j140] SRT: connection manager run, conns=0 [2023-05-09 09:16:37.549][Trace][23152][136728yr] RTC: connection manager run, conns=0 [2023-05-09 09:16:42.535][Trace][23152][93417j86] Hybrid cpu=1.00%,10MB [2023-05-09 09:16:42.539][Trace][23152][038co826] Process: cpu=1.00%,10MB, threads=2 [2023-05-09 09:16:47.535][Trace][23152][93417j86] Hybrid cpu=0.00%,11MB [2023-05-09 09:16:47.544][Trace][23152][038co826] Process: cpu=1.00%,11MB, threads=2 [2023-05-09 09:16:52.536][Trace][23152][93417j86] Hybrid cpu=0.00%,11MB, cid=1,1, timer=62,0,0, clock=1,48,1,0,0,0,0,0,0 [2023-05-09 09:16:52.549][Trace][23152][038co826] Process: cpu=0.00%,11MB, threads=2 [2023-05-09 09:16:54.413][Trace][23152][912120dm] HTTP #0 172.172.0.1:59078 GET http://172.16.160.251:9998/activity/5fcb99c0491b8e8c9dbd62c1093c2adf.flv, content-length=-1 [2023-05-09 09:16:54.413][Trace][23152][912120dm] new live source, stream_url=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf [2023-05-09 09:16:54.413][Trace][23152][912120dm] http: mount flv stream for sid=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, mount=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf.flv [2023-05-09 09:16:54.413][Trace][23152][912120dm] flv: source url=/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, is_edge=1, source_id=/ [2023-05-09 09:16:54.413][Trace][23152][912120dm] ignore disabled exec for vhost=defaultVhost [2023-05-09 09:16:54.414][Trace][23152][912120dm] dispatch cached gop success. count=0, duration=-1 [2023-05-09 09:16:54.414][Trace][23152][912120dm] create consumer, active=1, queue_size=0.00, jitter=30000000 [2023-05-09 09:16:54.414][Trace][23152][912120dm] set fd=12, SO_SNDBUF=87040=>175000, buffer=350ms [2023-05-09 09:16:54.414][Trace][23152][912120dm] FLV /activity/5fcb99c0491b8e8c9dbd62c1093c2adf.flv, encoder=FLV, nodelay=0, mw_sleep=350ms, cache=0, msgs=128 [2023-05-09 09:16:54.414][Trace][23152][912120dm] update source_id=912120dm/912120dm [2023-05-09 09:16:54.417][Trace][23152][3ab513lo] complex handshake success. [2023-05-09 09:16:54.417][Trace][23152][3ab513lo] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128 [2023-05-09 09:16:54.458][Trace][23152][3ab513lo] connected, version=5.0.33.0, ip=172.172.0.13, pid=26425, id=0, dsu=1 [2023-05-09 09:16:54.458][Trace][23152][3ab513lo] edge-pull publish url rtmp://172.172.0.13:1935/activity/5fcb99c0491b8e8c9dbd62c1093c2adf, stream=5fcb99c0491b8e8c9dbd62c1093c2adf as 5fcb99c0491b8e8c9dbd62c1093c2adf [2023-05-09 09:16:54.458][Trace][23152][3ab513lo] edge change from 100 to state 101 (pull). [2023-05-09 09:16:54.751][Trace][23152][3ab513lo] got metadata, width=1920, height=1080, vcodec=7, acodec=10 [2023-05-09 09:16:54.752][Trace][23152][3ab513lo] 7B audio sh, codec(10, profile=LC, 2channels, 0kbps, 48000HZ), flv(16bits, 2channels, 44100HZ) [2023-05-09 09:16:54.752][Trace][23152][3ab513lo] 58B video sh, codec(7, profile=High, level=4, 1920x1080, 0kbps, 0.0fps, 0.0s) [2023-05-09 09:16:54.763][Trace][23152][912120dm] update source_id=3ab513lo/912120dm [2023-05-09 09:16:54.763][Trace][23152][912120dm] FLV: write header audio=1, video=1 [2023-05-09 09:16:57.537][Trace][23152][93417j86] Hybrid cpu=0.00%,12MB, cid=1,1, timer=62,0,0, clock=1,48,1,0,0,0,0,0,0 [2023-05-09 09:16:57.554][Trace][23152][038co826] Process: cpu=1.00%,12MB, threads=2 [2023-05-09 09:17:02.537][Trace][23152][93417j86] Hybrid cpu=2.00%,13MB, cid=3,1, timer=62,0,0, clock=0,47,1,0,0,0,0,0,0, objs=(pkt:0,raw:0,fua:0,msg:165,oth:0,buf:0) [2023-05-09 09:17:02.559][Trace][23152][038co826] Process: cpu=0.00%,13MB, threads=2 [2023-05-09 09:17:04.519][Trace][23152][3ab513lo] <- EIG time=10074914, okbps=2,0,0, ikbps=1983,0,0 [2023-05-09 09:17:04.577][Trace][23152][912120dm] -> HTS http: got 26 msgs, age=10156839, min=8, mw=350 [2023-05-09 09:17:07.538][Trace][23152][93417j86] Hybrid cpu=0.00%,14MB, cid=3,1, timer=62,0,0, clock=0,47,1,0,0,0,0,0,0, objs=(pkt:0,raw:0,fua:0,msg:165,oth:0,buf:0) [2023-05-09 09:17:07.564][Trace][23152][038co826] Process: cpu=1.00%,14MB, threads=2 ^C[2023-05-09 09:17:08.967][Trace][23152][e938j5e4] sig=2, user terminate program, fast quit [2023-05-09 09:17:09.553][Trace][23152][e938j5e4] cleanup for quit signal fast=1, grace=0 [2023-05-09 09:17:09.566][Trace][23152][038co826] quit for thread #2(hybrid) finished

  1. SRS Config:

    Origin Configuration File

    listen 1935; max_connections 1000; pid ./objs/origin.pid; daemon off; srs_log_tank console; http_server { enabled on; listen 8080; dir ./objs/nginx/html; } http_api { enabled on; listen 1985; crossdomain on; raw_api { enabled on; allow_reload off; allow_query on; allow_update off; }

} vhost defaultVhost { publish{ firstpkt_timeout 600000; normal_timeout 600000; } http_remux { enabled on; mount [vhost]/[app]/[stream].flv; hstrs on; }

hls { enabled on; hls_fragment 5; hls_window 86400; hls_path ./objs/nginx/html; hls_m3u8_file [app]/[stream]/[stream].m3u8; hls_ts_file [app]/[stream]/[2006][01][02]/[15][04][05].ts; hls_cleanup off; hls_dispose 86400; } http_hooks { enabled on; on_connect http://172.16.160.251:1120/api/v2/intranet/push_success; on_publish http://172.16.160.251:1120/api/v2/intranet/push_success; on_unpublish http://172.16.160.251:1120/api/v2/intranet/push_success; on_dvr http://172.16.160.251:1120/api/v2/intranet/record_video_complete_video; }

dvr { enabled on; dvr_path ./objs/nginx/html/activity-transcoding/[stream].[timestamp].mp4; dvr_plan { segment_duration 30; dvr_wait_keyframe on; time_jitter full; ts_duration 15; } } cluster { mode local; origin_cluster on; coworkers 127.127.0.1:1985; }

}

Edge Node Configuration File

listen 1935; max_connections 1000; pid objs/edge.pid; daemon off; srs_log_tank console; http_server { enabled on; listen 8080; dir ./objs/nginx/html; } vhost defaultVhost { mode remote;

cluster { mode remote; origin 172.172.0.13:1935; }

http_remux {
enabled on; mount [vhost]/[app]/[stream].flv; hstrs on; }

}

Replay

Please describe how to replay the bug?

Step 1: Using OBS to stream to the origin source node.

Step 2: When pulling the stream from the edge node, it will first be disconnected, and then the disconnection callback will be triggered.

Expect

Please describe your expectation.

TRANS_BY_GPT3

duiniuluantanqin commented 1 year ago

Did you modify the source code?

TRANS_BY_GPT3

cat1555 commented 1 year ago

image Does it have any impact if I only modified this?

TRANS_BY_GPT3

duiniuluantanqin commented 1 year ago

Please pull the latest code first and try not to make any modifications.

TRANS_BY_GPT3

cat1555 commented 1 year ago

Okay, I'll give it a try

TRANS_BY_GPT3

cat1555 commented 1 year ago

image The code that is pulled again will still break the flow.

TRANS_BY_GPT3

cat1555 commented 1 year ago

image

cat1555 commented 1 year ago

Already resolved the issue with the interface callback. Thank you for your help.

TRANS_BY_GPT3

duiniuluantanqin commented 1 year ago

ok