search

ossrs / srs-gb28181

GB28181 server based on SRS
https://ossrs.net
MIT License
83 stars 41 forks source link

H265: FFMPEG 对 H265 的 RTMP 直播流转码支持 #2562 #31

Open winlinvip opened 2 years ago

winlinvip commented 2 years ago

描述(Description)

  1. 摄像头通过 RTMP 推 H265 流到 SRS，265 web 播放器能够正常播放，由于大屏需要同时播放至多 10+ 路的视频， 受限于 Web 端机器的解码能力无法做到同时播放多路的 H265 视频；所以我打算在Origin-cluster 的部署架构下， 在 edge 端使用 FFmpeg Transcode 对 265 视频流按需转码为 H264 进行分发，但是不能正常拉流

    1. 以下日志为 h265 -> srs -> 播放 h264 单进程模式进行的测试，不能正常拉流

能否扩展 FFmpeg 转码对 h265 的支持

  1. SRS版本(Version): feature/h265 v4.0.156
  2. SRS的日志如下(Log): 
    [2021-08-30 09:41:21.965][Trace][11][7k5ts90s] Hybrid cpu=0.00%,15MB, cid=3,1, timer=61,0,0, clock=0,40,5,1,1,1,0,1,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:117,oth:0,buf:0)
[2021-08-30 09:41:26.966][Trace][11][7k5ts90s] Hybrid cpu=1.00%,15MB, cid=3,1, timer=62,0,0, clock=0,46,2,1,0,0,0,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:8,oth:0,buf:0)
[2021-08-30 09:41:31.967][Trace][11][7k5ts90s] Hybrid cpu=0.00%,15MB, cid=3,1, timer=62,0,0, clock=0,46,2,1,0,0,0,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:8,oth:0,buf:0)
[2021-08-30 09:41:36.967][Trace][11][7k5ts90s] Hybrid cpu=0.00%,15MB, cid=3,1, timer=62,0,0, clock=0,46,2,1,0,0,0,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:8,oth:0,buf:0)
[2021-08-30 09:41:39.589][Trace][11][41i2w1qj] RTMP client ip=10.121.192.0:36656, fd=13
[2021-08-30 09:41:39.599][Trace][11][41i2w1qj] simple handshake success.
[2021-08-30 09:41:39.599][Trace][11][41i2w1qj] connect app, tcUrl=rtmp://push-pek-dev.domain.com:1935/live, pageUrl=, swfUrl=, schema=rtmp, vhost=push-pek-dev.domain.com, port=1935, app=live, args=null
[2021-08-30 09:41:39.599][Trace][11][41i2w1qj] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128
[2021-08-30 09:41:39.647][Trace][11][41i2w1qj] client identified, type=fmle-publish, vhost=push-pek-dev.domain.com, app=live, stream=PkdiDbzcHJnN, param=?auth_sign=1630402035757-0-03C80017C73E19AC-ecfdcf4a9883665eadd0dae1e554533442c63654188908c560398ee64e1ea5d9, duration=0ms
[2021-08-30 09:41:39.647][Trace][11][41i2w1qj] connected stream, tcUrl=rtmp://push-pek-dev.domain.com:1935/live, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, stream=PkdiDbzcHJnN, param=?auth_sign=1630402035757-0-03C80017C73E19AC-ecfdcf4a9883665eadd0dae1e554533442c63654188908c560398ee64e1ea5d9, args=null
[2021-08-30 09:41:39.648][Trace][11][41i2w1qj] source url=/live/PkdiDbzcHJnN, ip=10.121.192.0, cache=0, is_edge=0, source_id=/bhk472ek
[2021-08-30 09:41:39.694][Trace][11][41i2w1qj] hls: win=30000ms, frag=6000ms, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream].m3u8, ts=[app]/[stream]/[timestamp].ts, aof=2.00, floor=0, clean=1, waitk=1, dispose=180000ms, dts_directly=1
[2021-08-30 09:41:39.695][Trace][11][41i2w1qj] ignore disabled exec for vhost=__defaultVhost__
[2021-08-30 09:41:39.696][Trace][11][41i2w1qj] set fd=13 TCP_NODELAY 0=>1
[2021-08-30 09:41:39.696][Trace][11][41i2w1qj] start publish mr=0/350, p1stpt=20000, pnt=5000, tcp_nodelay=1
[2021-08-30 09:41:39.701][Trace][11][7073iy8q] RTMP client ip=127.0.0.1:40460, fd=15
[2021-08-30 09:41:39.703][Trace][11][7073iy8q] complex handshake success
[2021-08-30 09:41:39.704][Trace][11][41i2w1qj] fored process, pid=36, bin=./objs/ffmpeg/bin/ffmpeg, stdout=./objs/ffmpeg-encoder-__defaultVhost__-live-PkdiDbzcHJnN-h265.log, stderr=./objs/ffmpeg-encoder-__defaultVhost__-live-PkdiDbzcHJnN-h265.log, argv=./objs/ffmpeg/bin/ffmpeg -f flv -i rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__/PkdiDbzcHJnN -vcodec libx264 -threads 12 -profile:v main -preset medium -acodec  -f flv -y rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__/PkdiDbzcHJnN_h265
[2021-08-30 09:41:39.704][Trace][11][41i2w1qj] -> ENC time=519829558, encoders=1, input=__defaultVhost__/live/PkdiDbzcHJnN
[2021-08-30 09:41:39.742][Trace][11][7073iy8q] connect app, tcUrl=rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, args=null
[2021-08-30 09:41:39.742][Trace][11][7073iy8q] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128
[2021-08-30 09:41:39.777][Trace][11][41i2w1qj] 110B video sh,  codec(12, profile=Other, level=Other, 0x0, 0kbps, 0.0fps, 0.0s)
[2021-08-30 09:41:39.777][Trace][11][41i2w1qj] -> HLS time=519890958ms, sno=26, ts=PkdiDbzcHJnN/1630316499694.ts, dur=0.00, dva=0p
[2021-08-30 09:41:39.823][Trace][11][7073iy8q] ignore AMF0/AMF3 command message.
[2021-08-30 09:41:39.862][Trace][11][7073iy8q] ignore AMF0/AMF3 command message.
[2021-08-30 09:41:39.862][Trace][11][7073iy8q] client identified, type=rtmp-play, vhost=__defaultVhost__, app=live, stream=PkdiDbzcHJnN, param=, duration=-1ms
[2021-08-30 09:41:39.862][Trace][11][7073iy8q] connected stream, tcUrl=rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, stream=PkdiDbzcHJnN, param=, args=null
[2021-08-30 09:41:39.862][Trace][11][7073iy8q] source url=/live/PkdiDbzcHJnN, ip=127.0.0.1, cache=0, is_edge=0, source_id=41i2w1qj/bhk472ek
[2021-08-30 09:41:39.863][Trace][11][7073iy8q] set fd=15 TCP_NODELAY 0=>1
[2021-08-30 09:41:39.863][Trace][11][7073iy8q] dispatch cached gop success. count=0, duration=9
[2021-08-30 09:41:39.863][Trace][11][7073iy8q] create consumer, active=1, queue_size=0.00, jitter=10000000
[2021-08-30 09:41:39.863][Trace][11][7073iy8q] set fd=15, SO_SNDBUF=2626560=>50000, buffer=100ms
[2021-08-30 09:41:39.863][Trace][11][7073iy8q] start play smi=0ms, mw_sleep=100, mw_msgs=0, realtime=1, tcp_nodelay=1
[2021-08-30 09:41:39.949][Warn][11][41i2w1qj][11] AUDIO: stream not monotonically increase, please open mix_correct.
[2021-08-30 09:41:39.949][Trace][11][41i2w1qj] 4B audio sh, codec(10, profile=Main, 1channels, 0kbps, 16000HZ), flv(16bits, 2channels, 44100HZ)
[2021-08-30 09:41:41.967][Trace][11][7k5ts90s] Hybrid cpu=2.00%,15MB, cid=3,1, timer=62,0,0, clock=0,45,2,1,1,1,1,0,0, objs=(pkt:0,raw:0,fua:0,msg:21,oth:0,buf:0)
[2021-08-30 09:41:44.874][Trace][11][v3h3p70e] RTMP client ip=10.121.192.0:36684, fd=16
[2021-08-30 09:41:44.910][Trace][11][v3h3p70e] simple handshake success.
[2021-08-30 09:41:44.911][Trace][11][v3h3p70e] connect app, tcUrl=rtmp://push-pek-dev.domain.com/live, pageUrl=, swfUrl=, schema=rtmp, vhost=push-pek-dev.domain.com, port=1935, app=live, args=null
[2021-08-30 09:41:44.912][Trace][11][v3h3p70e] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128
[2021-08-30 09:41:44.989][Trace][11][v3h3p70e] ignore AMF0/AMF3 command message.
[2021-08-30 09:41:45.025][Trace][11][v3h3p70e] client identified, type=rtmp-play, vhost=push-pek-dev.domain.com, app=live, stream=PkdiDbzcHJnN_h265, param=, duration=-1ms
[2021-08-30 09:41:45.025][Trace][11][v3h3p70e] connected stream, tcUrl=rtmp://push-pek-dev.domain.com/live, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, stream=PkdiDbzcHJnN_h265, param=, args=null
[2021-08-30 09:41:45.025][Trace][11][v3h3p70e] source url=/live/PkdiDbzcHJnN_h265, ip=10.121.192.0, cache=0, is_edge=0, source_id=/
[2021-08-30 09:41:45.026][Trace][11][v3h3p70e] set fd=16 TCP_NODELAY 0=>1
[2021-08-30 09:41:45.026][Trace][11][v3h3p70e] create consumer, active=0, queue_size=0.00, jitter=10000000
[2021-08-30 09:41:45.026][Trace][11][v3h3p70e] set fd=16, SO_SNDBUF=46080=>50000, buffer=100ms
[2021-08-30 09:41:45.026][Trace][11][v3h3p70e] start play smi=0ms, mw_sleep=100, mw_msgs=0, realtime=1, tcp_nodelay=1
[2021-08-30 09:41:46.968][Trace][11][7k5ts90s] Hybrid cpu=3.00%,15MB, cid=3,1, timer=62,0,0, clock=0,45,2,1,1,1,1,0,0, objs=(pkt:0,raw:0,fua:0,msg:21,oth:0,buf:0)
[2021-08-30 09:41:51.971][Trace][11][7k5ts90s] Hybrid cpu=1.00%,15MB, cid=3,1, timer=62,0,0, clock=0,45,2,1,1,1,1,0,0, objs=(pkt:0,raw:0,fua:0,msg:21,oth:0,buf:0)
[2021-08-30 09:41:54.263][Trace][11][7073iy8q] TCP: before dispose resource(RtmpConn)(0x28d3190), conns=3, zombies=0, ign=0, inz=0, ind=0
[2021-08-30 09:41:54.263][Warn][11][7073iy8q][104] client disconnect peer. ret=1007
[2021-08-30 09:41:54.263][Trace][11][32223m50] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0
[2021-08-30 09:41:54.263][Trace][11][7073iy8q] TCP: disposing #0 resource(RtmpConn)(0x28d3190), conns=3, disposing=1, zombies=0
[2021-08-30 09:41:54.708][Trace][11][41i2w1qj] process pid=36 terminate, please restart it.
[2021-08-30 09:41:56.971][Trace][11][7k5ts90s] Hybrid cpu=2.00%,15MB, cid=2,1, timer=61,0,0, clock=0,42,5,1,1,0,0,1,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:120,oth:0,buf:0)
[2021-08-30 09:41:57.715][Trace][11][5144281r] RTMP client ip=127.0.0.1:42344, fd=15
[2021-08-30 09:41:57.716][Trace][11][5144281r] complex handshake success
[2021-08-30 09:41:57.719][Trace][11][41i2w1qj] fored process, pid=37, bin=./objs/ffmpeg/bin/ffmpeg, stdout=./objs/ffmpeg-encoder-__defaultVhost__-live-PkdiDbzcHJnN-h265.log, stderr=./objs/ffmpeg-encoder-__defaultVhost__-live-PkdiDbzcHJnN-h265.log, argv=./objs/ffmpeg/bin/ffmpeg -f flv -i rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__/PkdiDbzcHJnN -vcodec libx264 -threads 12 -profile:v main -preset medium -acodec  -f flv -y rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__/PkdiDbzcHJnN_h265
[2021-08-30 09:41:57.755][Trace][11][5144281r] connect app, tcUrl=rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, args=null
[2021-08-30 09:41:57.756][Trace][11][5144281r] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128
[2021-08-30 09:41:57.836][Trace][11][5144281r] ignore AMF0/AMF3 command message.
[2021-08-30 09:41:57.875][Trace][11][5144281r] ignore AMF0/AMF3 command message.
[2021-08-30 09:41:57.875][Trace][11][5144281r] client identified, type=rtmp-play, vhost=__defaultVhost__, app=live, stream=PkdiDbzcHJnN, param=, duration=-1ms
[2021-08-30 09:41:57.875][Trace][11][5144281r] connected stream, tcUrl=rtmp://127.0.0.1:1935/live?vhost=__defaultVhost__, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, stream=PkdiDbzcHJnN, param=, args=null
[2021-08-30 09:41:57.875][Trace][11][5144281r] source url=/live/PkdiDbzcHJnN, ip=127.0.0.1, cache=0, is_edge=0, source_id=41i2w1qj/bhk472ek
[2021-08-30 09:41:57.876][Trace][11][5144281r] set fd=15 TCP_NODELAY 0=>1
[2021-08-30 09:41:57.876][Trace][11][5144281r] dispatch cached gop success. count=0, duration=10
[2021-08-30 09:41:57.876][Trace][11][5144281r] create consumer, active=1, queue_size=0.00, jitter=10000000
[2021-08-30 09:41:57.876][Trace][11][5144281r] set fd=15, SO_SNDBUF=2626560=>50000, buffer=100ms
[2021-08-30 09:41:57.876][Trace][11][5144281r] start play smi=0ms, mw_sleep=100, mw_msgs=0, realtime=1, tcp_nodelay=1
[2021-08-30 09:41:59.803][Trace][11][41i2w1qj] -> HLS time=539926370ms, sno=28, ts=PkdiDbzcHJnN/1630316513804.ts, dur=0.00, dva=5996p
[2021-08-30 09:42:00.056][Trace][11][6t7912z5] HTTP #0 127.0.0.1:53554 GET http://localhost:1985/api/v1/streams/, content-length=-1
[2021-08-30 09:42:00.057][Trace][11][6t7912z5] TCP: before dispose resource(HttpConn)(0x28697f0), conns=4, zombies=0, ign=0, inz=0, ind=0
[2021-08-30 09:42:00.057][Trace][11][6t7912z5] client finished.
[2021-08-30 09:42:00.057][Trace][11][32223m50] TCP: clear zombies=1 resources, conns=4, removing=0, unsubs=0
[2021-08-30 09:42:00.057][Trace][11][6t7912z5] TCP: disposing #0 resource(HttpConn)(0x28697f0), conns=4, disposing=1, zombies=0
[2021-08-30 09:42:00.722][Trace][11][41i2w1qj] -> ENC time=540827381, encoders=1, input=__defaultVhost__/live/PkdiDbzcHJnN
[2021-08-30 09:42:01.972][Trace][11][7k5ts90s] Hybrid cpu=1.00%,15MB, cid=2,1, timer=61,0,0, clock=0,42,5,1,1,0,0,1,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:120,oth:0,buf:0)
[2021-08-30 09:42:02.932][Trace][11][5144281r] -> PLA time=4910942, msgs=6, okbps=1764,0,0, ikbps=5,0,0, mw=100/0
[2021-08-30 09:42:04.695][Trace][11][41i2w1qj] <- CPB time=19995278, okbps=1,0,0, ikbps=2446,0,0, mr=0/350, p1stpt=20000, pnt=5000
[2021-08-30 09:42:06.972][Trace][11][7k5ts90s] Hybrid cpu=2.00%,15MB, cid=2,1, timer=61,0,0, clock=0,42,5,1,1,0,0,1,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:120,oth:0,buf:0)
[2021-08-30 09:42:11.973][Trace][11][7k5ts90s] Hybrid cpu=2.00%,15MB, cid=2,1, timer=61,0,0, clock=0,41,5,1,1,1,1,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:126,oth:0,buf:0)
[2021-08-30 09:42:14.697][Trace][11][41i2w1qj] <- CPB time=30009654, okbps=0,1,0, ikbps=2381,2358,0, mr=0/350, p1stpt=20000, pnt=5000
[2021-08-30 09:42:15.106][Trace][11][5144281r] -> PLA time=17126509, msgs=4, okbps=2002,0,0, ikbps=1,0,0, mw=100/0
[2021-08-30 09:42:15.417][Trace][11][v3h3p70e] TCP: before dispose resource(RtmpConn)(0x2868f20), conns=3, zombies=0, ign=0, inz=0, ind=0
[2021-08-30 09:42:15.417][Warn][11][v3h3p70e][104] client disconnect peer. ret=1007
[2021-08-30 09:42:15.417][Trace][11][32223m50] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0
[2021-08-30 09:42:15.417][Trace][11][v3h3p70e] TCP: disposing #0 resource(RtmpConn)(0x2868f20), conns=3, disposing=1, zombies=0
[2021-08-30 09:42:16.974][Trace][11][7k5ts90s] Hybrid cpu=2.00%,15MB, cid=2,1, timer=61,0,0, clock=0,41,5,1,1,1,1,0,0, free=1, objs=(pkt:0,raw:0,fua:0,msg:126,oth:0,buf:0)
  3. SRS的配置如下(Config): 
    
listen              1935;
max_connections     1000;
daemon              off;
srs_log_tank        console;

grace_start_wait 2300; grace_final_wait 3200; force_grace_quit on; inotify_auto_reload on;

http_api { enabled on; listen 1985; }

http_server { enabled on; listen 8080; }

vhost defaultVhost { min_latency on; tcp_nodelay on;

publish {
    mr off;
}

hls {
    enabled         on;
    hls_fragment    6;
    hls_window      30;
    hls_path        ./objs/nginx/html;
    hls_m3u8_file   [app]/[stream].m3u8;
    hls_ts_file     [app]/[stream]/[timestamp].ts;
    hls_cleanup     on;
    hls_dispose     180;
    hls_wait_keyframe       on;
}

 play {
    gop_cache       off;
    queue_length    10;
    mw_latency      100;
}

http_remux {
    enabled     on;
    mount       [vhost]/[app]/[stream].flv;
    hstrs       on;
}

transcode live {
    enabled     on;
    ffmpeg      ./objs/ffmpeg/bin/ffmpeg;
    engine h265{
        enabled     on;
        vcodec          libx264;
        vthreads        12;
        vprofile        main;
        vpreset         medium;
        output          rtmp://127.0.0.1:[port]/[app]?vhost=[vhost]/[stream]_[engine];
    }
}

}



**重现(Replay)**

> 重现Bug的步骤(How to replay bug?)

1. 推 h265 编码格式视频流
2. 观看转码流：rtmp://192.168.1.170:1935/live/livestream_h265

**期望行为(Expect)**

> 期望 FFmpeg 转码功能能支持对 h265 的转码
> 
> 在 originCluster 模式部署下，edge 端能够按需对 H265 的视频流进行转码