ossrs / srs

SRS is a simple, high-efficiency, real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181.
https://ossrs.io
MIT License
25.76k stars 5.39k forks source link

CLUSTER: Edge Kbps calling pure virtual function crashes #765

Closed pbesedm closed 7 years ago

pbesedm commented 7 years ago

CentOS release 6.7 (Final) x86_64 SRS-V2.0 b4 Accessing http-flv edge with VLC: http://x.x.x.x:xxxx/live/livestream.flv?token=xxxx&time=xxxxx can be played. In the case of only one VLC accessing the edge, if this user disconnects, the edge will output the log: origin disconnected, retry. ret=1007. At this time, the SRS edge program crashes and the terminal prints: pure virtual method called terminate called without an active exception Below is the call stack:

(gdb) bt
#0  0x00007f432c6195e5 in raise () from /lib64/libc.so.6
#1  0x00007f432c61adc5 in abort () from /lib64/libc.so.6
#2  0x00007f432ced3a7d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib64/libstdc++.so.6
#3  0x00007f432ced1bd6 in ?? () from /usr/lib64/libstdc++.so.6
#4  0x00007f432ced1c03 in std::terminate() () from /usr/lib64/libstdc++.so.6
#5  0x00007f432ced255f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
#6  0x0000000000475c17 in SrsKbps::set_io (this=0x1d55430, in=0x0, out=0x0)
    at src/protocol/srs_protocol_kbps.cpp:123
#7  0x00000000005136f3 in SrsEdgeIngester::stop (this=0x1d553d0) at src/app/srs_app_edge.cpp:115
#8  0x00000000005190ec in SrsPlayEdge::on_all_client_stop (this=0x1d55b80) at src/app/srs_app_edge.cpp:747
#9  0x0000000000499590 in SrsSource::on_consumer_destroy (this=0x1d558b0, consumer=0x1d5b730)
    at src/app/srs_app_source.cpp:2313
#10 0x00000000004907af in SrsConsumer::~SrsConsumer (this=0x1d5b730, __in_chrg=<value optimized out>)
    at src/app/srs_app_source.cpp:443
#11 0x00000000004908ac in SrsConsumer::~SrsConsumer (this=0x1d5b730, __in_chrg=<value optimized out>)
    at src/app/srs_app_source.cpp:450
#12 0x000000000048ecf2 in impl__SrsAutoFree<SrsConsumer>::~impl__SrsAutoFree (this=0x7f432d4d58d0, 
    __in_chrg=<value optimized out>) at src/core/srs_core_autofree.hpp:79
#13 0x00000000004b24b9 in SrsLiveStream::serve_http (this=0x1d5b2b0, w=0x7f432d4d5b40, r=0x1d52c70)
    at src/app/srs_app_http_stream.cpp:591
#14 0x000000000046f31d in SrsHttpServeMux::serve_http (this=0x1d122e0, w=0x7f432d4d5b40, r=0x1d52c70)
    at src/protocol/srs_http_stack.cpp:661
#15 0x00000000004f767c in SrsHttpServer::serve_http (this=0x1d122a0, w=0x7f432d4d5b40, r=0x1d52c70)
    at src/app/srs_app_http_conn.cpp:1426
#16 0x00000000004f688c in SrsHttpConn::process_request (this=0x1d32640, w=0x7f432d4d5b40, r=0x1d52c70)
    at src/app/srs_app_http_conn.cpp:1304
#17 0x00000000004f65ef in SrsHttpConn::do_cycle (this=0x1d32640) at src/app/srs_app_http_conn.cpp:1282
#18 0x00000000004858da in SrsConnection::cycle (this=0x1d32640) at src/app/srs_app_conn.cpp:89
#19 0x00000000004b83c9 in SrsOneCycleThread::cycle (this=0x1d326a0) at src/app/srs_app_thread.cpp:372
#20 0x00000000004b7b21 in internal::SrsThread::thread_cycle (this=0x1d326c0) at src/app/srs_app_thread.cpp:207
#21 0x00000000004b7d34 in internal::SrsThread::thread_fun (arg=0x1d326c0) at src/app/srs_app_thread.cpp:245
#22 0x0000000000545a36 in _st_thread_main () at sched.c:327
#23 0x00000000005461d0 in st_thread_create (start=0x54694a <st_usleep+202>, arg=0x7f432d50bb30, joinable=32579, 
    stk_size=759676464) at sched.c:591
#24 0x00000000004b7785 in internal::SrsThread::start (this=Cannot access memory at address 0x8ffffffd9
) at src/app/srs_app_thread.cpp:110

TRANS_BY_GPT3

winlinvip commented 7 years ago

Directly configuring FLV did not reproduce, let me check if there is a race condition in the code.

TRANS_BY_GPT3

winlinvip commented 7 years ago

http://www.kuqin.com/language/20100307/80980.html The behavior of a wild pointer is undefined, but from what I understand, we generally get an "access violation" rather than a "pure virtual function call". The situations we encounter in reality can be more complex than mentioned above: during the destruction process, a subclass object encounters an exception and is not completely destroyed, leaving behind a "defective" object. The program continues to use this defective object and calls a pure virtual function.

TRANS_BY_GPT3

liguanpei commented 7 years ago

SRS version: srs-2.0-b4

Origin

Source (192.168.4.172):

Distributor ID: Ubuntu
Description:    Ubuntu 12.04.5 LTS
Release:        12.04
Codename:       precise

Configuration (conf/http.flv.live.conf):

listen              1935;
max_connections     1000;
http_server {
    enabled         on;
    listen          8080;
    dir             ./objs/nginx/html;
}
vhost __defaultVhost__ {
    http_remux {
        enabled     on;
        mount       [vhost]/[app]/[stream].flv;
        hstrs       on;
    }
}

Edge

edge1 (192.168.1.128):

Distributor ID: Ubuntu
Description:    Ubuntu 10.04.4 LTS
Release:        10.04
Codename:       lucid

Configuration (conf/http.flv.live.edge1.conf)

listen              19351;
max_connections     1000;
pid                 objs/srs.flv.19351.pid;
srs_log_file        objs/srs.flv.19351.log;
http_server {
    enabled         on;
    listen          8080;
    dir             ./objs/nginx/html;
}
vhost __defaultVhost__ {
    mode remote;
    origin 192.168.4.172;
    http_remux {
        enabled     on;
        mount       [vhost]/[app]/[stream].flv;
        hstrs       on;
    }
}

Reproduction steps:

  1. Push OBS stream to origin.
  2. Play edge1 HTTP or RTMP stream with ffplay or VLC, it works fine.
  3. Close ffplay or VLC to disconnect the stream. At this point, SRS crashes on edge1. Log:
    [2017-02-25 11:48:16.659][trace][27708][108] complex handshake success.
    [2017-02-25 11:48:16.659][trace][27708][108] edge ingest from 192.168.4.172:1935 at rtmp://192.168.4.172/live
    [2017-02-25 11:48:16.737][trace][27708][108] connected, version=2.0.230, ip=192.168.4.172, pid=31970, id=124, dsu=1
    [2017-02-25 11:48:16.738][trace][27708][108] edge change from 100 to state 101 (pull).
    [2017-02-25 11:48:16.750][trace][27708][108] got metadata, width=1280, height=720
    [2017-02-25 11:48:16.750][trace][27708][108] protocol in.buffer=0, in.ack=2500000, out.ack=2500000, in.chunk=60000, out.chunk=60000
    [2017-02-25 11:48:16.750][trace][27708][108] 7B audio sh, codec(10, profile=LC, 2channels, 0kbps, 22050HZ), flv(16bits, 2channels, 22050HZ)
    [2017-02-25 11:48:16.751][trace][27708][108] 45B video sh,  codec(7, profile=High, level=3.1, 1280x720, 0kbps, 0fps, 0s)
    [2017-02-25 11:48:17.116][trace][27708][105] update source_id=108[108]
    [2017-02-25 11:48:19.666][warn][27708][108][4] origin disconnected, retry. ret=1007

Terminal output:

pure virtual method called
terminate called without an active exception

TRANS_BY_GPT3

winlinvip commented 7 years ago

Fixed in #752