ts转hls关键帧的问题

alexliyu7352 commented 2 years ago

最近我在开发ts转hls功能时遇到了一些问题. 因为您的hls库, ts转hls需要标注关键帧, 也就是IDR帧.

但是直接使用ts_demuxer_create回调的flags并不会有IDR. 所以只能靠判断NAL来判断是否是IDR帧.

我先使用了下方您的相关代码, 有些网络测试流可以判断出来, 但是遇到了一些测试时就没办法了.

static inline const uint8_t* h264_startcode(const uint8_t *data, size_t bytes)
{
    size_t i;
    for (i = 2; i + 1 < bytes; i++)
    {
        if (0x01 == data[i] && 0x00 == data[i - 1] && 0x00 == data[i - 2])
            return data + i + 1;
    }

    return NULL;
}

static inline uint8_t h264_idr(const uint8_t *data, size_t bytes)
{
    uint8_t naltype;
    const uint8_t *p;

    do
    {
        p = h264_startcode(data, bytes);
        if (p)
        {
            naltype = p[0] & 0x1f;
            // 1: no-IDR slice
            // 2: A-slice
            // 3: B-slice
            // 4: C-slice
            // 5: IDR frame
            if (naltype > 0 && naltype < 6)
            {
                return 5 == naltype ? 1 : 0;
            }

            bytes -= p - data;
            data = p;
        }
    } while (p);

    return 0;
}

static inline int h265_irap(const uint8_t* p, size_t bytes)
{
    size_t i;
    uint8_t type;
    for (i = 2; i + 1 < bytes; i++)
    {
        if (0x01 == p[i] && 0x00 == p[i - 1] && 0x00 == p[i - 2])
        {
            type = (p[i + 1] >> 1) & 0x3f;
            if (type < 32)
                return (16 <= type && type <= 23) ? 1 : 0;
        }
    }

    return 0;
}

后来我又使用了zlm的判断方法

H264_TYPE(v) ((uint8_t)(v) & 0x1F)
auto nal_ptr = (uint8_t *) this->data() + this->prefixSize();
auto type = H264_TYPE(*nal_ptr);

但是仍旧获得的type没有IDR类型的, 但是使用ffprobe获取, 是显示有关键帧存在的.

我网上搜索看到说, h264不能简单的按照nal来判断是否是关键帧.

我应该怎么做才能确切的获取关键帧呢?

alexliyu7352 commented 2 years ago

测试视频流, 我发送给您邮件了

alexliyu7352 commented 2 years ago

是否,我需要集成类似 https://github.com/Piasy/VideoCodecAnatomy/tree/master/webrtc/common_video/h264 https://github.com/SoonyangZhang/h264-parser 这种的第三方库专门parse nal?才可以? 还是生成hls, 不需要关键帧也可以?

ireader commented 2 years ago

ffmpeg对关键帧判断，除了跟进nal type外，还有一些其它措施。

ireader commented 2 years ago

HLS本身并不依赖关键帧。

如果暂时无法获取准确的关键帧信息，可以在hls_media_input函数的flags参数增加个force标志位，强制切片。

alexliyu7352 commented 2 years ago

强制切片会不会造成切片分布不均? 因为获取不到关键帧, 所以我自己写了一个hls切片逻辑. 我尝试使用dts来, 但是发现效果很不好.有些切片大有些小, 播放容易花屏.

大佬能否在libmpeg中提供一个类似ffmpeg的关键帧判断?

alexliyu7352 commented 2 years ago

我使用dts来计算, 每个切片10秒中. 那么计算dts如果累计到了10秒就强制切片了. 但是我发现效果不是很理想

ireader commented 2 years ago

我试试能不能参考ffmpeg实现关键帧判断逻辑

alexliyu7352 commented 2 years ago

大佬不愧是大佬!大气!

我真的特喜欢您这个库, 比ffmpeg简洁易用, 而且性能也棒.

就是关键帧这个判断基本上就没有判断成功过.

太感谢大佬了.

alexliyu7352 commented 2 years ago

大佬, 有好消息了嘛

ireader commented 2 years ago

先做个简单的方案，解析SEI，如果sei类型为6，就判断为关键帧。

do
{
  int v, sei = 0, len = 0;
  do
  {
      v = *p++;
      sei += v;
  } while (v == 0xFF);

  do
  {
      v = *p++;
      len += v;
  } while (v == 0xFF);

  switch (sei)
  {
  case H264_SEI_RECOVERY_POINT:
      // keyframe
      break;
  default:
      break;
  }

  p += len;
} while (h264_more_rbsp_data(stream));

alexliyu7352 commented 2 years ago

nal为5是idr, sei不是nal为6时的么?

还有h264_more_rbsp_data没有在您的库中发现

alexliyu7352 commented 2 years ago

大佬, 这个怎么用呢?

Chen @.***> 于2022年3月8日周二 08:49写道：

先做个简单的方案，解析SEI，如果sei类型为6，就判断为关键帧。

do

{

int v, sei = 0, len = 0;

do

{
v = *p++;

sei += v;
} while (v == 0xFF);

do

{
v = *p++;

len += v;
} while (v == 0xFF);

switch (sei)

{

case H264_SEI_RECOVERY_POINT:
// keyframe

break;
default:
break;
}

p += len;

} while (h264_more_rbsp_data(stream));

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1061293221, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRLHV26WT7COFODSJ6LU62P3BANCNFSM5QB6VYWA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

alexliyu7352 commented 2 years ago

大佬, 怎么样了?

ireader commented 2 years ago

h264关键帧判断代码提交到了 h264-parser.h(ireader/avcodec libh264 )

大致使用方式：

h264_parser_create()
对mpeg-ts解析出来的每一帧调用h264_parser_input, 然后调用h264_parser_iskeyframe判断是否关键帧

alexliyu7352 commented 2 years ago

好的, 感谢大神, 我试试看

Chen @.***> 于2022年3月13日周日 22:49写道：

h264关键帧判断代码提交到了 h264-parser.h(ireader/avcodec libh264 )

大致使用方式：

h264_parser_create()

对mpeg-ts解析出来的每一帧调用h264_parser_input, 然后调用h264_parser_iskeyframe判断是否关键帧

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1066117732, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRIJCIO3PZ37PWV4D33U7X555ANCNFSM5QB6VYWA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

alexliyu7352 commented 2 years ago

的确已经可以识别出来关键帧了, 能否和libmpeg库整合在一起?

因为您的mpegts库本身也有判断的方法, 是否可以整合在一起?这样其他人也可以使用.

int mpeg_h264_find_keyframe(const uint8_t* p, size_t bytes) { size_t i; uint8_t type; for (i = 2; i + 1 < bytes; i++) { if (0x01 == p[i] && 0x00 == p[i - 1] && 0x00 == p[i - 2]) { type = p[i + 1] & 0x1f; if (H264_NAL_IDR >= type && 1 <= type) return H264_NAL_IDR == type ? 1 : 0; } }

return 0; }

Message ID: @.***>

alexliyu7352 commented 2 years ago

大佬, 发现一个问题. 程序运行一段时间后, 发现一个线程把cpu一个核心跑满了. gdb检查堆栈如下, 发现是这个新的parse造成的.

Thread 43 (Thread 0x7f87607ff700 (LWP 6260)):
#0  0x00005558181762b8 in bitstream_read_bits ()
#1  0x0000555818174f65 in h264_sei ()
#2  0x00005558181743db in h264_parser_nal ()
#3  0x0000555818175e06 in h264_stream ()
#4  0x0000555818174697 in h264_parser_input ()
#5  0x00005558180fc557 in mediakit::TSDecoder::TSDecoder()::{lambda(void*, int, int, int, int, long, long, void const*, unsigned long)#2}::_FUN(void*, int, int, int, int, long, long, void const*, unsigned long) ()
#6  0x0000555818172173 in pes_packet ()
#7  0x000055581816fb40 in ts_demuxer_input ()

这块的代码很简单, 搞不懂为何这样, _h264_parser_t = h264_parser_create();难道要每个包都创建一次么?

_ts_segment.setOnSegment([this](const char *data, size_t len){
        ts_demuxer_input(_demuxer_ctx,(uint8_t*)data,len);
    });
    _h264_parser_t = h264_parser_create();
    _demuxer_ctx = ts_demuxer_create([](void* param, int program, int stream, int codecid, int flags, int64_t pts, int64_t dts, const void* data, size_t bytes){
        TSDecoder *thiz = (TSDecoder*)param;
        if(thiz->_on_decode){
            if(flags & MPEG_FLAG_PACKET_CORRUPT) {
                WarnL << "ts packet lost, dts:" << dts << " pts:" << pts << " bytes:" << bytes;
            }else{
                if(codecid == PSI_STREAM_H264) {
                    h264_parser_input((struct h264_parser_t *) thiz->_h264_parser_t, data, bytes);
                    flags = h264_parser_iskeyframe((struct h264_parser_t *) thiz->_h264_parser_t) ? MPEG_FLAG_IDR_FRAME : 0;
                }
                thiz->_on_decode(stream, codecid, flags, pts, dts, data, bytes);
            }
        }
        return 0;
    },this);
    ts_demuxer_notify_t notify = {
            [](void *param, int stream, int codecid, const void *extra, int bytes, int finish) {
                TSDecoder *thiz = (TSDecoder *) param;
                if (thiz->_on_stream) {
                    thiz->_on_stream(stream, codecid, extra, bytes, finish);
                }
            }
    };
    ts_demuxer_set_notify((struct ts_demuxer_t *) _demuxer_ctx, &notify, this);

alexliyu7352 commented 2 years ago

这里死循环了, 应该是卡在这里了.

int h264_sei(bitstream_t* stream, struct h264_context_t* h264)
{
    uint8_t v;
    uint32_t n, t;
    size_t off, bits;
    struct h264_sei_t* sei;
    sei = &h264->sei;

    do
    {
        t = 0;
        do
        {
            v = (uint8_t)bitstream_read_bits(stream, 8);
            t += v;
        } while (v == 0xFF);

        n = 0;
        do
        {
            v = (uint8_t)bitstream_read_bits(stream, 8);
            n += v;
        } while (v == 0xFF);

        bitstream_get_offset(stream, &off, &bits);
        switch (t)
        {
        case H264_SEI_BUFFERING_PERIOD:
            h264_sei_buffering_period(stream, h264, &sei->buffering_period);
            break;

        case H264_SEI_PIC_TIMING:
            h264_sei_pic_timing(stream, h264, &sei->pic_timing);
            break;

        case H264_SEI_RECOVERY_POINT:
            h264_sei_recovery_point(stream, &sei->recovery_point);
            break;

        default:
            break;
        }

        // restore
        bitstream_set_offset(stream, off, bits);
        while(n > 0)
        {
            bitstream_read_bits(stream, 8);
            n--;
        }
    } while (h264_more_rbsp_data(stream));

    h264_rbsp_trailing_bits(stream);
    return 0;
}

ireader commented 2 years ago

这里是新加的代码，可能是会有问题。

从gdb把sei二进制导出来我看看。

alexliyu7352 commented 2 years ago

gdb怎么导出sei的二进制? 这个还真没有尝试过

alexliyu7352 commented 2 years ago

而且在生产服务器上使用这个, 使用前cpu差不多在340%左右浮动. 使用后cpu飙升到450%左右.

其中除了死循环造成两个cpu核心使用变成100%以外, 其余的cpu差不多多了5%左右的使用率.

Chen @.***> 于2022年3月21日周一 09:02写道：

这里是新加的代码，可能是会有问题。

从gdb把sei二进制导出来我看看。

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1073394818, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPROIEGHBSL34WJ4MQM3VA7DA3ANCNFSM5QB6VYWA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

ireader commented 2 years ago

gdb怎么导出sei的二进制? 这个还真没有尝试过

用x命令可以看二进制数据。

alexliyu7352 commented 2 years ago

搞不了了, 之前我已经重新编译发步到生产服务器上了. 用以前出问题时dump出来的core文件, 没办法用了.

Chen @.***> 于2022年3月21日周一 14:35写道：

gdb怎么导出sei的二进制? 这个还真没有尝试过

用x命令可以看二进制数据。

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1073525229, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRLHF2GXZQ44SGN6IT3VBAKEHANCNFSM5QB6VYWA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

ireader commented 2 years ago

生成coredump文件了吗？

alexliyu7352 commented 2 years ago

大佬, 怎么办?

alex lee @.***> 于2022年3月21日周一 15:57写道：

只有之前出问题版本的core文件,但是貌似我重新编译后, 因为代码已经修改了. gdb也没办法读取具体的对应堆栈信息了.

现在怎么办?我只能恢复代码然后在部署在生产服务器上?

Chen @.***> 于2022年3月21日周一 15:28写道：

生成coredump文件了吗？

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1073552156, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRIMDNXRC5GQYFYCAM3VBAQLVANCNFSM5QB6VYWA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

ireader commented 2 years ago

libh264中， h264-rbsp.c::h264_more_rbsp_data函数实现有bug，参照如下修改：

int h264_more_rbsp_data(bitstream_t* stream)
{
    int rbsp_next_bits;
    size_t bytes, bits, n;

    bitstream_get_offset(stream, &bytes, &bits);
    //if (bytes + 1 >= stream->bytes && bits + 1 >= 8)
    if (bytes * 8 + bits >= stream->bytes * 8)
        return 0; // no more data

    n = bits < 8 ? 8 - bits : 8;
    rbsp_next_bits = bitstream_next_bits(stream, (int)n);
    return rbsp_next_bits != (1 << (n - 1));
}

alexliyu7352 commented 2 years ago

有些流这里会触发断言 assert(stream && bits > 0 && bits <= 32);

bits==0 以下是堆栈信息

[MediaServer] bitstream_read_bits bitstream.c:92
[MediaServer] h264_sei_pic_timing h264-sei.c:99
[MediaServer] h264_sei h264-sei.c:145
[MediaServer] h264_parser_nal h264-parser.c:108
[MediaServer] h264_stream h264-stream.c:31
[MediaServer] h264_parser_input h264-parser.c:143
[MediaServer] mediakit::TSDecoder::<lambda>::operator()(void *, int, int, int, int, int64_t, int64_t, const void *, size_t) const TSDecoder.cpp:82
[MediaServer] mediakit::TSDecoder::<lambda>::_FUN(void *, int, int, int, int, int64_t, int64_t, const void *, size_t) TSDecoder.cpp:89
[MediaServer] mpeg_packet_h264_h265_filter mpeg-packet.c:62
[MediaServer] mpeg_packet_h264_h265 mpeg-packet.c:94
[MediaServer] pes_packet mpeg-packet.c:131
[MediaServer] ts_demuxer_input mpeg-ts-dec.c:257
[MediaServer] mediakit::TSDecoder::<lambda>::operator()(const char *, size_t) const TSDecoder.cpp:72
[MediaServer] std::_Function_handler<void (const char *, long unsigned int), <lambda> >::_M_invoke(const std::_Any_data &, const char *&&, unsigned long &&) std_function.h:297
[MediaServer] std::function<void (const char *, unsigned long)>::operator()(const char *, unsigned long) const std_function.h:687
[MediaServer] mediakit::TSSegment::onRecvHeader TSDecoder.cpp:27
[MediaServer] mediakit::HttpRequestSplitter::input HttpRequestSplitter.cpp:65
[MediaServer] mediakit::TSDecoder::input TSDecoder.cpp:111
[MediaServer] mediakit::DecoderImp::input Decoder.cpp:66
[MediaServer] mediakit::TsPlayerImp::onResponseBody TsplayerImp.cpp:41
[MediaServer] mediakit::HttpClient::onRecvContent HttpClient.cpp:267
[MediaServer] mediakit::HttpRequestSplitter::input HttpRequestSplitter.cpp:115
[MediaServer] mediakit::HttpClient::onRecv HttpClient.cpp:197
[MediaServer] toolkit::TcpClientWithSSL<mediakit::HttpClient>::onRecv TcpClient.h:111
[MediaServer] toolkit::TcpClient::<lambda>::operator()(const toolkit::Buffer::Ptr &, sockaddr *, int) const TcpClient.cpp:119
[MediaServer] std::_Function_handler<void (const std::shared_ptr<toolkit::Buffer> &, sockaddr *, int), <lambda> >::_M_invoke(const std::_Any_data &, const std::shared_ptr<toolkit::Buffer> &, sockaddr *&&, int &&) std_function.h:297
[MediaServer] std::function<void (const std::shared_ptr<toolkit::Buffer> &, sockaddr *, int)>::operator()(const std::shared_ptr<toolkit::Buffer> &, sockaddr *, int) const std_function.h:687
[MediaServer] toolkit::Socket::onRead Socket.cpp:302
[MediaServer] toolkit::Socket::<lambda>::operator()(int) const Socket.cpp:248
[MediaServer] std::_Function_handler<void (int), <lambda> >::_M_invoke(const std::_Any_data &, int &&) std_function.h:297
[MediaServer] std::function<void (int)>::operator()(int) const std_function.h:687
[MediaServer] toolkit::EventPoller::runLoop EventPoller.cpp:297
[MediaServer] std::__invoke_impl<void, void (toolkit::EventPoller::*)(bool, bool), toolkit::EventPoller *, bool, bool> invoke.h:73
[MediaServer] std::__invoke<void (toolkit::EventPoller::*)(bool, bool), toolkit::EventPoller *, bool, bool> invoke.h:95
[MediaServer] std::thread::_Invoker<std::tuple<void (toolkit::EventPoller::*)(bool, bool), toolkit::EventPoller *, bool, bool> >::_M_invoke<0ul, 1ul, 2ul, 3ul> thread:244
[MediaServer] std::thread::_Invoker<std::tuple<void (toolkit::EventPoller::*)(bool, bool), toolkit::EventPoller *, bool, bool> >::operator() thread:253
[MediaServer] std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (toolkit::EventPoller::*)(bool, bool), toolkit::EventPoller *, bool, bool> > >::_M_run thread:196
[libstdc++.so.6] std::execute_native_thread_routine 0x00007f2f4e4d26df
[libpthread.so.0] start_thread 0x00007f2f4f1966db
[libc.so.6] clone 0x00007f2f4df2d61f

ireader commented 2 years ago

32位改成64位吧

assert(stream && bits > 0 && bits <= 64);

alexliyu7352 commented 2 years ago

不用改bits =>0么? 我看bits等于0

ireader commented 2 years ago

可以

alexliyu7352 commented 2 years ago

今天出现崩溃了.直接服务器挂了

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./media_server -d'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055a2ca6c6be3 in h264_pps ()
[Current thread is 1 (Thread 0x7f31553ff700 (LWP 21470))]
(gdb) bt full
#0  0x000055a2ca6c6be3 in h264_pps ()
No symbol table info available.
#1  0x000055a2ca6c6693 in h264_parser_nal ()
No symbol table info available.
#2  0x000055a2ca6c80a6 in h264_stream ()
No symbol table info available.
#3  0x000055a2ca6c6937 in h264_parser_input ()
No symbol table info available.
#4  0x000055a2ca64de84 in mediakit::TSDecoder::TSDecoder()::{lambda(void*, int, int, int, int, long, long, void const*, unsigned long)#2}::_FUN(void*, int, int, int, int, long, long, void const*, unsigned long) ()
No symbol table info available.
#5  0x000055a2ca6c4413 in pes_packet ()
No symbol table info available.
#6  0x000055a2ca6c1de0 in ts_demuxer_input ()
No symbol table info available.
#7  0x000055a2ca64e14b in mediakit::TSSegment::onRecvHeader(char const*, unsigned long) ()
No symbol table info available.
#8  0x000055a2ca4db3e1 in mediakit::HttpRequestSplitter::input(char const*, unsigned long) ()
No symbol table info available.
#9  0x000055a2ca64e2e6 in mediakit::TSDecoder::input(unsigned char const*, unsigned long) ()
No symbol table info available.
#10 0x000055a2ca6461b1 in mediakit::TsPlayerImp::onResponseBody(char const*, unsigned long) ()
No symbol table info available.
#11 0x000055a2ca4c5b26 in mediakit::HttpClient::onRecvContent(char const*, unsigned long) ()
No symbol table info available.
#12 0x000055a2ca4db53a in mediakit::HttpRequestSplitter::input(char const*, unsigned long) ()
No symbol table info available.
#13 0x000055a2ca661487 in std::_Function_handler<void (std::shared_ptr<toolkit::Buffer> const&, sockaddr*, int), toolkit::TcpClient::onSockConnect(toolkit::SockException const&)::{lambda(std::shared_ptr<toolkit::Buffer> const&, sockaddr*, int)#2}>::_M_invoke(std::_Any_data const&, std::shared_ptr<toolkit::Buffer> const&, sockaddr*&&, int&&) ()
No symbol table info available.
#14 0x000055a2ca659cbb in toolkit::Socket::onRead(std::shared_ptr<toolkit::SockFD> const&, bool) ()
No symbol table info available.
#15 0x000055a2ca65ba49 in std::_Function_handler<void (int), toolkit::Socket::attachEvent(std::shared_ptr<toolkit::SockFD> const&, bool)::{lambda(int)#1}>::_M_invoke(std::_Any_data const&, int&&) ()
No symbol table info available.
#16 0x000055a2ca6774ef in toolkit::EventPoller::runLoop(bool, bool) ()
No symbol table info available.
#17 0x00007f316444c6df in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#18 0x00007f31651106db in start_thread (arg=0x7f31553ff700) at pthread_create.c:463
        pd = 0x7f31553ff700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139849860380416, 4525315895594878524, 139849860378496, 0, 139850092062128, 140727962407904, -4562273229676444100, 
                -4562167353108298180}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#19 0x00007f3163ea771f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
(gdb) x 0x000055a2ca6c6be3
0x55a2ca6c6be3 <h264_pps+643>:  0x448a0489
(gdb) x 0x000055a2ca6c6be3;
Invalid character ';' in expression.
(gdb) x
0x55a2ca6c6be7 <h264_pps+647>:  0x77186339

alexliyu7352 commented 2 years ago

表现为占用了几十GB的内存, oom被系统杀了. 代码内存泄露了?

ireader commented 2 years ago

内存泄露跑了多长时间， h264_parser_create/h264_parser_destroy成对调用了吗？

alexliyu7352 commented 2 years ago

2022-03-27 10:49:37.276 I media_server[31765-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 10:49:37.302 I media_server[31765-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-27 18:15:49.324 I media_server[21455-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 18:15:49.354 I media_server[21455-media_server] main.cpp:330 start_main | 已启动http hook 接口

看日志差不多8个小时, 服务器72gb的内存.

Chen @.***> 于2022年3月28日周一 09:22写道：

内存泄露跑了多长时间， h264_parser_create/h264_parser_destroy成对调用了吗？

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1080084424, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRNUGOZ7Z4ISKQKIW6DVCECWBANCNFSM5QB6VYWA . You are receiving this because you authored the thread.Message ID: @.***>

alexliyu7352 commented 2 years ago

2022-03-25 11:22:18.875 I media_server[1254-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-25 11:22:18.910 I media_server[1254-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-26 14:40:11.608 I media_server[17473-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-26 14:40:11.654 I media_server[17473-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-26 21:18:27.428 I media_server[27823-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-26 21:18:27.462 I media_server[27823-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-27 10:49:37.276 I media_server[31765-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 10:49:37.302 I media_server[31765-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-27 18:15:49.324 I media_server[21455-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 18:15:49.354 I media_server[21455-media_server] main.cpp:330 start_main | 已启动http hook 接口

不一定, 看更早的, 是持续了一天然后进程才被杀死

alex lee @.***> 于2022年3月28日周一 10:17写道：

2022-03-27 10:49:37.276 I media_server[31765-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 10:49:37.302 I media_server[31765-media_server] main.cpp:330 start_main | 已启动http hook 接口 2022-03-27 18:15:49.324 I media_server[21455-media_server] main.cpp:328 start_main | 已启动http api 接口 2022-03-27 18:15:49.354 I media_server[21455-media_server] main.cpp:330 start_main | 已启动http hook 接口

看日志差不多8个小时, 服务器72gb的内存.

Chen @.***> 于2022年3月28日周一 09:22写道：

内存泄露跑了多长时间， h264_parser_create/h264_parser_destroy成对调用了吗？

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1080084424, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRNUGOZ7Z4ISKQKIW6DVCECWBANCNFSM5QB6VYWA . You are receiving this because you authored the thread.Message ID: @.***>

alexliyu7352 commented 2 years ago

内存泄露跑了多长时间， h264_parser_create/h264_parser_destroy成对调用了吗？

是的, 成对调用.

在一个类的构造函数调用create, 析构函数调用destory

alexliyu7352 commented 2 years ago

另外还有问题,在 ts_demuxer_create(ts_demuxer_onpacket onpacket, void* param) 的ts_demuxer_onpacket回调中检查发现flag是MPEG_FLAG_PACKET_CORRUPT后会连续很多个包都是MPEG_FLAG_PACKET_CORRUPT这个flag.

ireader commented 2 years ago

MPEG_FLAG_PACKET_CORRUPT表示丢包了，连续出现多个MPEG_FLAG_PACKET_CORRUPT的情况应该是真的丢包了。
MPEG_FLAG_PACKET_CORRUPT后，可能会出现多个MPEG_FLAG_PACKET_LOST，直到下一个关键帧。

alexliyu7352 commented 2 years ago

当出现MPEG_FLAG_PACKET_CORRUPT后, 直接把这个包抛掉是吧?

Chen @.***> 于2022年3月28日周一 16:43写道：

1. MPEG_FLAG_PACKET_CORRUPT表示丢包了，连续出现多个MPEG_FLAG_PACKET_CORRUPT的情况应该是真的丢包了。

MPEG_FLAG_PACKET_CORRUPT后，可能会出现多个MPEG_FLAG_PACKET_LOST，直到下一个关键帧。

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1080363634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRM2NXTI76QQ5NOTRVTVCFWLBANCNFSM5QB6VYWA . You are receiving this because you authored the thread.Message ID: @.***>

ireader commented 2 years ago

MPEG_FLAG_PACKET_CORRUPT 包一般直接丢弃。

MPEG_FLAG_PACKET_LOST表示之前发生过丢包，如果当前包是关键帧，可以忽略MPEG_FLAG_PACKET_LOST标记。如果当前帧是视频并且非关键帧，需要根据参考关系判断是否需要丢弃。

alexliyu7352 commented 2 years ago

好的明白了, 大佬是否有找到泄漏的原因? 我觉得应该是一个bug导致的.

目前从早上到现在运行了10多个小时了, 暂时没有看到内存明显上升.

但是之前就是这样突然一下, 一个cpu核心开始爆增, 然后内存开始上升.

Chen @.***> 于2022年3月28日周一 21:36写道：

MPEG_FLAG_PACKET_CORRUPT 包一般直接丢弃。

MPEG_FLAG_PACKET_LOST表示之前发生过丢包，如果当前包是关键帧，可以忽略MPEG_FLAG_PACKET_LOST标记。如果当前帧是视频并且非关键帧，需要根据参考关系判断是否需要丢弃。

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1080660758, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRMYIVLV2IAKSP5X3U3VCGYVXANCNFSM5QB6VYWA . You are receiving this because you authored the thread.Message ID: @.***>

ireader commented 2 years ago

libh264 内部只有2处分配内存，暂时没有找到内存泄露地方。

alexliyu7352 commented 2 years ago

那就很奇怪了, 我在跟踪看看吧. 主要是我测试找不到问题, 放到生产服务器上会崩. 但是生产服务器我又不能编译个debug版本上去.或者用valgrind之类的.

所以还有什么比较好的方法可以检测到内存泄露? 看崩溃后的coredump就在这个h264的库上

Chen @.***> 于2022年3月28日周一 21:55写道：

libh264 内部只有2处分配内存，暂时没有找到内存泄露地方。

— Reply to this email directly, view it on GitHub https://github.com/ireader/media-server/issues/204#issuecomment-1080684254, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3RPRNRQFCDH5SLGU2E6C3VCG243ANCNFSM5QB6VYWA . You are receiving this because you authored the thread.Message ID: @.***>

alexliyu7352 commented 2 years ago

大佬, 又卡住了, cpu核心使用100%, 而且开始大量内存溢出.

(gdb) thread 5
[Switching to thread 5 (Thread 0x7fd951bff700 (LWP 26339))]
#0  0x000055caf0937700 in bitstream_read_bits ()
(gdb) bt full
#0  0x000055caf0937700 in bitstream_read_bits ()
No symbol table info available.
#1  0x000055caf09363f5 in h264_sei ()
No symbol table info available.
#2  0x000055caf093584b in h264_parser_nal ()
No symbol table info available.
#3  0x000055caf0937276 in h264_stream ()
No symbol table info available.
#4  0x000055caf0935b07 in h264_parser_input ()
No symbol table info available.
#5  0x000055caf08bc2f0 in mediakit::TSDecoder::TSDecoder()::{lambda(void*, int, int, int, int, long, long, void const*, unsigned long)#2}::_FUN(void*, int, int, int, int, long, long, void const*, unsigned long) ()
No symbol table info available.
#6  0x000055caf09335e3 in pes_packet ()
No symbol table info available.
#7  0x000055caf0930fb0 in ts_demuxer_input ()
No symbol table info available.
#8  0x000055caf08bc63b in mediakit::TSSegment::onRecvHeader(char const*, unsigned long) ()
No symbol table info available.
#9  0x000055caf0740451 in mediakit::HttpRequestSplitter::input(char const*, unsigned long) ()
No symbol table info available.
#10 0x000055caf08bc7d6 in mediakit::TSDecoder::input(unsigned char const*, unsigned long) ()
No symbol table info available.
#11 0x000055caf08b51d7 in mediakit::TsPlayerImp::onResponseBody(char const*, unsigned long) ()
No symbol table info available.
#12 0x000055caf072ab96 in mediakit::HttpClient::onRecvContent(char const*, unsigned long) ()
No symbol table info available.
#13 0x000055caf07405aa in mediakit::HttpRequestSplitter::input(char const*, unsigned long) ()
No symbol table info available.
#14 0x000055caf08d05e7 in std::_Function_handler<void (std::shared_ptr<toolkit::Buffer> const&, sockaddr*, int), toolkit::TcpClient::onSockConnect(toolkit::SockException const&)::{lambda(std::shared_ptr<toolkit::Buffer> const&, sockaddr*, int)#2}>::_M_invoke(std::_Any_data const&, std::shared_ptr<toolkit::Buffer> const&, sockaddr*&&, int&&) ()
No symbol table info available.
#15 0x000055caf08c8e1b in toolkit::Socket::onRead(std::shared_ptr<toolkit::SockFD> const&, bool) ()
No symbol table info available.
#16 0x000055caf08caba9 in std::_Function_handler<void (int), toolkit::Socket::attachEvent(std::shared_ptr<toolkit::SockFD> const&, bool)::{lambda(int)#1}>::_M_invoke(std::_Any_data const&, int&&) ()
No symbol table info available.
#17 0x000055caf08e666f in toolkit::EventPoller::runLoop(bool, bool) ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#18 0x00007fd955b636df in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#19 0x00007fd9568276db in start_thread (arg=0x7fd951bff700) at pthread_create.c:463
        pd = 0x7fd951bff700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140571356165888, -1374445095592880553, 140571356163968, 0, 140571402476848, 140722385437616, 1395656324668757591, 
                1395641390803490391}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#20 0x00007fd9555be71f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

使用x获得的是

(gdb) x/3uh 0x000055caf0937674
0x55caf0937674 <bitstream_read_bit+4>:  35660   2119    14668
(gdb) x 0x000055caf0937674
0x55caf0937674 <bitstream_read_bit+4>:  35660

alexliyu7352 commented 2 years ago

其他还需要什么数据???我现在先不杀掉进程, 您还需要我提供什么?我现在提供

alexliyu7352 commented 2 years ago

(gdb) x/500uh 0x000055caf0937700
0x55caf0937700 <bitstream_read_bits>:   21825   21569   21333   33608   2284    63109   17022   35137
0x55caf0937710 <bitstream_read_bits+16>:    18932   64905   56113   60721   7951    132 0   0
0x55caf0937720 <bitstream_read_bits+32>:    35148   495 33755   453 17384   65535   2559    16835
0x55caf0937730 <bitstream_read_bits+48>:    60473   1406    63619   30207   18663   50307   35080   23512
0x55caf0937740 <bitstream_read_bits+64>:    16733   16732   50013   11878   7951    132 0   0
0x55caf0937750 <bitstream_read_bits+80>:    33608   2244    56113   55433   23899   23617   23873   37059
0x55caf0937760 <bitstream_next_bits>:   21569   35137   22004   18515   64393   33608   12524   35656
0x55caf0937770 <bitstream_next_bits+16>:    2135    35656   25655   35656   9476    40  0   35144
0x55caf0937780 <bitstream_next_bits+32>:    9284    12584   18624   58761   35144   59631   65072   65535
0x55caf0937790 <bitstream_next_bits+48>:    4083    17263   17424   59017   35144   4079    17449   4132
0x55caf09377a0 <bitstream_next_bits+64>:    23528   65535   18687   19595   10276   18532   3123    10277
0x55caf09377b0 <bitstream_next_bits+80>:    0   29952   18441   50307   23344   16733   50012   19944
0x55caf09377c0 <bitstream_next_bits+96>:    52156   4095    31  11878   7951    132 0   0
0x55caf09377d0 <bitstream_read_ue>: 18517   64905   47955   65535   65535   33608   2284    36966
0x55caf09377e0 <bitstream_read_ue+16>:  35144   33775   451 34280   65534   34303   29888   12785
0x55caf09377f0 <bitstream_read_ue+32>:  34240   29915   35082   18654   61321   488 65535   35327
0x55caf0937800 <bitstream_read_ue+48>:  47833   1   0   33608   2244    58067   23899   17549
0x55caf0937810 <bitstream_read_ue+64>:  65296   4035    31  11878   7951    132 0   0
0x55caf0937820 <bitstream_read_se>: 33608   2284    42984   65535   35327   33730   482 64131
0x55caf0937830 <bitstream_read_se+16>:  6401    33737   448 33608   2244    49801   51587   49409
0x55caf0937840 <bitstream_read_se+32>:  8170    53249   63697   44815   50113   3942    17439   0
0x55caf0937850 <bitstream_read_me>: 35157   21493   54153   33608   2284    29160   65535   18687
0x55caf0937860 <bitstream_read_me+16>:  34200   29933   34099   30171   18455   5517    47760   116
0x55caf0937870 <bitstream_read_me+32>:  1163    18562   50307   23304   50013   3942    17439   0
0x55caf0937880 <bitstream_read_me+48>:  36168   14613   29883   35584   33284   33608   2244    23899
0x55caf0937890 <bitstream_read_me+64>:  4035    32799   0   0   56197   5236    36168   7445
0x55caf09378a0 <bitstream_read_me+80>:  29882   35584   33284   33608   2244    23899   4035    31
0x55caf09378b0 <bitstream_read_me+96>:  36168   51477   29881   35584   33284   33608   2244    23899
0x55caf09378c0 <bitstream_read_me+112>: 4035    17439   0   11878   7951    132 0   0
0x55caf09378d0 <bitstream_read_te>: 18515   64393   63464   65534   33791   504 628 50011
0x55caf09378e0 <bitstream_read_te+16>:  35144   59615   64904   65535   34139   4032    49300   46607
0x55caf09378f0 <bitstream_read_te+32>:  50112   11878   7951    132 0   0   7951    64
0x55caf0937900 <h264_hrd>:  21825   48449   31  0   21569   35145   22012   60721
0x55caf0937910 <h264_hrd+16>:   18515   62345   33608   2284    46056   65534   48895   4
0x55caf0937920 <h264_hrd+32>:   0   35148   35047   59395   64980   65535   1214    0
0x55caf0937930 <h264_hrd+48>:   19456   59273   49801   46607   33923   1   33536   4066
0x55caf0937940 <h264_hrd+64>:   57475   2544    35024   33923   1   59392   64944   65535
0x55caf0937950 <h264_hrd+80>:   49801   46607   33923   1   49408   1250    57475   2319
0x55caf0937960 <h264_hrd+96>:   35024   33923   1   26112   7951    132 0   0
0x55caf0937970 <h264_hrd+112>:  35148   59623   65112   65535   35148   35303   43844   59396
---Type <return> to continue, or q <return> to quit---
0x55caf0937980 <h264_hrd+128>:  65100   65535   35148   35303   43908   132 0   56808
0x55caf0937990 <h264_hrd+144>:  65532   35327   43908   260 0   46607   18435   50563
0x55caf09379a0 <h264_hrd+160>:  15361   16671   18191   16581   59448   50291   35148   48871
0x55caf09379b0 <h264_hrd+176>:  5   0   18408   65533   19711   59273   1470    0
0x55caf09379c0 <h264_hrd+192>:  35072   4034    33718   389 0   57987   33567   57568
0x55caf09379d0 <h264_hrd+208>:  53257   33672   389 0   9192    65533   19711   59273
0x55caf09379e0 <h264_hrd+224>:  1470    0   33536   8160    57537   35085   35778   33923
0x55caf09379f0 <h264_hrd+240>:  1   9472    8191    65532   53257   33673   388 0
0x55caf0937a00 <h264_hrd+256>:  64488   65532   19711   59273   1470    0   33536   8160
0x55caf0937a10 <h264_hrd+272>:  5261    133 0   3840    33718   390 0   57475
0x55caf0937a20 <h264_hrd+288>:  2435    35024   34435   1   59392   64722   65535   57475
0x55caf0937a30 <h264_hrd+304>:  49439   2016    49801   46863   34435   1   26112   32549
0x55caf0937a40 <h264_hrd+320>:  2544    26320   33673   390 0   33608   2244    49201
0x55caf0937a50 <h264_hrd+336>:  23899   23617   23873   26307   7951    132 0   0
0x55caf0937a60 <h264_nal>:  18517   64905   18515   62345   33608   2284    1735    0
0x55caf0937a70 <h264_nal+16>:   0   63976   65531   48895   2   0   35144   35311
0x55caf0937a80 <h264_nal+32>:   4034    950 57987   33537   65248   53257   904 28136
0x55caf0937a90 <h264_nal+48>:   65532   48895   5   0   35144   33775   992 5261
0x55caf0937aa0 <h264_nal+64>:   3840    950 57475   2553    35024   59395   64592   65535
0x55caf0937ab0 <h264_nal+80>:   5261    197 0   3840    950 57475   2311    35280
0x55caf0937ac0 <h264_nal+96>:   35010   33539   61664   57987   15608   29856   32773   28922
0x55caf0937ad0 <h264_nal+112>:  7797    35144   33007   43258   9844    37352   65531   35327
0x55caf0937ae0 <h264_nal+128>:  4034    17334   33537   482

ireader commented 2 years ago

直接生成 coredump 文件

ireader commented 2 years ago

加下qq：349256071

alexliyu7352 commented 2 years ago

需要生成dump文件么? 我是生产服务器, dump文件非常大, 差不多将近30gb,

ireader commented 2 years ago

生成dump文件，然后gdb 调试这个dump文件。 fr 4
x/256b annexb

ireader / media-server

ts转hls关键帧的问题 #204