stack buffer overflow is found in TSDemuxer::simpleDemuxBlock()

Our fuzzer found stack buffer overflow in tsDemuxer. in the current master(94cafe7). PoC is here.

#include "bufferedReaderManager.h"
#include "vod_common.h"
#include "abstractDemuxer.h"
#include "tsDemuxer.h"
#include <cstdint>
#include <fs/systemlog.h>

using namespace std;

BufferedReaderManager readManager(2, DEFAULT_FILE_BLOCK_SIZE, DEFAULT_FILE_BLOCK_SIZE + MAX_AV_PACKET_SIZE,
                                  DEFAULT_FILE_BLOCK_SIZE / 2);

int main(int argc, char* argv[]) {
    string fileName = argv[1];
    AbstractDemuxer* demuxer = new TSDemuxer(readManager, "");

    uint32_t fileBlockSize = demuxer->getFileBlockSize();
    demuxer->openFile(fileName);
    int64_t discardedSize = 0;
    DemuxedData demuxedData;
    map<int32_t, TrackInfo> acceptedPidMap;
    demuxer->getTrackList(acceptedPidMap);

    PIDSet acceptedPidSet;
    demuxer->simpleDemuxBlock(demuxedData, acceptedPidSet, discardedSize);

    return 0;
}

Following is an output of ASAN. vuln21.ts is in poc21.zip

$ tsmuxer ./crash/vuln21.ts
=================================================================
==13362==ERROR: AddressSanitizer: negative-size-param: (size=-43)
    #0 0x7f7a112163ff in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x55d3556adfe0 in TSDemuxer::simpleDemuxBlock(std::map<int, MemoryBlock, std::less<int>, std::allocator<std::pair<int const, MemoryBlock> > >&, std::set<int, std::less<int>, std::allocator<int> > const&, long&) (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x4f1fe0)
    #2 0x55d355520030 in main (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x364030)
    #3 0x7f7a10b88d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #4 0x7f7a10b88e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #5 0x55d3554350b4 in _start (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x2790b4)

0x7f7a0cb01f8b is located 87947 bytes inside of 2129920-byte region [0x7f7a0caec800,0x7f7a0ccf4800)
allocated by thread T0 here:
    #0 0x7f7a11292357 in operator new[](unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:102
    #1 0x55d355485163 in ReaderData::init() (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x2c9163)
    #2 0x55d3554851ff in ReaderData::openStream() (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x2c91ff)
    #3 0x55d355484510 in FileReaderData::openStream() (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x2c8510)
    #4 0x55d35548473c in BufferedFileReader::openStream(int, char const*, int, CodecInfo const*) (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x2c873c)
    #5 0x55d3556afd2c in TSDemuxer::openFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x4f3d2c)
    #6 0x55d35551fe87 in main (/home/vagrant/tsmuxer/for_build/build/tsMuxer/tsmuxer+0x363e87)
    #7 0x7f7a10b88d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: negative-size-param ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827 in __interceptor_memcpy
==13362==ABORTING

It is caused by these line. https://github.com/justdan96/tsMuxer/blob/94cafe7244213870aaab37035c827ef839a15929/tsMuxer/tsDemuxer.cpp#L295-L296

ASAN says negative size param, but it is actually stack buffer overflow. While declaring pmtBuffer as 4096 bytes as the destination Buffer in TSDemuxer::simpleDemuxBlock, it does not check if pmtBufferLen + TS_FRAME_SIZE - tsPacket->getHeaderSize() used in memcpy is smaller than 4096, so stack based BOF occurs.

We can confirm that the return address of main is actually destroyed by the stack BOF in this POC as well.

Ricerca Security, Inc.

justdan96 / tsMuxer

stack buffer overflow is found in TSDemuxer::simpleDemuxBlock() #861