search

justdan96 / tsMuxer

tsMuxer is a transport stream muxer for remuxing/muxing elementary streams, EVO/VOB/MPG, MKV/MKA, MP4/MOV, TS, M2TS to TS to M2TS. Supported video codecs H.264/AVC, H.265/HEVC, VC-1, MPEG2. Supported audio codecs AAC, AC3 / E-AC3(DD+), DTS/ DTS-HD.
Apache License 2.0
860 stars 144 forks source link

overflow in BitStreamReader::getCurVal #477

Closed aug5t7 closed 2 years ago

aug5t7 commented 2 years ago

Hi, I found a crash error.

System info:

Ubuntu 20.04.3 LTS
tsMuxeR version git-c6a0277

To reproduce

  1. Compile tsMuxer
  2. Run tsmuxer 
    tsmuxer ./poc

    POC poc.zip

gdb output

gdb-peda$ r ./poc
Starting program: tsMuxer/tsMuxer/build/tsMuxer/tsmuxer ./poc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
tsMuxeR version git-c6a0277. github.com/justdan96/tsMuxer
This HEVC stream doesn't contain fps value. Muxing fps is absent too. Set muxing FPS to default 25.0 value.

Program received signal SIGSEGV, Segmentation fault.
[----------------------------------registers-----------------------------------]
RAX: 0x343031415550524d ('MRPUA104')
RBX: 0x5555559019a0 --> 0x5555558c3f60 --> 0x5555557f2ba2 (<_ZN10VvcVpsUnit11deserializeEv>:    endbr64)
RCX: 0x20 (' ')
RDX: 0x36356500 ('')
RSI: 0x343031415550524d ('MRPUA104')
RDI: 0x555555901a30 --> 0x4e464f3636356500 ('')
RBP: 0x7fffffff6ee0 --> 0x7fffffff6f20 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 (--> ...)
RSP: 0x7fffffff6eb0 ("MRPUA1040\032\220UUU")
RIP: 0x5555556bb6cf (<_ZN15BitStreamReader9getCurValEPj+55>:    mov    eax,DWORD PTR [rax])
R8 : 0x3
R9 : 0x1
R10: 0x22 ('"')
R11: 0x7ffff7b63be0 --> 0x555555908aa0 --> 0x0
R12: 0x5
R13: 0x7fffffffe310 --> 0x2
R14: 0x0
R15: 0x0
EFLAGS: 0x10216 (carry PARITY ADJUST zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x5555556bb6c6 <_ZN15BitStreamReader9getCurValEPj+46>:   cmp    eax,0x1f
   0x5555556bb6c9 <_ZN15BitStreamReader9getCurValEPj+49>:
    jbe    0x5555556bb6dd <_ZN15BitStreamReader9getCurValEPj+69>
   0x5555556bb6cb <_ZN15BitStreamReader9getCurValEPj+51>:   mov    rax,QWORD PTR [rbp-0x30]
=> 0x5555556bb6cf <_ZN15BitStreamReader9getCurValEPj+55>:   mov    eax,DWORD PTR [rax]
   0x5555556bb6d1 <_ZN15BitStreamReader9getCurValEPj+57>:   mov    edi,eax
   0x5555556bb6d3 <_ZN15BitStreamReader9getCurValEPj+59>:   call   0x555555808265 <_Z8my_ntohlj>
   0x5555556bb6d8 <_ZN15BitStreamReader9getCurValEPj+64>:
    jmp    0x5555556bb7b7 <_ZN15BitStreamReader9getCurValEPj+287>
   0x5555556bb6dd <_ZN15BitStreamReader9getCurValEPj+69>:   mov    rax,QWORD PTR [rbp-0x28]
[------------------------------------stack-------------------------------------]
0000| 0x7fffffff6eb0 ("MRPUA1040\032\220UUU")
0008| 0x7fffffff6eb8 --> 0x555555901a30 --> 0x4e464f3636356500 ('')
0016| 0x7fffffff6ec0 --> 0x7fffffff6ee0 --> 0x7fffffff6f20 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 (--> ...)
0024| 0x7fffffff6ec8 ("MRPUA104")
0032| 0x7fffffff6ed0 --> 0x0
0040| 0x7fffffff6ed8 --> 0x5555559019a0 --> 0x5555558c3f60 --> 0x5555557f2ba2 (<_ZN10VvcVpsUnit11deserializeEv>:    endbr64)
0048| 0x7fffffff6ee0 --> 0x7fffffff6f20 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 (--> ...)
0056| 0x7fffffff6ee8 --> 0x5555556bb932 (<_ZN15BitStreamReader7getBitsEj+270>:  mov    rdx,QWORD PTR [rbp-0x28])
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
0x00005555556bb6cf in BitStreamReader::getCurVal(unsigned int*) ()
gdb-peda$ bt
#0  0x00005555556bb6cf in BitStreamReader::getCurVal(unsigned int*) ()
#1  0x00005555556bb932 in BitStreamReader::getBits(unsigned int) ()
#2  0x00005555557f5f1f in VvcHrdUnit::general_timing_hrd_parameters() ()
#3  0x00005555557f345f in VvcVpsUnit::deserialize() ()
#4  0x00005555557fa143 in VVCStreamReader::checkStream(unsigned char*, int) ()
#5  0x0000555555742753 in METADemuxer::detectTrackReader(unsigned char*, int, AbstractStreamReader::ContainerType, int, int) ()
#6  0x0000555555741afb in METADemuxer::DetectStreamReader(BufferedReaderManager&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) ()
#7  0x000055555571ca8a in detectStreamReader(char const*, MPLSParser*, bool) ()
#8  0x000055555571fafc in main ()
#9  0x00007ffff799f0b3 in __libc_start_main (main=0x55555571ed30 <main>, argc=0x2,
    argv=0x7fffffffe318, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7fffffffe308) at ../csu/libc-start.c:308
#10 0x00005555556bac2e in _start ()
gdb-peda$
justdan96 commented 2 years ago

Thanks for the report - can you suggest a suitable code change to resolve this issue?

jcdr428 commented 2 years ago

@AugJujube I can't reproduce the issue. On my Windows64, tsMuxer exits normally with "Can't detect stream type".

jcdr428 commented 2 years ago

@aug5t7 can you please test with latest release, to see whether this error is still there.

aug5t7 commented 2 years ago

still crash with a segmentation fault on Linux...

$ ./tsmuxer poc
tsMuxeR version git-2448c36. github.com/justdan96/tsMuxer
This HEVC stream doesn't contain fps value. Muxing fps is absent too. Set muxing FPS to default 25.0 value.
[1]    4127918 segmentation fault  ./tsmuxer poc

I found that int rez = VvcUnit::deserialize(); in vvc.cpp:209 do not set the buffer of this->m_vps_hrd->m_reader, but VvcHrdUnit::general_timing_hrd_parameters->BitStreamReader::getBits invokes to m_vps_hrd->m_reader->m_buffer which cause a invalid memory access resulting crash.

I don’t know much about the code details. here's part of gdb output, hope this help.

gdb-peda$ r
Starting program: /path/to/tsMuxer/build/tsMuxer/tsmuxer ./poc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
tsMuxeR version git-2448c36. github.com/justdan96/tsMuxer
This HEVC stream doesn't contain fps value. Muxing fps is absent too. Set muxing FPS to default 25.0 value.
[----------------------------------registers-----------------------------------]
RAX: 0x0
RBX: 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
RCX: 0x7ffff393b042 --> 0x20c1000f0007143
RDX: 0x5555557f2d96 (<VvcVpsUnit::deserialize()>:   endbr64)
RSI: 0x7ffff393b042 --> 0x20c1000f0007143
RDI: 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
RBP: 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 --> 0x7fffffffd850 --> 0x7fffffffda30 --> 0x7fffffffe220 (--> ...)
RSP: 0x7fffffff6f60 --> 0x100000003
RIP: 0x5555557f2dbe (<VvcVpsUnit::deserialize()+40>:    mov    rax,QWORD PTR [rbp-0x118])
R8 : 0x5555558f7080 --> 0x20c1000f0007143
R9 : 0x0
R10: 0x22 ('"')
R11: 0x7ffff7b63be0 --> 0x555555909aa0 --> 0x0
R12: 0x5555556bac00 (<_start>:  endbr64)
R13: 0x7fffffffe310 --> 0x2
R14: 0x0
R15: 0x0
EFLAGS: 0x246 (carry PARITY adjust ZERO sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x5555557f2daf <VvcVpsUnit::deserialize()+25>:   mov    rax,QWORD PTR fs:0x28
   0x5555557f2db8 <VvcVpsUnit::deserialize()+34>:   mov    QWORD PTR [rbp-0x18],rax
   0x5555557f2dbc <VvcVpsUnit::deserialize()+38>:   xor    eax,eax
=> 0x5555557f2dbe <VvcVpsUnit::deserialize()+40>:   mov    rax,QWORD PTR [rbp-0x118]
   0x5555557f2dc5 <VvcVpsUnit::deserialize()+47>:   mov    rdi,rax
   0x5555557f2dc8 <VvcVpsUnit::deserialize()+50>:   call   0x5555557f1fc0 <VvcUnit::deserialize()>
   0x5555557f2dcd <VvcVpsUnit::deserialize()+55>:   mov    DWORD PTR [rbp-0xc4],eax
   0x5555557f2dd3 <VvcVpsUnit::deserialize()+61>:   cmp    DWORD PTR [rbp-0xc4],0x0
[------------------------------------stack-------------------------------------]
0000| 0x7fffffff6f60 --> 0x100000003
0008| 0x7fffffff6f68 --> 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
0016| 0x7fffffff6f70 --> 0x5555558f7070 --> 0x40 ('@')
0024| 0x7fffffff6f78 --> 0x7ffff7b63be0 --> 0x555555909aa0 --> 0x0
0032| 0x7fffffff6f80 --> 0x1
0040| 0x7fffffff6f88 --> 0x5555558f5320 --> 0x5555558c3bc8 --> 0x5555556fff20 (<HevcSpsUnit::deserialize()>:    endbr64)
0048| 0x7fffffff6f90 --> 0xc7
0056| 0x7fffffff6f98 --> 0x55550000000d ('\r')
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value

Breakpoint 2, VvcVpsUnit::deserialize (this=0x5555559029a0) at /path/to/tsMuxer/tsMuxer/vvc.cpp:209
209     int rez = VvcUnit::deserialize();
gdb-peda$ p * this
$22 = {
  <VvcUnitWithProfile> = {
    <VvcUnit> = {
      _vptr.VvcUnit = 0x5555558c4f60 <vtable for VvcVpsUnit+16>,
      nal_unit_type = 0x0,
      nuh_layer_id = 0x0,
      nuh_temporal_id_plus1 = 0x0,
      m_nalBuffer = 0x5555558f7080 "Cq",
      m_nalBufferLen = 0xc7,
      m_reader = {
        <BitStream> = {
          m_totalBits = 0x500052d,
          m_buffer = 0x4d8b2258f17e17a5,
          m_initBuffer = 0x40055f002410b1c,
          static m_maskInitialized = 0x1,
          static m_masks = {0x0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff,
            0x3fff, 0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0xffffff,
            0x1ffffff, 0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff}
        },
        members of BitStreamReader:
        m_curVal = 0x2f8eac62,
        m_bitLeft = 0xabf5be34
      }
    },
    members of VvcUnitWithProfile:
    profile_idc = 0x0,
    level_idc = 0x0
  },
  members of VvcVpsUnit:
  vps_id = 0x0,
  vps_max_layers = 0x0,
  vps_max_sublayers = 0x0,
  num_units_in_tick = 0x0,
  time_scale = 0x0,
  num_units_in_tick_bit_pos = 0xffffffff,
  m_vps_hrd = {
    <VvcUnit> = {
      _vptr.VvcUnit = 0x5555558c4fa8 <vtable for VvcHrdUnit+16>,
      nal_unit_type = 0x0,
      nuh_layer_id = 0x0,
      nuh_temporal_id_plus1 = 0x0,
      m_nalBuffer = 0x0,
      m_nalBufferLen = 0x0,
      m_reader = {
        <BitStream> = {
          m_totalBits = 0x36356500,
          m_buffer = 0x3430314155505249,
          m_initBuffer = 0xb2d747069726373,
          static m_maskInitialized = 0x1,
          static m_masks = {0x0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff,
            0x3fff, 0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0xffffff,
            0x1ffffff, 0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff}
        },
        members of BitStreamReader:
        m_curVal = 0x7,
        m_bitLeft = 0x0
      }
    },
    members of VvcHrdUnit:
    num_units_in_tick = 0x0,
    time_scale = 0x0,
    general_nal_hrd_params_present_flag = 0x0,
    general_vcl_hrd_params_present_flag = 0x0,
    general_du_hrd_params_present_flag = 0x0,
    hrd_cpb_cnt_minus1 = 0x0
  }
}
gdb-peda$ n
[----------------------------------registers-----------------------------------]
RAX: 0x0
RBX: 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
RCX: 0x4371 ('qC')
RDX: 0x1
RSI: 0x3
RDI: 0x5555559029c8 --> 0x6ef0313600000628
RBP: 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 --> 0x7fffffffd850 --> 0x7fffffffda30 --> 0x7fffffffe220 (--> ...)
RSP: 0x7fffffff6f60 --> 0x100000003
RIP: 0x5555557f2dd3 (<VvcVpsUnit::deserialize()+61>:    cmp    DWORD PTR [rbp-0xc4],0x0)
R8 : 0x5555558f7080 --> 0x20c1000f0007143
R9 : 0x0
R10: 0x22 ('"')
R11: 0x7ffff7b63be0 --> 0x555555909aa0 --> 0x0
R12: 0x5555556bac00 (<_start>:  endbr64)
R13: 0x7fffffffe310 --> 0x2
R14: 0x0
R15: 0x0
EFLAGS: 0x206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x5555557f2dc5 <VvcVpsUnit::deserialize()+47>:   mov    rdi,rax
   0x5555557f2dc8 <VvcVpsUnit::deserialize()+50>:   call   0x5555557f1fc0 <VvcUnit::deserialize()>
   0x5555557f2dcd <VvcVpsUnit::deserialize()+55>:   mov    DWORD PTR [rbp-0xc4],eax
=> 0x5555557f2dd3 <VvcVpsUnit::deserialize()+61>:   cmp    DWORD PTR [rbp-0xc4],0x0
   0x5555557f2dda <VvcVpsUnit::deserialize()+68>:   je     0x5555557f2de7 <VvcVpsUnit::deserialize()+81>
   0x5555557f2ddc <VvcVpsUnit::deserialize()+70>:   mov    ebx,DWORD PTR [rbp-0xc4]
   0x5555557f2de2 <VvcVpsUnit::deserialize()+76>:   jmp    0x5555557f37f3 <VvcVpsUnit::deserialize()+2653>
   0x5555557f2de7 <VvcVpsUnit::deserialize()+81>:   mov    rax,QWORD PTR [rbp-0x118]
[------------------------------------stack-------------------------------------]
0000| 0x7fffffff6f60 --> 0x100000003
0008| 0x7fffffff6f68 --> 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
0016| 0x7fffffff6f70 --> 0x5555558f7070 --> 0x40 ('@')
0024| 0x7fffffff6f78 --> 0x7ffff7b63be0 --> 0x555555909aa0 --> 0x0
0032| 0x7fffffff6f80 --> 0x1
0040| 0x7fffffff6f88 --> 0x5555558f5320 --> 0x5555558c3bc8 --> 0x5555556fff20 (<HevcSpsUnit::deserialize()>:    endbr64)
0048| 0x7fffffff6f90 --> 0xc7
0056| 0x7fffffff6f98 --> 0x55550000000d ('\r')
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
210     if (rez)
gdb-peda$ p * this
$23 = {
  <VvcUnitWithProfile> = {
    <VvcUnit> = {
      _vptr.VvcUnit = 0x5555558c4f60 <vtable for VvcVpsUnit+16>,
      nal_unit_type = 0xe,
      nuh_layer_id = 0x3,
      nuh_temporal_id_plus1 = 0x1,
      m_nalBuffer = 0x5555558f7080 "Cq",
      m_nalBufferLen = 0xc7,
      m_reader = {
        <BitStream> = {
          m_totalBits = 0x628,
          m_buffer = 0x5555558f7080,
          m_initBuffer = 0x5555558f7080,
          static m_maskInitialized = 0x1,
          static m_masks = {0x0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff,
            0x3fff, 0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0xffffff,
            0x1ffffff, 0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff}
        },
        members of BitStreamReader:
        m_curVal = 0x437100f0,
        m_bitLeft = 0x10
      }
    },
    members of VvcUnitWithProfile:
    profile_idc = 0x0,
    level_idc = 0x0
  },
  members of VvcVpsUnit:
  vps_id = 0x0,
  vps_max_layers = 0x0,
  vps_max_sublayers = 0x0,
  num_units_in_tick = 0x0,
  time_scale = 0x0,
  num_units_in_tick_bit_pos = 0xffffffff,
  m_vps_hrd = {
    <VvcUnit> = {
      _vptr.VvcUnit = 0x5555558c4fa8 <vtable for VvcHrdUnit+16>,
      nal_unit_type = 0x0,
      nuh_layer_id = 0x0,
      nuh_temporal_id_plus1 = 0x0,
      m_nalBuffer = 0x0,
      m_nalBufferLen = 0x0,
      m_reader = {
        <BitStream> = {
          m_totalBits = 0x36356500,
          m_buffer = 0x3430314155505249,
          m_initBuffer = 0xb2d747069726373,
          static m_maskInitialized = 0x1,
          static m_masks = {0x0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff,
            0x3fff, 0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0xffffff,
            0x1ffffff, 0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff}
        },
        members of BitStreamReader:
        m_curVal = 0x7,
        m_bitLeft = 0x0
      }
    },
    members of VvcHrdUnit:
    num_units_in_tick = 0x0,
    time_scale = 0x0,
    general_nal_hrd_params_present_flag = 0x0,
    general_vcl_hrd_params_present_flag = 0x0,
    general_du_hrd_params_present_flag = 0x0,
    hrd_cpb_cnt_minus1 = 0x0
  }
}
gdb-peda$ c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[----------------------------------registers-----------------------------------]
RAX: 0x343031415550524d ('MRPUA104')
RBX: 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
RCX: 0x20 (' ')
RDX: 0x36356500 ('')
RSI: 0x343031415550524d ('MRPUA104')
RDI: 0x555555902a30 --> 0x4e464f3636356500 ('')
RBP: 0x7fffffff6ee0 --> 0x7fffffff6f20 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 (--> ...)
RSP: 0x7fffffff6eb0 ("MRPUA1040*\220UUU")
RIP: 0x5555556bb6cf (<BitStreamReader::getCurVal(unsigned int*)+55>:    mov    eax,DWORD PTR [rax])
R8 : 0x3
R9 : 0x1
R10: 0x22 ('"')
R11: 0x7ffff7b63be0 --> 0x555555909aa0 --> 0x0
R12: 0x5
R13: 0x7fffffffe310 --> 0x2
R14: 0x0
R15: 0x0
EFLAGS: 0x10216 (carry PARITY ADJUST zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x5555556bb6c6 <BitStreamReader::getCurVal(unsigned int*)+46>:   cmp    eax,0x1f
   0x5555556bb6c9 <BitStreamReader::getCurVal(unsigned int*)+49>:
    jbe    0x5555556bb6dd <BitStreamReader::getCurVal(unsigned int*)+69>:       jbe    0x5555556bb6dd <BitStreamReader::getCurVal(unsigned int*)+69>
   0x5555556bb6cb <BitStreamReader::getCurVal(unsigned int*)+51>:   mov    rax,QWORD PTR [rbp-0x30]
=> 0x5555556bb6cf <BitStreamReader::getCurVal(unsigned int*)+55>:   mov    eax,DWORD PTR [rax]
   0x5555556bb6d1 <BitStreamReader::getCurVal(unsigned int*)+57>:   mov    edi,eax
   0x5555556bb6d3 <BitStreamReader::getCurVal(unsigned int*)+59>:
    call   0x555555807ae2 <my_ntohl(unsigned int)>
   0x5555556bb6d8 <BitStreamReader::getCurVal(unsigned int*)+64>:
    jmp    0x5555556bb7b7 <BitStreamReader::getCurVal(unsigned int*)+287>:      jmp    0x5555556bb7b7 <BitStreamReader::getCurVal(unsigned int*)+287>
   0x5555556bb6dd <BitStreamReader::getCurVal(unsigned int*)+69>:   mov    rax,QWORD PTR [rbp-0x28]
[------------------------------------stack-------------------------------------]
0000| 0x7fffffff6eb0 ("MRPUA1040*\220UUU")
0008| 0x7fffffff6eb8 --> 0x555555902a30 --> 0x4e464f3636356500 ('')
0016| 0x7fffffff6ec0 --> 0x0
0024| 0x7fffffff6ec8 ("MRPUA104Po\377\377\377\177")
0032| 0x7fffffff6ed0 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 --> 0x7fffffffd850 (--> ...)
0040| 0x7fffffff6ed8 --> 0x5555559029a0 --> 0x5555558c4f60 --> 0x5555557f2d96 (<VvcVpsUnit::deserialize()>: endbr64)
0048| 0x7fffffff6ee0 --> 0x7fffffff6f20 --> 0x7fffffff6f50 --> 0x7fffffff7080 --> 0x7fffffff71b0 --> 0x7fffffffd380 (--> ...)
0056| 0x7fffffff6ee8 --> 0x5555556bb932 (<BitStreamReader::getBits(unsigned int)+270>:  mov    rdx,QWORD PTR [rbp-0x28])
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
0x00005555556bb6cf in BitStreamReader::getCurVal (this=0x555555902a30, buff=0x343031415550524d)
    at /path/to/tsMuxer/tsMuxer/bitStream.h:56
56              return my_ntohl(*buff);
gdb-peda$ p * this
$24 = {
  <BitStream> = {
    m_totalBits = 0x36356500,
    m_buffer = 0x343031415550524d,
    m_initBuffer = 0xb2d747069726373,
    static m_maskInitialized = 0x1,
    static m_masks = {0x0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff, 0x3fff,
      0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0xffffff, 0x1ffffff,
      0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff}
  },
  members of BitStreamReader:
  m_curVal = 0x7,
  m_bitLeft = 0x0
}
gdb-peda$ backtrace
#0  0x00005555556bb6cf in BitStreamReader::getCurVal (this=0x555555902a30, buff=0x343031415550524d)
    at /path/to/tsMuxer/tsMuxer/bitStream.h:56
#1  0x00005555556bb932 in BitStreamReader::getBits (this=0x555555902a30, num=0x20)
    at /path/to/tsMuxer/tsMuxer/bitStream.h:88
#2  0x00005555557f613d in VvcHrdUnit::general_timing_hrd_parameters (this=0x555555902a08)
    at /path/to/tsMuxer/tsMuxer/vvc.cpp:910
#3  0x00005555557f3669 in VvcVpsUnit::deserialize (this=0x5555559029a0)
    at /path/to/tsMuxer/tsMuxer/vvc.cpp:347
#4  0x00005555557fa361 in VVCStreamReader::checkStream (this=0x7fffffff7630, buffer=0x7ffff393b010 "", len=0x13c)
    at /path/to/tsMuxer/tsMuxer/vvcStreamReader.cpp:62
#5  0x00005555557428ed in METADemuxer::detectTrackReader (tmpBuffer=0x7ffff393b010 "", len=0x13c,
    containerType=AbstractStreamReader::ctNone, containerDataType=0x0, containerStreamIndex=0x0)
    at /path/to/tsMuxer/tsMuxer/metaDemuxer.cpp:776
#6  0x0000555555741c95 in METADemuxer::DetectStreamReader (readManager=..., fileName="./poc",
    calcDuration=0x1) at /path/to/tsMuxer/tsMuxer/metaDemuxer.cpp:685
#7  0x000055555571cc24 in detectStreamReader (fileName=0x7fffffffe603 "./poc", mplsParser=0x0,
    isSubMode=0x0) at /path/to/tsMuxer/tsMuxer/main.cpp:120
#8  0x000055555571fc96 in main (argc=0x2, argv=0x7fffffffe318)
    at /path/to/tsMuxer/tsMuxer/main.cpp:699
#9  0x00007ffff799f0b3 in __libc_start_main (main=0x55555571eeca <main(int, char**)>, argc=0x2, argv=0x7fffffffe318,
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe308)
    at ../csu/libc-start.c:308
#10 0x00005555556bac2e in _start ()
gdb-peda$
lighterowl commented 2 years ago

Not reproducible with current HEAD - closing.