strukturag / libde265

Open h.265 video codec implementation.
Other
1.7k stars 457 forks source link

undefined-behavior: left shift of 1073741826 by 2 places cannot be represented in type 'int' #289

Open hardik05 opened 3 years ago

hardik05 commented 3 years ago

While fuzzing, I found a issue in slice.c file slice.cc:2481:35: runtime error: left shift of 1073741826 by 2 places cannot be represented in type 'int' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior slice.cc:248

Following is the call stack:

0 decode_coeff_abs_level_remaining (tctx=0x7fffffff7de0, cRiceParam=0) at slice.cc:2481

1 0x00007ffff163b099 in residual_coding (tctx=0x7fffffff7de0, x0=0, y0=0, log2TrafoSize=4, cIdx=0)

at slice.cc:3349

2 0x00007ffff1640d30 in read_transform_unit (tctx=, x0=0, y0=0, xBase=0, yBase=0,

xCUBase=0, yCUBase=0, log2TrafoSize=4, trafoDepth=-33312, blkIdx=0, cbf_luma=1, cbf_cb=1, cbf_cr=0)
at slice.cc:3662

3 0x00007ffff164b3cd in read_transform_tree (tctx=, x0=0, y0=0, xBase=0, yBase=0,

xCUBase=0, yCUBase=<optimized out>, log2TrafoSize=<optimized out>, trafoDepth=<optimized out>,
blkIdx=0, MaxTrafoDepth=<optimized out>, IntraSplitFlag=<optimized out>,
cuPredMode=<optimized out>, parent_cbf_cb=1 '\001', parent_cbf_cr=0 '\000') at slice.cc:3963

4 0x00007ffff1659297 in read_coding_unit (tctx=0x7fffffff7de0, x0=0, y0=,

log2CbSize=4, ctDepth=<optimized out>) at slice.cc:4575

5 0x00007ffff1630b52 in read_coding_quadtree (tctx=0x7fffffff7de0, x0=, y0=0,

log2CbSize=<optimized out>, ctDepth=<optimized out>) at slice.cc:4652

can send POC over email if needed. Please let me know.

farindk commented 1 year ago

Is this still relevant? If yes, please attach a POC file.