search

ericmckean / webm

Automatically exported from code.google.com/p/webm
0 stars 0 forks source link

integer overflows with large frame sizes (4096x4096) #828

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
v1.3.0-3554-ga9b30b3

There are likely other issues above 16K, for now the visible issue is with 
VP9FrameSizeTestsLarge.

$ ./test_libvpx --gtest_filter=VP9FrameSizeTestsLarge
Note: Google Test filter = VP9FrameSizeTestsLarge.ValidSizes-:AVX2/*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from VP9FrameSizeTestsLarge
[ RUN      ] VP9FrameSizeTestsLarge.ValidSizes
../vp9/common/vp9_prob.h:59:45: runtime error: signed integer overflow: 
11468505 * 256 cannot be represented in type 'int'
../vp9/common/vp9_prob.h:59:45: runtime error: signed integer overflow: 
11468505 * 256 cannot be represented in type 'int'

Original issue reported on code.google.com by jz...@google.com on 22 Jul 2014 at 11:32

GoogleCodeExporter commented 9 years ago

Original comment by jz...@google.com on 22 Jul 2014 at 11:41

GoogleCodeExporter commented 9 years ago 
https://gerrit.chromium.org/gerrit/#/c/70992/

Original comment by jingn...@google.com on 25 Jul 2014 at 11:34