encoder/decode makes use of uninitialized memory

[GoogleCodeExporter]

The issue was first reported in a valgrind test: 

configure --enable-debug --disable-optimizations

valgrind --malloc-fill=AB --track-origins=yes --leak-check=full 
--show-reachable=yes --db-attach=yes ./test_libvpx 
--gtest_filter=VP9/BordersTest.TestEncodeHighBitrate/0

valgrind then reports a number of locations in the encoder code that values in 
uninitialized memory are used in branch decsions in RD decision process. 

Original issue reported on code.google.com by ya...@google.com on 23 Aug 2013 at 4:54

[GoogleCodeExporter]

see https://gerrit.chromium.org/gerrit/#/c/66677/ for proposed work around. 

Original comment by ya...@google.com on 23 Aug 2013 at 5:18

[GoogleCodeExporter]

further investigation has tracked down the using of uninitialized memory to the 
source buffer to be encoded as alt ref frame. Since all source buffer are 
copied and extended, only alt ref frame that is the result of ARNR temporal 
filtering is not extended. When the non-extended source buffer is used as 
encoding source and image size is not multiple of 64 in both dimension, the 
encoder rd process may use the uninitialized pixel values outside the valid 
image area 

Original comment by ya...@google.com on 23 Aug 2013 at 6:35

[GoogleCodeExporter]

Original comment by ya...@google.com on 23 Aug 2013 at 7:09

[GoogleCodeExporter]

as patch 6 of  https://gerrit.chromium.org/gerrit/#/c/66677/

The problem appeared to be fixed.

Original comment by ya...@google.com on 23 Aug 2013 at 7:32

• Changed state: Fixed