[VP9] Encoder crash in lossless mode with auto-alt-ref enabled

[GoogleCodeExporter]

Setting --max-q=0 with default value of --auto-alt-ref leads to encoder crash.

Full exec line is the following:
vpxenc.exe  foreman_352x288_300.yuv --codec=vp9 -w 352 -h 288 -o foreman.vp9 -p 
1 --end-usage=q --cq-level=0  --good --fps=25000/1000 --ivf --limit=50 
--lossless=1 --max-q=0

Debugging showed that it happens in function temporal_filter_iterate_c() due to 
out-of-bounds access to cpi->fixed_divide array. This gave me a cue to set 
auto-alt-ref=0, and then I got able to encode the full stream.

Link to initial discussion:
https://groups.google.com/a/webmproject.org/forum/#!topic/webm-discuss/HphEa5Sl3
pA

There were two issue initially and one of them was fixed, so I wonder why 
--lossless=1 doesn't lead to crash, but after adding max-q parameter crash 
happens.

Windows 8, build of 21-Mar-2014 with all SIMD optimizations disabled.

Original issue reported on code.google.com by ru.xa...@gmail.com on 21 Mar 2014 at 9:10

[GoogleCodeExporter]

Tested with the exact same command line at #9e46dcd2, the encoding completed 
fine without any issue. 

Can you let us know the commit# that you had the problem as well as your 
configure/build info?

thanks

Original comment by ya...@google.com on 28 Mar 2014 at 10:16

• Changed state: Unconfirmed

[GoogleCodeExporter]

Checked this on today snapshot, still able to reproduce. 
Commit ID: 94e082993c07b705880aa4e36e782dd80af37610

And I prepared a patch (attached) containing only one assert pointing to the 
place where problem happens. 

Also I pointed that I disable SIMD optimization, but didn't described how I do 
this. My configure command params: 
../libvpx/configure --target=x86-win32-vs11 --enable-static-msvcrt 
--disable-vp8 --disable-docs --disable-runtime_cpu_detect --disable-mmx 
--disable-sse --disable-sse2 --disable-sse3 --disable-ssse3 --disable-sse4_1 
--disable-avx --disable-avx2 

Original comment by ru.xa...@gmail.com on 3 Apr 2014 at 9:44

Attachments:

• 0001-Assert-checking-cpi-fixed_divide-out-of-bounds-acces.patch

[GoogleCodeExporter]

Proposed fix here: 
https://gerrit.chromium.org/gerrit/#/c/69463/

Original comment by ya...@google.com on 3 Apr 2014 at 7:15

• Changed state: Assigned

[GoogleCodeExporter]

Original comment by ya...@google.com on 4 Apr 2014 at 4:20

• Changed state: Started

[GoogleCodeExporter]

Tested with current head, the issue is no longer reproducing, has been fixed. 

Original comment by ya...@google.com on 15 Jul 2014 at 7:07

• Changed state: Fixed

[GoogleCodeExporter]

Tested with current head, the issue is no longer reproducing, has been fixed. 

Original comment by ya...@google.com on 15 Jul 2014 at 7:07