Vpxdec crashes when decoding a file encoded using vpxenc

[GoogleCodeExporter]

./vpxenc --limit=10 --codec=vp9 --ivf -o video.ivf deadline_cif.y4m 
--arnr-maxframes=7 --arnr-strength=5 --arnr-type=3 --auto-alt-ref=1 --passes=2 
--test-decode=fatal
Pass 1/2 frame   10/11      1672B    1337b/f   40087b/s   62280 us (160.57 fps)
Pass 2/2 frame   10/10     11262B    9009b/f  270017b/s   39331 ms (0.25 fps)

./vpxdec video.ivf -o /dev/null
Error: Read invalid frame size (4243144692)
Assertion failed: (data_sz), function parse_superframe_index, file 
../vp9/vp9_dx_iface.c, line 398.
Abort trap: 6

Experimental branch, 70dd5021536772e53e8f818cd3474d496adda5c5, 
--enable-oneshotq --enable-experimental.

(gdb) r video.ivf -o /dev/null 
Starting program: /Users/rbultje/Projects/libvpx/x86-64/vpxdec video.ivf -o 
/dev/null
Reading symbols for shared libraries +............................. done
Error: Read invalid frame size (4243144692)
Assertion failed: (data_sz), function parse_superframe_index, file 
../vp9/vp9_dx_iface.c, line 398.

Program received signal SIGABRT, Aborted.
0x00007fff93468d46 in __kill ()
(gdb) bt
#0  0x00007fff93468d46 in __kill ()
#1  0x00007fff93caddf0 in abort ()
#2  0x00007fff93caee2a in __assert_rtn ()
#3  0x00000001000ddc9b in parse_superframe_index (data=0x0, data_sz=0, 
sizes=0x7fff5fbfee8c, count=0x7fff5fbfee88) at vp9_dx_iface.c:398
#4  0x00000001000dde81 in vp9_decode (ctx=0x100203a90, data=0x0, data_sz=0, 
user_priv=0x0, deadline=0) at vp9_dx_iface.c:436
#5  0x000000010000c4a2 in vpx_codec_decode (ctx=0x7fff5fbff130, data=0x0, 
data_sz=0, user_priv=0x0, deadline=0) at vpx_decoder.c:127
#6  0x000000010000b531 in main ()
(gdb) 

Original issue reported on code.google.com by rsbul...@gmail.com on 12 Jun 2013 at 5:09

[GoogleCodeExporter]

Original comment by jimbankoski@google.com on 12 Jun 2013 at 9:18

[GoogleCodeExporter]

Original comment by ya...@google.com on 31 Jul 2013 at 5:12

• Changed state: Fixed

[GoogleCodeExporter]

This is still broken with: v1.2.0-3964-gbb07200

using:
--target-bitrate=1000 --ivf --kf-min-dist=0 --kf-max-dist=16 --limit=1 -D -v 
--i420 --codec=vp9 --stereo-mode=mono --best --end-usage=vbr --resize-allowed=0 
--passes=2 --auto-alt-ref=1 --lag-in-frames=16 --min-q=0 --max-q=63 --tune=psnr 
--psnr

with the bitrate likely forcing a large frame size.

WebM Project VP9 Decoder v1.2.0-3964-gbb07200
Error: Read invalid frame size (3349380801)
vpxdec: /home/jzern/external/libvpx/git/vp9/vp9_dx_iface.c:333: 
parse_superframe_index: Assertion `data_sz' failed.

Original comment by jz...@google.com on 14 Aug 2013 at 8:44

• Changed state: Assigned

[GoogleCodeExporter]

Original comment by jz...@google.com on 14 Aug 2013 at 8:45

[GoogleCodeExporter]

This was actually caused by the first frame's header being overwritten on file 
close.

https://gerrit.chromium.org/gerrit/65921 addresses this and
https://gerrit.chromium.org/gerrit/65920 avoids the assert/crash.

Original comment by jz...@google.com on 15 Aug 2013 at 1:37

[GoogleCodeExporter]

This issue was closed by revision 8cb09719a35a.

Original comment by jz...@google.com on 15 Aug 2013 at 6:07

• Changed state: Fixed

[GoogleCodeExporter]

These were both merged:
2039518 vp9_dx_iface: check for NULL/0-size input
8cb0971 vpxenc: open output file after setting pass #

Original comment by jz...@google.com on 15 Aug 2013 at 9:31