vp9 realtime encoder crash with visual studio 2013/2015 builds targeting win32

[GoogleCodeExporter]

v1.4.0-1020-g9f7fdcb

First noticed after:
e8103f3 Merge "Eliminate num_8x8 and num_4x4 width/height lookups"
13a4f14 Eliminate num_8x8 and num_4x4 width/height lookups
with
./test_libvpx --gtest_filter=*TestNoMisMatchAQ1/*

A closely related vpxenc command line [1] (minus the aq) will crash at speed 
3-5.
The bug only occurs in release builds, /O1 and /O2 with most other settings 
removed reproduce the problem in VS2013/2015 [2][3].
The divergence on the second frame stems from vp9_rdopt.c:block_rd_txfm() and 
the value stored to sse from bsse[]:

    } else {
      // SKIP_TXFM_AC_DC
      // skip forward transform
      x->plane[plane].eobs[block] = 0;
      sse  = x->bsse[(plane << 2) + (block >> (tx_size << 1))] << 4;
      dist = sse;
    }

---
      sse  = x->bsse[(plane << 2) + (block >> (tx_size << 1))] << 4;
012734D7  lea         eax,[esi+ebx*4]
012734DA  mov         esi,dword ptr [edi+eax*8+13F80h]  
012734E1  mov         eax,dword ptr [edi+eax*8+4]  
012734E5  shld        eax,esi,4  
012734E9  shl         esi,4  

---

The second mov should be 13F84h to load the top half of the 64-bit value. 
struct size changes/reordering will workaround the problem, but are of course 
fragile.

[1]
./vpxenc --ivf \
  --codec=vp9 --lag-in-frames=0 \
  hantro_collage_w352h288.yuv -w 352 -h 288 --fps=30/1 
  --cpu-used=3 --end-usage=cbr \
  --buf-sz=1000 --buf-optimal-sz=500 --buf-initial-sz=500 \
  --min-q=8 --max-q=56 --limit=2 --rt -o h.ivf
[2]
Microsoft Visual Studio Professional 2013
Version 12.0.21005.1 REL
[3]
Microsoft Visual Studio Professional 2015
Version 14.0.23107.0 D14REL

Original issue reported on code.google.com by jz...@google.com on 10 Aug 2015 at 7:34

[GoogleCodeExporter]

Last configure used while debugging this:

$ ./configure \
  --enable-debug --target=x86-win32-vs12 \
  --disable-libyuv --disable-webm-io \
  --disable-avx2 --disable-avx --disable-sse4_1 \
  --disable-ssse3 --disable-sse3 --disable-sse2 \
  --disable-sse --disable-mmx --disable-vp8 --disable-postproc

Original comment by jz...@google.com on 10 Aug 2015 at 7:37

[GoogleCodeExporter]

Original comment by ya...@google.com on 12 Aug 2015 at 10:04

• Changed state: Assigned

[GoogleCodeExporter]

This failure still occurs with:
Microsoft Visual Studio Professional 2013
Version 12.0.40629.00 Update 5

Original comment by jz...@google.com on 21 Aug 2015 at 5:53