LAVPixFmtConverter.cpp error shift left

[GoogleCodeExporter]

Describe the issue you're having:
an MP4 video crashes in some context

How can the issue be reproduced? Sample File?

What version of the product are you using? In which Player?

Please provide any additional information below.

There is an error in  decoder/LAVVideo/LAVPixFmtConverter.cpp
Line 419 you shift left << 3  so you multiply by 8, but in reality you should 
shift right >> 8 to divide by 8 , 
that causes a crash in some cases because av_malloc return bad pointer, it 
could not allocate the buffer, I change it to shift right >>  3, and it works
HRESULT CLAVPixFmtConverter::Convert(LAVFrame *pFrame, uint8_t *dst, int width, 
int height, int dstStride , int planeHeight) {
  uint8_t *out = dst;
  int outStride = dstStride, i;
  planeHeight = max(height, planeHeight);
  // Check if we have proper pixel alignment and the dst memory is actually aligned
  if (m_RequiredAlignment && (FFALIGN(dstStride, m_RequiredAlignment) != dstStride || ((uintptr_t)dst % 16u))) {
    outStride = FFALIGN(dstStride, m_RequiredAlignment);
    size_t requiredSize = (outStride * planeHeight * lav_pixfmt_desc[m_OutputPixFmt].bpp) << 3; //  here should be >> 3
    if (requiredSize > m_nAlignedBufferSize) {
      DbgLog((LOG_TRACE, 10, L"::Convert(): Conversion requires a bigger stride (need: %d, have: %d), allocating buffer...", outStride, dstStride));
      av_freep(&m_pAlignedBuffer);
      m_nAlignedBufferSize = requiredSize;
      m_pAlignedBuffer = (uint8_t *)av_malloc(m_nAlignedBufferSize+FF_INPUT_BUFFER_PADDING_SIZE);
    }
    out = m_pAlignedBuffer;
  }

Original issue reported on code.google.com by phatw...@gmail.com on 5 Feb 2014 at 3:04

[GoogleCodeExporter]

Multiplying by 8 makes the buffer too big for your memory?
What kind of videos are triggering that?

Original comment by h.lepp...@gmail.com on 5 Feb 2014 at 7:33

[GoogleCodeExporter]

This issue was closed by revision d1fe5a5a3e50.

Original comment by h.lepp...@gmail.com on 5 Feb 2014 at 9:14

• Changed state: Fixed