DrMemory reports uninitialized reads in scale.cc

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. On Windows, setup a WebRTC checkout using this .gclient contents:
solutions = [
  { "name"        : "src",
    "url"         : "http://webrtc.googlecode.com/svn/trunk",
    "deps_file"   : "DEPS",
    "managed"     : True,
    "custom_vars" : {
      "root_dir": "src",
    },
    "safesync_url": "",
  },
  { "name":"drmemory.DEPS",
    "url":"svn://svn.chromium.org/chrome/trunk/deps/third_party/drmemory/drmemory.DEPS",
  },
]

2. Run:
set GYP_DEFINES=build_for_tool=drmemory
gclient sync
3. Compile:
ninja -C out\Debug modules_unittests
4. Run modules_unittests with Dr Memory (standing in the src/ dir):
tools\valgrind-webrtc\webrtc_tests.bat --build_dir out\Debug --test 
modules_unittests --tool drmemory_full 
--gtest_filter=VideoProcessingModuleTest.PreprocessorLogic*

What is the expected output? What do you see instead?
All tests should pass without errors. Dr Memory reports two UNINITIALIZED READ 
errors:

04:36:52 drmemory_analyze.py [INFO] Found 2 error reports
04:36:52 drmemory_analyze.py [INFO] Report #1
UNINITIALIZED READ: reading 0x0432b880-0x0432b890 16 byte(s) within 
0x0432b880-0x0432b890
#0 ScaleAddRows_SSE2                                                          
[third_party\libyuv\source\scale.cc:769]
#1 ScalePlaneDown                                                             
[third_party\libyuv\source\scale.cc:2229]
#2 ScalePlane                                                                 
[third_party\libyuv\source\scale.cc:2277]
#3 I420Scale                                                                  
[third_party\libyuv\source\scale.cc:2346]
#4 webrtc::Scaler::Scale                                                      
[webrtc\common_video\libyuv\scaler.cc:77]
#5 webrtc::VPMSimpleSpatialResampler::ResampleFrame                           
[webrtc\modules\video_processing\main\source\spatial_resampler.cc:86]
#6 webrtc::VPMFramePreprocessor::PreprocessFrame                              
[webrtc\modules\video_processing\main\source\frame_preprocessor.cc:159]
#7 webrtc::VideoProcessingModuleImpl::PreprocessFrame                         
[webrtc\modules\video_processing\main\source\video_processing_impl.c:275]
#8 webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody         
[webrtc\modules\video_processing\main\test\unit_test\video_processin:220]
#9 
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void> 
[testing\gtest\src\gtest.cc:1981]
Note: @0:52:08.826 in thread 3804
Note: instruction: data16 movdqa (%esi) -> %xmm0
Suppression (error hash=#AA3F5D4A49383B9F#):
  For more info on using suppressions see http://dev.chromium.org/developers/how-tos/using-drmemory#TOC-Suppressing-error-reports-from-the-
{
UNINITIALIZED READ
name=<insert_a_suppression_name_here>
*!ScaleAddRows_SSE2
*!ScalePlaneDown
*!ScalePlane
*!I420Scale
*!webrtc::Scaler::Scale
*!webrtc::VPMSimpleSpatialResampler::ResampleFrame
*!webrtc::VPMFramePreprocessor::PreprocessFrame
*!webrtc::VideoProcessingModuleImpl::PreprocessFrame
*!webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody
*!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>
}

04:36:52 drmemory_analyze.py [INFO] Report #2
UNINITIALIZED READ: reading 0x0432b9e0-0x0432b9f0 16 byte(s) within 
0x0432b9e0-0x0432b9f0
#0 ScaleAddRows_SSE2                                                          
[third_party\libyuv\source\scale.cc:782]
#1 ScalePlaneBox                                                              
[third_party\libyuv\source\scale.cc:2062]
#2 ScalePlaneDown                                                             
[third_party\libyuv\source\scale.cc:2229]
#3 ScalePlane                                                                 
[third_party\libyuv\source\scale.cc:2277]
#4 I420Scale                                                                  
[third_party\libyuv\source\scale.cc:2346]
#5 webrtc::Scaler::Scale                                                      
[webrtc\common_video\libyuv\scaler.cc:77]
#6 webrtc::VPMSimpleSpatialResampler::ResampleFrame                           
[webrtc\modules\video_processing\main\source\spatial_resampler.cc:86]
#7 webrtc::VPMFramePreprocessor::PreprocessFrame                              
[webrtc\modules\video_processing\main\source\frame_preprocessor.cc:159]
#8 webrtc::VideoProcessingModuleImpl::PreprocessFrame                         
[webrtc\modules\video_processing\main\source\video_processing_impl.c:275]
#9 webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody         
[webrtc\modules\video_processing\main\test\unit_test\video_processin:220]
#10 
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void> 
[testing\gtest\src\gtest.cc:1981]
Note: @0:52:08.826 in thread 3804
Note: instruction: data16 movdqa (%eax) -> %xmm2
Suppression (error hash=#DC9ECE9FB2426F99#):
  For more info on using suppressions see http://dev.chromium.org/developers/how-tos/using-drmemory#TOC-Suppressing-error-reports-from-the-
{
UNINITIALIZED READ
name=<insert_a_suppression_name_here>
*!ScaleAddRows_SSE2
*!ScalePlaneBox
*!ScalePlaneDown
*!ScalePlane
*!I420Scale
*!webrtc::Scaler::Scale
*!webrtc::VPMSimpleSpatialResampler::ResampleFrame
*!webrtc::VPMFramePreprocessor::PreprocessFrame
*!webrtc::VideoProcessingModuleImpl::PreprocessFrame
*!webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody
*!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>
}

04:36:52 drmemory_analyze.py [INFO] Total: 2 error reports

Please use labels and text to provide additional information.

I will suppress this error as I'm bringing up Dr Memory bots. Please edit the 
tools\valgrind-webrtc\drmemory\suppressions.txt temporarily if you want to 
re-trigger the failure.

Original issue reported on code.google.com by kjellan...@google.com on 30 Aug 2013 at 8:01

GoogleCodeExporter commented 9 years ago

FYI, when passing the --gtest_filter flag, it seems like the wrapper script 
fails to add the suppressions. Ending the command with "" seems to enable them 
(see the resulting output and look for suppressions.txt).

Original comment by kjellan...@google.com on 30 Aug 2013 at 8:06

GoogleCodeExporter commented 9 years ago

Actually, you need to use a -- for the command line to be able to pass 
--gtest_filter properly. Like this:
tools\valgrind-webrtc\webrtc_tests.bat --build_dir out\Debug --test 
modules_unittests --tool drmemory_full -- 
--gtest_filter=VideoProcessingModuleTest.PreprocessorLogic*

Original comment by kjellan...@google.com on 30 Aug 2013 at 9:54

GoogleCodeExporter commented 9 years ago

Looking at the code, I dont check width
#if defined(HAS_SCALEADDROWS_SSE2)
    if (TestCpuFlag(kCpuHasSSE2) &&
        IS_ALIGNED(src_ptr, 16) && IS_ALIGNED(src_stride, 16)) {
      ScaleAddRows = ScaleAddRows_SSE2;
    }
#endif

Its somewhat safe, since the results are discarded.
If I add a check that width is a multiple of 16, it will hurt performance on 
odd width images, but they don't happen very often.

Original comment by fbarch...@google.com on 30 Aug 2013 at 7:58

Changed state: New

GoogleCodeExporter commented 9 years ago

Fixed in r776

Original comment by fbarch...@google.com on 2 Sep 2013 at 10:34

Changed state: Fixed

GoogleCodeExporter commented 9 years ago

DrMemory on scale is not fixed apparently.  Reopened.

Original comment by fbarch...@chromium.org on 2 Sep 2013 at 6:32

Changed state: New

GoogleCodeExporter commented 9 years ago

Installed DrMemory and scale tests all pass:
drmemory out\debug\libyuv_unittest.exe --gtest_catch_exceptions=0 
--gtest_filter=*Scale*

Dr. Memory version 1.6.0 build 2 built on Sep  7 2013 03:37:07
Dr. Memory results for pid 1112: "libyuv_unittest.exe"
Application cmdline: "out\debug\libyuv_unittest.exe --gtest_catch_exceptions=0 
--gtest_filter=*Scale*"
Recorded 97 suppression(s) from default C:\Program Files (x86)\Dr. 
Memory\bin\suppress-default.txt

===========================================================================
FINAL SUMMARY:

DUPLICATE ERROR COUNTS:

SUPPRESSIONS USED:

NO ERRORS FOUND:
      0 unique,     0 total unaddressable access(es)
      0 unique,     0 total uninitialized access(es)
      0 unique,     0 total invalid heap argument(s)
      0 unique,     0 total GDI usage error(s)
      0 unique,     0 total warning(s)
      0 unique,     0 total,      0 byte(s) of leak(s)
      0 unique,     0 total,      0 byte(s) of possible leak(s)
ERRORS IGNORED:
     10 potential leak(s) (suspected false positives)
         (details: C:\Users\fbarchard\AppData\Roaming\Dr. Memory\DrMemory-libyuv_unittest.exe.1112.000\potential_errors.txt)
      7 unique,    79 total,  11228 byte(s) of still-reachable allocation(s)
         (re-run with "-show_reachable" for details)
Details: C:\Users\fbarchard\AppData\Roaming\Dr. 
Memory\DrMemory-libyuv_unittest.exe.1112.000\results.txt

Others fail, so I'll leave the other DrMemory bug open until all pass.

Original comment by fbarch...@google.com on 24 Sep 2013 at 1:35

Changed state: Fixed

GoogleCodeExporter commented 9 years ago

Unfortunately it seems like the way the WebRTC modules_unittests is using the 
code still triggers the failure. 
I get this error using libyuv r800 and WebRTC r4824 with the suppression 
removed, when building and running with the instructions at the top of this 
issue:

14:43:37 drmemory_analyze.py [INFO] Report #1
UNINITIALIZED READ: reading 0x037dfd40-0x037dfd50 16 byte(s) within 
0x037dfd40-0x037dfd50
# 0 ScaleAddRows_SSE2                                                          
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:769]
# 1 ScalePlaneDown                                                             
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2245]
# 2 ScalePlane                                                                 
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2300]
# 3 I420Scale                                                                  
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2368]
# 4 webrtc::Scaler::Scale                                                      
[e:\src\webrtc\src\webrtc\common_video\libyuv\scaler.cc:76]
# 5 webrtc::VPMSimpleSpatialResampler::ResampleFrame                           
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\spatial_resampler.cc:86]
# 6 webrtc::VPMFramePreprocessor::PreprocessFrame                              
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\frame_preprocessor.cc:159]
# 7 webrtc::VideoProcessingModuleImpl::PreprocessFrame                         
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\video_processing_impl.cc:275]
# 8 webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody         
[e:\src\webrtc\src\webrtc\modules\video_processing\main\test\
unit_test\video_processing_unittest.cc:220]
# 9 
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void> 
[e:\src\webrtc\src\testing\gtest\src\gtest.cc:1981]
Note: @0:00:04.992 in thread 2064
Note: instruction: data16 movdqa (%esi) -> %xmm0
Suppression (error hash=#AA3F5D4A49383B9F#):
  For more info on using suppressions see http://dev.chromium.org/developers/how-tos/using-drmemory#TOC-Suppressing-error-reports-from-the-
{
UNINITIALIZED READ
name=<insert_a_suppression_name_here>
*!ScaleAddRows_SSE2
*!ScalePlaneDown
*!ScalePlane
*!I420Scale
*!webrtc::Scaler::Scale
*!webrtc::VPMSimpleSpatialResampler::ResampleFrame
*!webrtc::VPMFramePreprocessor::PreprocessFrame
*!webrtc::VideoProcessingModuleImpl::PreprocessFrame
*!webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody
*!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>
}

14:43:37 drmemory_analyze.py [INFO] Report #2
UNINITIALIZED READ: reading 0x037dfea0-0x037dfeb0 16 byte(s) within 
0x037dfea0-0x037dfeb0
# 0 ScaleAddRows_SSE2                                                          
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:782]
# 1 ScalePlaneBox                                                              
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2070]
# 2 ScalePlaneDown                                                             
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2245]
# 3 ScalePlane                                                                 
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2300]
# 4 I420Scale                                                                  
[e:\src\webrtc\src\third_party\libyuv\source\scale.cc:2368]
# 5 webrtc::Scaler::Scale                                                      
[e:\src\webrtc\src\webrtc\common_video\libyuv\scaler.cc:76]
# 6 webrtc::VPMSimpleSpatialResampler::ResampleFrame                           
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\spatial_resampler.cc:86]
# 7 webrtc::VPMFramePreprocessor::PreprocessFrame                              
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\frame_preprocessor.cc:159]
# 8 webrtc::VideoProcessingModuleImpl::PreprocessFrame                         
[e:\src\webrtc\src\webrtc\modules\video_processing\main\sourc
e\video_processing_impl.cc:275]
# 9 webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody         
[e:\src\webrtc\src\webrtc\modules\video_processing\main\test\
unit_test\video_processing_unittest.cc:220]
#10 
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void> 
[e:\src\webrtc\src\testing\gtest\src\gtest.cc:1981]
Note: @0:00:04.992 in thread 2064
Note: instruction: data16 movdqa (%eax) -> %xmm2
Suppression (error hash=#DC9ECE9FB2426F99#):
  For more info on using suppressions see http://dev.chromium.org/developers/how-tos/using-drmemory#TOC-Suppressing-error-reports-from-the-
{
UNINITIALIZED READ
name=<insert_a_suppression_name_here>
*!ScaleAddRows_SSE2
*!ScalePlaneBox
*!ScalePlaneDown
*!ScalePlane
*!I420Scale
*!webrtc::Scaler::Scale
*!webrtc::VPMSimpleSpatialResampler::ResampleFrame
*!webrtc::VPMFramePreprocessor::PreprocessFrame
*!webrtc::VideoProcessingModuleImpl::PreprocessFrame
*!webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody
*!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>
}

14:43:37 drmemory_analyze.py [INFO] Total: 2 error reports

Original comment by kjellan...@google.com on 24 Sep 2013 at 12:46

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

This is drmemory test procedure for libyuv

set GYP_DEFINES=build_for_tool=drmemory target_arch=ia32
call python build\gyp_chromium -fninja -G msvs_version=2012 --depth=. 
libyuv_test.gyp
ninja -C out\Debug
drmemory out\debug\libyuv_unittest.exe --gtest_catch_exceptions=0 
--gtest_filter=*

similar should work on webrtc

Original comment by fbarch...@google.com on 25 Sep 2013 at 6:37

Changed state: Started

GoogleCodeExporter commented 9 years ago

Couldn't it just be that 
https://code.google.com/p/webrtc/source/browse/trunk/webrtc/modules/video_proces
sing/main/source/spatial_resampler.cc#86 is using the code in scaler.cc in a 
different way than the libyuv test?

Have you retried with libyuv as a third_party dependency of WebRTC in a WebRTC 
checkout, building and running with the steps in the initial bug description?

Original comment by kjellan...@google.com on 1 Oct 2013 at 10:32

GoogleCodeExporter commented 9 years ago

FYI  Still working on reproducing this result.
Requires a sync/build of webrtc with drmemory, and one or more of the unittests 
that fail when run under drmemory.

libyuv unittests all pass

Original comment by fbarch...@google.com on 13 Oct 2013 at 4:11

GoogleCodeExporter commented 9 years ago

The original bug description contains everything needed to build and run the 
WebRTC test on Windows. Please ping me if you need any help setting it up.

Original comment by kjellan...@google.com on 15 Oct 2013 at 12:52

GoogleCodeExporter commented 9 years ago

As discussed offline, unable to build with vs2010 or vs2012.
livyuv builds with both and passes DrMemory.

Original comment by fbarch...@google.com on 31 Oct 2013 at 7:32

Changed state: New

GoogleCodeExporter commented 9 years ago

[ RUN      ] VideoProcessingModuleTest.PreprocessorLogic
Tested, reproduced
~~Dr.M~~ Note: instruction: data16 movdqa (%esi) -> %xmm0
~~Dr.M~~
~~Dr.M~~ Error #59: UNINITIALIZED READ: reading 0x04754eb0-0x04754ec0 16 
byte(s) within 0x04754eb0-0x04754ec0
~~Dr.M~~ # 0 ScaleAddRows_SSE2                                                  
        [d:\src\webrtc\trunk\third_party\libyuv\sour
ce\scale.cc:845]
~~Dr.M~~ # 1 ScalePlaneBox                                                      
        [d:\src\webrtc\trunk\third_party\libyuv\sour
ce\scale.cc:2564]
~~Dr.M~~ # 2 ScalePlane                                                         
        [d:\src\webrtc\trunk\third_party\libyuv\sour
ce\scale.cc:2931]
~~Dr.M~~ # 3 I420Scale                                                          
        [d:\src\webrtc\trunk\third_party\libyuv\sour
ce\scale.cc:3013]
~~Dr.M~~ # 4 webrtc::Scaler::Scale                                              
        [d:\src\webrtc\trunk\webrtc\common_video\lib
yuv\scaler.cc:76]
~~Dr.M~~ # 5 webrtc::VPMSimpleSpatialResampler::ResampleFrame                   
        [d:\src\webrtc\trunk\webrtc\modules\video_pr
ocessing\main\source\spatial_resampler.cc:68]
~~Dr.M~~ # 6 webrtc::VPMFramePreprocessor::PreprocessFrame                      
        [d:\src\webrtc\trunk\webrtc\modules\video_pr
ocessing\main\source\frame_preprocessor.cc:124]
~~Dr.M~~ # 7 webrtc::VideoProcessingModuleImpl::PreprocessFrame                 
        [d:\src\webrtc\trunk\webrtc\modules\video_pr
ocessing\main\source\video_processing_impl.cc:207]
~~Dr.M~~ # 8 webrtc::VideoProcessingModuleTest_PreprocessorLogic_Test::TestBody 
        [d:\src\webrtc\trunk\webrtc\modules\video_pr
ocessing\main\test\unit_test\video_processing_unittest.cc:210]
~~Dr.M~~ # 9 
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void> 
[d:\src\webrtc\trunk\testing\gtest\src\gt
est.cc:1981]
~~Dr.M~~ #10 
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,void> 
[d:\src\webrtc\trunk\testing\gtest\src\gtest
.cc:2048]
~~Dr.M~~ #11 testing::Test::Run                                                 
        [d:\src\webrtc\trunk\testing\gtest\src\gtest
.cc:2068]
~~Dr.M~~ Note: @0:49:31.586 in thread 2220
~~Dr.M~~ Note: instruction: data16 movdqa (%eax) -> %xmm2
[       OK ] VideoProcessingModuleTest.PreprocessorLogic (207 ms)

Original comment by fbarch...@google.com on 25 Nov 2013 at 7:58

Changed state: Started

GoogleCodeExporter commented 9 years ago

fixed in r5198 of webrtc.
The issue was exactly what it seemed like - the memory used in webrtc scale 
unittest was uninitialized.  So I initialized it.

Original comment by fbarch...@google.com on 25 Nov 2013 at 9:12

Changed state: Fixed

watery01 / libyuv

DrMemory reports uninitialized reads in scale.cc #263