search

Cyclonecat / libyuv

Automatically exported from code.google.com/p/libyuv
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

test msan #564

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
compile and test run with msan=1 

set all these: GYP_DEFINES="msan=1 msan_track_origins=2 
use_prebuilt_instrumented_libraries=1"

fix any issues.
document how to do it in Getting Started.

Original issue reported on code.google.com by fbarch...@google.com on 2 Feb 2016 at 11:13

GoogleCodeExporter commented 8 years ago 
To get the instrumented libraries binary package downloaded, you need the 
GYP_DEFINES set in the environment.
To make sure you trigger a new Chromium sync when you have that, do like this:

export GYP_DEFINES="msan=1 msan_track_origins=2 
use_prebuilt_instrumented_libraries=1"
rm chromium/.last_sync_chromium 
gclient sync

I landed https://codereview.chromium.org/1665573003/ that allows compile to 
work.

Then I get this error:

[317/318] LINK libyuv_unittest
FAILED: ../../third_party/llvm-build/Release+Asserts/bin/clang++ -Wl,-z,now 
-Wl,-z,relro -Wl,--fatal-warnings -pthread -Wl,-z,noexecstack -fPIC 
-fuse-ld=gold 
-B/usr/local/google/home/kjellander/src/libyuv/libyuv/third_party/binutils/Linux
_x64/Release/bin -Wl,--disable-new-dtags -Wl,-u_sanitizer_options_link_helper 
-m64 
--sysroot=/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/debian
_wheezy_amd64-sysroot 
-L/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/debian_wheezy_
amd64-sysroot/lib/x86_64-linux-gnu 
-Wl,-rpath-link=/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/
debian_wheezy_amd64-sysroot/lib/x86_64-linux-gnu 
-L/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/debian_wheezy_
amd64-sysroot/usr/lib/x86_64-linux-gnu 
-Wl,-rpath-link=/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/
debian_wheezy_amd64-sysroot/usr/lib/x86_64-linux-gnu 
-L/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/debian_wheezy_
amd64-sysroot/usr/lib/gcc/x86_64-linux-gnu/4.6 
-Wl,-rpath-link=/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/
debian_wheezy_amd64-sysroot/usr/lib/gcc/x86_64-linux-gnu/4.6 
-L/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/debian_wheezy_
amd64-sysroot/usr/lib 
-Wl,-rpath-link=/usr/local/google/home/kjellander/src/libyuv/libyuv/build/linux/
debian_wheezy_amd64-sysroot/usr/lib -fsanitize=memory -stdlib=libc++ 
-Wl,-R,\$ORIGIN/lib/ -Wl,-R,\$ORIGIN/instrumented_libraries_prebuilt/msan/lib/ 
-Wl,-z,origin -Wl,-O1 -Wl,--gc-sections -o libyuv_unittest -Wl,--start-group 
obj/unit_test/libyuv_unittest.basictypes_test.o 
obj/unit_test/libyuv_unittest.compare_test.o 
obj/unit_test/libyuv_unittest.color_test.o 
obj/unit_test/libyuv_unittest.convert_test.o 
obj/unit_test/libyuv_unittest.cpu_test.o 
obj/unit_test/libyuv_unittest.math_test.o 
obj/unit_test/libyuv_unittest.planar_test.o 
obj/unit_test/libyuv_unittest.rotate_argb_test.o 
obj/unit_test/libyuv_unittest.rotate_test.o 
obj/unit_test/libyuv_unittest.scale_argb_test.o 
obj/unit_test/libyuv_unittest.scale_test.o 
obj/unit_test/libyuv_unittest.unit_test.o 
obj/unit_test/libyuv_unittest.video_common_test.o 
obj/chromium/src/build/sanitizers/libsanitizer_options.a libyuv.a 
obj/chromium/src/testing/libgtest.a obj/third_party/gflags/libgflags.a 
obj/chromium/src/third_party/libjpeg_turbo/libjpeg_turbo.a -Wl,--end-group  
-Llib/ 
../../unit_test/math_test.cc:30: error: undefined reference to 'FixedDiv_X86'
../../unit_test/math_test.cc:31: error: undefined reference to 'FixedDiv_X86'
../../unit_test/math_test.cc:36: error: undefined reference to 'FixedDiv_X86'
../../unit_test/math_test.cc:37: error: undefined reference to 'FixedDiv_X86'
../../unit_test/math_test.cc:142: error: undefined reference to 'FixedDiv1_X86'
../../unit_test/planar_test.cc:1364: error: undefined reference to 
'ARGBAffineRow_SSE2'
../../unit_test/planar_test.cc:1371: error: undefined reference to 
'ARGBAffineRow_SSE2'
../../unit_test/planar_test.cc:1402: error: undefined reference to 
'SobelXRow_SSE2'
../../unit_test/planar_test.cc:1439: error: undefined reference to 
'SobelYRow_SSE2'
../../unit_test/planar_test.cc:1483: error: undefined reference to 
'SobelRow_SSE2'
../../unit_test/planar_test.cc:1521: error: undefined reference to 
'SobelToPlaneRow_SSE2'
../../unit_test/planar_test.cc:1561: error: undefined reference to 
'SobelXYRow_SSE2'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Original comment by kjellander@chromium.org on 3 Feb 2016 at 10:51

GoogleCodeExporter commented 8 years ago 
The following revision refers to this bug:
  https://chromium.googlesource.com/libyuv/libyuv.git/+/af90e82155a42ec8239685ca8626c547c7173220

commit af90e82155a42ec8239685ca8626c547c7173220
Author: Henrik Kjellander <kjellander@chromium.org>
Date: Wed Feb 03 10:52:19 2016

Add empty MSan blacklist.txt to make MSan builds pass.

TBR=fbarchard@chromium.org
BUG=libyuv:564

Review URL: https://codereview.chromium.org/1665573003 .

[add] 
http://crrev.com/af90e82155a42ec8239685ca8626c547c7173220/tools/msan/OWNERS
[add] 
http://crrev.com/af90e82155a42ec8239685ca8626c547c7173220/tools/msan/blacklist.t
xt

Original comment by bugdroid1@chromium.org on 3 Feb 2016 at 10:53

GoogleCodeExporter commented 8 years ago 
Confirmed.

Here is a typical line from the build:

[118/318] ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF 
obj/source/libyuv.scale_common.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 
-D_FILE_OFFSET_BITS=64 -DNO
_TCMALLOC -DCHROMIUM_BUILD -DCR_CLANG_REVISION=256316-1 
-DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_ASH=1 -DUSE_PANGO=1 
-DUSE_CAIRO=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJ
PEG_TURBO=1 -DUSE_X11=1 -DUSE_CLIPBOARD_AURAX11=1 -DENABLE_ONE_CLICK_SIGNIN 
-DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_PEPPER_CDMS 
-DENABLE_CONFIGURATION_POLICY -DENABLE_NO
TIFICATIONS -DENABLE_HIDPI=1 -DENABLE_TOPCHROME_MD=1 -DUSE_UDEV 
-DDONT_EMBED_BUILD_METADATA -DFIELDTRIAL_TESTING_ENABLED 
-DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PDF=
1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 
-DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_PRINTING=1 
-DENABLE_BASIC_PRINTING=1 -DENABLE_PRINT_PREV
IEW=1 -DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 
-DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=1 -DENABLE_SUPERVISED_USERS=1 
-DENABLE_MDNS=1 -DENABLE_SERVICE_DISCOVERY=1
 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DHAVE_JPEG -DLIBYUV_DISABLE_X86 -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DUSE_GLIB=1 -DUSE
_NSS_CERTS=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS 
-DMEMORY_TOOL_REPLACES_ALLOCATOR -DMEMORY_SANITIZER_INITIAL_SIZE 
-DMEMORY_SANITIZER -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DW
TF_USE_DYNAMIC_ANNOTATIONS=1 -D_DEBUG -D_GLIBCXX_DEBUG=1 -Igen -I../../include 
-I../.. -I../../chromium/src/third_party/libjpeg_turbo -fstack-protector 
--param=ssp-buffer-size=4 -W
error -pthread -fno-strict-aliasing -Wall -Wextra -Wno-unused-parameter 
-Wno-missing-field-initializers -fvisibility=hidden -pipe -fPIC -Xclang -load 
-Xclang /usr/local/google/home
/fbarchard/src/build/libyuv/libyuv/third_party/llvm-build/Release+Asserts/lib/li
bFindBadConstructs.so -Xclang -add-plugin -Xclang find-bad-constructs -Xclang 
-plugin-arg-find-bad-c
onstructs -Xclang check-templates -fcolor-diagnostics 
-B/usr/local/google/home/fbarchard/src/build/libyuv/libyuv/third_party/binutils/
Linux_x64/Release/bin -Wheader-hygiene -Wfor-l
oop-analysis -Wno-char-subscripts -Wno-unneeded-internal-declaration 
-Wno-covered-switch-default -Wstring-conversion -Wno-c++11-narrowing 
-Wno-deprecated-register -Wno-inconsistent
-missing-override -Wno-shift-negative-value -m64 -march=x86-64 
--sysroot=/usr/local/google/home/fbarchard/src/build/libyuv/libyuv/build/linux/d
ebian_wheezy_amd64-sysroot -fno-omit-
frame-pointer -gline-tables-only -fsanitize=memory 
-fsanitize-memory-track-origins=2 
-fsanitize-blacklist=../../tools/msan/blacklist.txt 
-isystem../../buildtools/third_party/libc++
/trunk/include 
-isystem../../buildtools/third_party/libc++/../libc++abi/trunk/include -O0 -g 
-funwind-tables -gsplit-dwarf -fno-exceptions -fno-rtti -fno-threadsafe-statics 
-fvisib
ility-inlines-hidden -std=gnu++11 -nostdinc++  -c ../../source/scale_common.cc 
-o obj/source/libyuv.scale_common.o

My guess is its this define:
-DLIBYUV_DISABLE_X86

Original comment by fbarch...@google.com on 8 Feb 2016 at 11:27

GoogleCodeExporter commented 8 years ago 
Build flags are inconsistent.
The library has -DLIBYUV_DISABLE_X86
But the unittests do not.

/319] ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF 
obj/unit_test/libyuv_unittest.math_test.o.d -DV8_DEPRECATION_WARNINGS 
-DCLD_VERSION=2 -D_FILE_OFFSET_BIT
 -DNO_TCMALLOC -DCHROMIUM_BUILD -DCR_CLANG_REVISION=259395-1 -DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_PANGO=1 -DUSE_CAIRO=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJPEG
BO=1 -DUSE_X11=1 -DUSE_CLIPBOARD_AURAX11=1 -DENABLE_ONE_CLICK_SIGNIN 
-DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_PEPPER_CDMS 
-DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIF
IONS -DENABLE_HIDPI=1 -DENABLE_TOPCHROME_MD=1 -DUSE_UDEV 
-DDONT_EMBED_BUILD_METADATA -DFIELDTRIAL_TESTING_ENABLED 
-DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PDF=1 -
BLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 
-DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_PRINTING=1 
-DENABLE_BASIC_PRINTING=1 -DENABLE_PRINT_PREVIEW
DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 
-DENABLE_SETTINGS_APP=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_MDNS=1 
-DENABLE_SERVICE_DISCOVERY=1 -D
SE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD 
-DSAFE_BROWSING_DB_LOCAL -DHAVE_JPEG -DGTEST_HAS_POSIX_RE=0 
-DGTEST_LANG_CXX11=0 -DUNIT_TEST -DGTEST_HAS_RTTI=
GFLAGS_DLL_DECL= -DGFLAGS_DLL_DECLARE_FLAG= -DGFLAGS_DLL_DEFINE_FLAG= 
-DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 
-DMEMORY_TOOL_REPLACES_ALLOCATOR -DMEMORY_SA
ZER_INITIAL_SIZE -DMEMORY_SANITIZER -DDYNAMIC_ANNOTATIONS_ENABLED=1 
-DWTF_USE_DYNAMIC_ANNOTATIONS=1 -D_DEBUG -D_GLIBCXX_DEBUG=1 -Igen 
-I../../include -I../.. -I../../chromium/s
esting/gtest/include -I../../third_party/gflags/gen/posix/include 
-I../../third_party/gflags/src -fstack-protector --param=ssp-buffer-size=4 
-Werror -pthread -fno-strict-aliasi
Wall -Wno-unused-parameter -Wno-missing-field-initializers -fvisibility=hidden 
-pipe -fPIC -Xclang -load -Xclang 
/usr/local/google/home/fbarchard/src/build/libyuv/libyuv/third_
y/llvm-build/Release+Asserts/lib/libFindBadConstructs.so -Xclang -add-plugin 
-Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang 
check-templates -Xclang
ugin-arg-find-bad-constructs -Xclang follow-macro-expansion -fcolor-diagnostics 
-B/usr/local/google/home/fbarchard/src/build/libyuv/libyuv/third_party/binutils/
Linux_x64/Releas
n -Wheader-hygiene -Wno-char-subscripts -Wno-unneeded-internal-declaration 
-Wno-covered-switch-default -Wstring-conversion -Wno-c++11-narrowing 
-Wno-deprecated-register -Wno-in
istent-missing-override -Wno-shift-negative-value -Wno-unused-variable 
-fexceptions -m64 -march=x86-64 
--sysroot=/usr/local/google/home/fbarchard/src/build/libyuv/libyuv/build/
x/debian_wheezy_amd64-sysroot -fno-omit-frame-pointer -gline-tables-only 
-fsanitize=memory -fsanitize-memory-track-origins=2 
-fsanitize-blacklist=../../tools/msan/blacklist.txt
 -g -funwind-tables -gsplit-dwarf -fno-exceptions -fno-rtti -fno-threadsafe-statics -fvisibility-inlines-hidden -std=gnu++11 -Wno-deprecated -nostdinc++ -isystem../../buildtool
ird_party/libc++/trunk/include 
-isystem../../buildtools/third_party/libc++/../libc++abi/trunk/include  -c 
../../unit_test/math_test.cc -o obj/unit_test/libyuv_unittest.math_tes

The disable occurs in libyuv.gyp

        # MemorySanitizer does not support assembly code yet.
        # http://crbug.com/344505
        [ 'msan == 1', {
          'defines': [
            'LIBYUV_DISABLE_X86',
          ],
        }],
      ], #conditions

Original comment by fbarch...@google.com on 9 Feb 2016 at 7:07

GoogleCodeExporter commented 8 years ago 
crbug for msan assembly issue
https://code.google.com/p/chromium/issues/detail?id=344505

Original comment by fbarch...@google.com on 9 Feb 2016 at 7:07

GoogleCodeExporter commented 8 years ago 
The following revision refers to this bug:
  https://chromium.googlesource.com/libyuv/libyuv.git/+/6ea3755330fcd8e5531e341c5460fe147324ccbd

commit 6ea3755330fcd8e5531e341c5460fe147324ccbd
Author: Frank Barchard <fbarchard@google.com>
Date: Tue Feb 09 19:57:03 2016

add 'LIBYUV_DISABLE_X86' to msan for unittests

R=harryjin@google.com
BUG=libyuv:564

Review URL: https://codereview.chromium.org/1685723002 .

[modify] http://crrev.com/6ea3755330fcd8e5531e341c5460fe147324ccbd/.gitignore
[modify] 
http://crrev.com/6ea3755330fcd8e5531e341c5460fe147324ccbd/README.chromium
[modify] 
http://crrev.com/6ea3755330fcd8e5531e341c5460fe147324ccbd/include/libyuv/version
.h
[modify] http://crrev.com/6ea3755330fcd8e5531e341c5460fe147324ccbd/libyuv.gyp
[modify] 
http://crrev.com/6ea3755330fcd8e5531e341c5460fe147324ccbd/libyuv_test.gyp

Original comment by bugdroid1@chromium.org on 9 Feb 2016 at 7:57

GoogleCodeExporter commented 8 years ago 
fixed for gyp

Original comment by fbarch...@google.com on 9 Feb 2016 at 7:58

GoogleCodeExporter commented 8 years ago

Original comment by fbarch...@google.com on 9 Feb 2016 at 7:59