PPC support?

[GoogleCodeExporter]

What steps will reproduce the problem?
1.shell script doesn't support ppc architecture.
2.
3.

What is the expected output? What do you see instead?
"Unsupported Architecture"

What version of the product are you using? On what operating system?
svn head, on RHEL 5, PPC64.

Original issue reported on code.google.com by xen...@gmail.com on 23 Nov 2010 at 1:07

[GoogleCodeExporter]

Supporting PPC Linux requires a) time, b) PCC machines and c) motivation. 
We have neither at the moment. 
If you can prepare a patch and give us access to a PPC machine to test it, we 
will be happy to accept it. 
Porting ThreadSanitizer to PPC Linux should not be terribly hard because 
memcheck and helgrind work there. 

Original comment by konstant...@gmail.com on 23 Nov 2010 at 5:55

• Added labels: Priority-Low, Type-Enhancement
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

I've changed the script but got such error. The flags are set to "ppc64 linux" 
while building tsan.

make all OS=linux ARCH=ppc64  DEBUG=1
make[1]: Entering directory `/home/john/drt/tsan'
g++ -O2 -g -Wall  -Wno-deprecated -fno-exceptions  -fno-rtti 
-fno-stack-protector -m64 -I../third_party/valgrind 
-I../third_party/valgrind/include -I../third_party/valgrind/VEX/pub 
-I../third_party/stlport -DVGA_ppc64=1 -DVGO_linux=1 -DVGP_ppc64_linux=1 
-D_STLP_NO_IOSTREAMS=1 -DTS_VALGRIND=1 -o 
bin/ppc64-linux-debug-vg-thread_sanitizer.o -c thread_sanitizer.cc 
-DTS_VERSION=\"2729M\" -DDEBUG=1 -I../dynamic_annotations
/tmp/ccajz14G.s: Assembler messages:
/tmp/ccajz14G.s:14596: Error: Unrecognized opcode: `bswapq'

I couldn't find the solution from google. Is it related the specific 
implementation of tsan to PPC architecture?

Original comment by xen...@gmail.com on 24 Nov 2010 at 5:26

[GoogleCodeExporter]

You will need to modify the code around 
http://code.google.com/p/data-race-test/source/browse/trunk/tsan/ts_util.h#335

Original comment by konstant...@gmail.com on 24 Nov 2010 at 5:32

[GoogleCodeExporter]

Hi, we tried to modify scripts and succeeded to have tsan successfully built on 
our ppc64 linux box. However, when we tested a simple data race program, we've 
got such error:

==20930== ThreadSanitizer, a data race detector
==20930== Copyright (C) 2008-2010, and GNU GPL'd, by Google Inc.
==20930== Using Valgrind-3.7.0.SVN and LibVEX; rerun with -h for copyright info
==20930== Command: test
==20930== 
==20930== ThreadSanitizerValgrind r2754M: hybrid=no
==20930== INFO: ThreadSanitizer memory limit: 3072MB
==20930== INFO: Allocating 192Mb (24 * 8M) for Segments.
==20930== INFO: Will allocate up to 320Mb for 'previous' stack traces.
LDle:I32(0x382C2B34:I32)
vex: the `impossible' happened:
   iselIntExpr_R(ppc): cannot reduce tree
vex storage: T total 20467364 bytes allocated
vex storage: P total 144 bytes allocated

valgrind: the 'impossible' happened:
   LibVEX called failure_exit().
==20930==    at 0x3808A408: report_and_quit (m_libcassert.c:193)
==20930==    by 0x3808A48F: panic (m_libcassert.c:277)
==20930==    by 0x3808A507: vgPlain_core_panic_at (m_libcassert.c:282)
==20930==    by 0x3808A523: vgPlain_core_panic (m_libcassert.c:287)
==20930==    by 0x380A809B: failure_exit (m_translate.c:674)
==20930==    by 0x3813372F: vpanic (main_util.c:226)
==20930==    by 0x38234587: iselWordExpr_R (host_ppc_isel.c:1954)
==20930==    by 0x3823AF1F: iselSB_PPC (host_ppc_isel.c:3877)
==20930==    by 0x3813212B: LibVEX_Translate (main_main.c:591)
==20930==    by 0x380A5C3B: vgPlain_translate (m_translate.c:1518)
==20930==    by 0x380D050F: vgPlain_scheduler (scheduler.c:868)
==20930==    by 0x380FF27F: run_a_thread_NORETURN (syswrap-linux.c:94)

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable
==20930==    at 0xFF431D0: ??? (in 
/tmp/valgrind.J20923/lib/valgrind/vgpreload_tsan-ppc32-linux.so)
==20930==    by 0xFFBEB4F: _dl_init (in /lib/ld-2.5.so)
==20930==    by 0xFFC6CF3: _start (in /lib/ld-2.5.so)

How to understand such error and fix it? Thanks.

Original comment by xen...@gmail.com on 26 Nov 2010 at 6:08

[GoogleCodeExporter]

[Just to remind: none of us has experience with PPC and we don't have access to 
a PPC box. So, just speculating...]
Quick check: did you try to run helgrind (valgrind --tool=helgrind)? 
If it doesn't work, please report the problem to the valgrind developers. 
If helgrind works, most likely tsan could be fixed as well... 

Original comment by konstant...@gmail.com on 26 Nov 2010 at 6:51

[GoogleCodeExporter]

General comment: most of ARCH-specific for valgrind-based threadsanitizer is 
located in tsan/ts_valgrind.c 
The code contains a few x86-specific things. 
For example, it assumes low-endiannes (search for Iend_LE). I suspect this code 
will not work on bigendian arch w/o modification.

Original comment by konstant...@gmail.com on 26 Nov 2010 at 6:56

[GoogleCodeExporter]

Ah, yea. 
The assert says: 
LDle:I32(0x382C2B34:I32)
vex: the `impossible' happened:
   iselIntExpr_R(ppc): cannot reduce tree

So, if you change Iend_LE to Iend_BE, this assert will most likely disappear.

Original comment by konstant...@gmail.com on 26 Nov 2010 at 7:08

[GoogleCodeExporter]

One more concern: I am not entirely sure that the way we maintain the shadow 
call stack (see UpdateCallStack) will work for PPC. 

Original comment by konstant...@gmail.com on 26 Nov 2010 at 7:10

[GoogleCodeExporter]

Hi, I have made some changes to tsan's makefile and built it on our ppc64 
machine. But the result of demo tests seem strange. Besides the true data race 
problems, there are always two more warnings. Here's the detail,

==13660== WARNING: Possible data race during read of size 8 at 0x80103C0DB0: {{{
==13660==    T2 (test-thread-2) (L{}):
==13660==     #0  arena_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #1  __libc_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #2  start_thread /lib64/power6/libpthread-2.5.so
==13660==   Concurrent write(s) happened at (OR AFTER) these points:
==13660==    T1 (test-thread-1) (L{}):
==13660==     #0  arena_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #1  __libc_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #2  start_thread /lib64/power6/libpthread-2.5.so
==13660==   Address 0x80103C0DB0 is 0 bytes inside data symbol "free_list"
==13660==    Race verifier data: 0x801035ACD8,0x801035ACD4
==13660== }}}
==13660== WARNING: Possible data race during write of size 8 at 0x80103C0D18: 
{{{
==13660==    T2 (test-thread-2) (L{}):
==13660==     #0  arena_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #1  __libc_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #2  start_thread /lib64/power6/libpthread-2.5.so
==13660==   Concurrent write(s) happened at (OR AFTER) these points:
==13660==    T1 (test-thread-1) (L{}):
==13660==     #0  arena_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #1  __libc_thread_freeres /lib64/power6/libc-2.5.so
==13660==     #2  start_thread /lib64/power6/libpthread-2.5.so
==13660==   Address 0x80103C0D18 is 2160 bytes inside data symbol "main_arena"
==13660==    Race verifier data: 0x801035ACDC,0x801035ACD4
==13660== }}}

I have checked all the demo tests and it happened every time.
It looks like some data races happened in starting threads and tsan caught them.
So does tsan monitor all the accesses inside thread creation?
What's more, on our amd64 machine, there are no such warnings.

Original comment by SenYe1...@gmail.com on 7 Jan 2011 at 7:04

[GoogleCodeExporter]

Wow! 
Do the rest of reports look sane?

I would not worry too much about arena_thread_freeres
It is either a real benign race inside this function, or tsan is not catching a 
happens-before arc that early at thread's live time. 

I would ignore the function arena_thread_freeres and see what happens next. 
In short, you need to create a file with one line: 
fun:arena_thread_freeres
and feed this file to tsan via --ignore=file_name
See http://code.google.com/p/data-race-test/wiki/ThreadSanitizerIgnores

Original comment by konstant...@gmail.com on 7 Jan 2011 at 8:03