Segmentation fault in ThreadSanitizerOffline with JavaThreadSanitizer tests

[GoogleCodeExporter]

Hello!
I wanted to try out JavaThreadSanitizer, but ran into some problems there:

What steps will reproduce the problem?
1. Build offline tsan according to 
http://code.google.com/p/data-race-test/wiki/ThreadSanitizerOffline
simple_race_1.tst work normally
2. Cd to third-party/java-thread-sanitizer, run `ant download; ant`
3. Run `ant test`

What is the expected output? What do you see instead?

Instead of all test passing:
Buildfile: 
/space/mermakov/data-race-test-read-only/third_party/java-thread-sanitizer/build
.xml

test:
     [echo] tsan_path = ../../tsan/bin/amd64-linux-debug-ts_offline
     [echo] agent_args = logfile=log/events
     [echo] args = 
     [echo] running tests under agent
     [echo] running tsan-offline
     [exec] Result: 139
     [echo] summarizing results
     [exec] log
     [exec] PASS noLockWWInt
     [exec] TSAN didn't handle test noLockWWShort
     [exec] Result: 1

BUILD FAILED
/space/mermakov/data-race-test-read-only/third_party/java-thread-sanitizer/build
.xml:128: if=test-fail.run

Total time: 8 seconds

It appears that running offline tsan generates a segmentation fault.
Running tests manually:
$ java -Xbootclasspath/p:dist/agent.jar 
-javaagent:dist/agent.jar=logfile=events -jar dist/tests.jar
Java Agent: appending threading events to file: events
>>>> org.jtsan.TestRunner: START
>>>> org.jtsan.TestRunner: Handle class EasyTests
======== noLockWWInt ========
======== noLockWWShort ========
======== noLockWWChar ========
======== noLockWWLong ========
======== noLockWWFloat ========
======== noLockWWDouble ========
======== noLockWWObject ========
======== lockedVsNoLockedWW ========
======== lockedVsNoLockedWW2 ========
======== differentLocksWW ========
======== recursiveObjectWW ========
======== systemArrayCopy ========
======== systemArrayCopy2 ========
======== systemArrayCopy3 ========
======== noOperation ========
======== noOperation2 ========
======== lockedInc ========
======== lockedInc2 ========
======== distinctFields ========
======== localVolatileBoolean ========
======== differentFields ========
======== differentFields2 ========
======== lockedWW ========
======== systemArrayCopyDiff ========
======== systemArrayCopySync ========
======== systemArrayCopyDiff2 ========
======== systemArrayCopyDiff3 ========
======== systemArrayCopyIndexOutOfBoundsException ========
Exception in thread "Thread-109" java.lang.ArrayIndexOutOfBoundsException
    at java.lang.System.arraycopy(Native Method)
    at EasyTests$32.thread2(Unknown Source)
    at ThreadRunner$2.run(Unknown Source)
======== systemArrayCopyNullPointerException ========
Exception in thread "Thread-113" java.lang.NullPointerException
    at java.lang.System.arraycopy(Native Method)
    at EasyTests$33.thread2(Unknown Source)
    at ThreadRunner$2.run(Unknown Source)
======== systemArrayCopyArrayStoreException ========
Exception in thread "Thread-117" java.lang.ArrayStoreException
    at java.lang.System.arraycopy(Native Method)
    at EasyTests$34.thread2(Unknown Source)
    at ThreadRunner$2.run(Unknown Source)
>>>> org.jtsan.TestRunner: Handle class MediumTests
======== treeMapAccessNoLocks ========
======== arrayAccessNoLocks ========
======== notifyWait ========
======== notifyWait2 ========
======== deepField ========
======== passingViaLockedMap ========
======== syncWithLocalVolatile ========
======== messageViaLockedObject ========
======== passingViaLockedBoolean ========
======== passingViaLockedBoolean2 ========
======== arrayDifferentOffsets ========
======== exceptionWithSync ========
Exception in thread "Thread-164" java.lang.RuntimeException: Exit from 
synchronized method with this Exception
    at MediumTests$14.thread1(Unknown Source)
    at ThreadRunner$1.run(Unknown Source)
>>>> org.jtsan.TestRunner: Handle class CustomTests
======== staticFinal ========
======== exceptionExit ========
Exception in thread "Thread-172" java.lang.RuntimeException: Exit from thread1 
with this Exception
    at CustomTests$2.thread1(Unknown Source)
    at ThreadRunner$1.run(Unknown Source)
======== staticSync ========
======== staticSync2 ========
======== staticSync3 ========
======== staticCollisions ========
>>>> org.jtsan.TestRunner: Handle class JUConcurrentTests
======== tryLock ========
======== writingUnderReaderLock ========
======== differentLocksWW2 ========
======== cyclicBarrierWrong ========
======== arrayBlockingQueue ========
======== reentrantLockInterruptibly ========
======== countDownLatch ========
======== cyclicBarrier ========
======== semaphore ========
======== writeLocksOnly ========
======== readAndWriteLocks ========
======== reentrantLockSimple ========
======== tryLock2 ========
======== atomicInteger ========
======== fifoMutexUser ========
======== futureTask ========
======== lockNeMonitor ========
>>>> org.jtsan.TestRunner: Exclude list:
EXCL systemArrayCopyException      Reason: We can not distinguish two types of 
ArrayStoreException in native code occurred in System.arraycopy
EXCL staticVolatileBoolean         Reason: We handle volatile fields in super 
classes incorrectly
EXCL superClassVolatile            Reason: We handle volatile fields in super 
classes incorrectly
EXCL syncWithStaticVolatile        Reason: We handle volatile fields in super 
classes incorrectly
EXCL joinWithoutStart              Reason: Incorrect handling of join() in Agent
EXCL hashSetAccessNoLocks          Reason: HashSet loads before instrumentation 
starts
EXCL lockInBetween                 Reason: Tsan finds inexact happens-before arc
EXCL recTest                       Reason: See tsan output for this test 
manually
EXCL synchronousQueue              Reason: SynchronousQueue is not supported yet

$ gdb ../../tsan/bin/amd64-linux-debug-ts_offline 
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from 
/space/mermakov/data-race-test-read-only/tsan/bin/amd64-linux-debug-ts_offline..
.done.
(gdb) run < events 
Starting program: 
/space/mermakov/data-race-test-read-only/tsan/bin/amd64-linux-debug-ts_offline 
< log/events
INFO: ThreadSanitizerOffline r4393
INFO: Allocating 256Mb (32 * 8M) for Segments.
INFO: Will allocate up to 640Mb for 'previous' stack traces.
sizeof(CacheLine) = 568
======== noLockWWInt ========
Description: Two writes to shared int without synchronization
Race = true

Program received signal SIGSEGV, Segmentation fault.
0x000000000042f5f5 in CallStack::size (this=<value optimized out>, 
child_tid=DWARF-2 expression error: DW_OP_reg operations must be used either 
alone or in conjuction with DW_OP_piece.
) at thread_sanitizer.h:173
173   size_t size() { return (size_t)(end_ - pcs_); }
(gdb) bt
#0  0x000000000042f5f5 in CallStack::size (this=<value optimized out>, 
child_tid=DWARF-2 expression error: DW_OP_reg operations must be used either 
alone or in conjuction with DW_OP_piece.
) at thread_sanitizer.h:173
#1  TSanThread::FillEmbeddedStackTrace (this=<value optimized out>, 
child_tid=DWARF-2 expression error: DW_OP_reg operations must be used either 
alone or in conjuction with DW_OP_piece.
) at thread_sanitizer.cc:5053
#2  TSanThread::NewSegmentWithoutUnrefingOld (this=<value optimized out>, 
child_tid=DWARF-2 expression error: DW_OP_reg operations must be used either 
alone or in conjuction with DW_OP_piece.
) at thread_sanitizer.cc:4690
#3  TSanThread (this=<value optimized out>, child_tid=DWARF-2 expression error: 
DW_OP_reg operations must be used either alone or in conjuction with 
DW_OP_piece.
) at thread_sanitizer.cc:4264
#4  Detector::HandleThreadStart (this=<value optimized out>, child_tid=DWARF-2 
expression error: DW_OP_reg operations must be used either alone or in 
conjuction with DW_OP_piece.
) at thread_sanitizer.cc:7725
#5  0x0000000000434c4c in Detector::HandleOneEvent (this=0x655f30, 
e=0x7fffffffdfa0) at thread_sanitizer.cc:6510
#6  0x0000000000402195 in ReadEventsFromFile (file=0x7ffff76296a0, 
event_reader_cb=0x4030b0 <ReadOneStrEventFromFile(_IO_FILE*, Event*)>) at 
ts_offline.cc:316
#7  0x0000000000404144 in main (argc=<value optimized out>, argv=<value 
optimized out>) at ts_offline.cc:372
(gdb) print end_
Cannot access memory at address 0x0
(gdb) print pcs_ 
Cannot access memory at address 0x8

What version of the product are you using? On what operating system?
Downloaded latest (r4393) version of this tool package.
Ubuntu 10.04 64-bit

Please provide any additional information below.
JavaThreadSanitizer development seems to have stopped more than a year ago. 
Does that mean that tool is complete (up to desired state) and can be used with 
ThreadSanitizerOffline for real application testing? If not, do you plan 
anything with JavaThreadSanitizer? Thanks in advance!

Original issue reported on code.google.com by m.k.erma...@gmail.com on 30 Jul 2012 at 9:29

[GoogleCodeExporter]

Speaking about ThreadSanitizer for Java its state is more like "unsupported" 
rather than "complete".
If you attach the log that you're passing to ts_offline we can check if there's 
a bug on the ts_offline side. However if the problem is in incorrect order of 
events generated by the instrumented code, it's unlikely that anyone is going 
to fix it anytime soon.

Original comment by ramosian.glider@gmail.com on 31 Jul 2012 at 3:10

[GoogleCodeExporter]

Thanks for your response.

Here's the log.

Original comment by m.k.erma...@gmail.com on 31 Jul 2012 at 4:53

Attachments:

• events.tar.gz

[GoogleCodeExporter]

The same problem on Mac OS X. Also tried with other test code - the same line 
is reported.

Original comment by tsite...@gmail.com on 27 Sep 2012 at 5:48

[GoogleCodeExporter]

As there is some activity here, I've just wanted to mention:

Revision r2582 seems to be working ok, so I presume something was changed in 
the format used by ts_offline, which leads to these inconsistencies.

We are currently working on concurrency analysis for Android applications and 
since we couldn't use Java Thread Sanitizer directly (due to ASM and its 
dynamic bytecode generation) we've implemented trace generation (more or less 
consistent with JTsan) via BCEL in our instrumentation tool 
(http://code.google.com/p/coffee-machine/).

I understand that JTSan is no longer developed, but still our plans include 
trying to investigate Java Thread Sanitizer compatibility issues with current 
version of ts_offline. If we manage to resolve these issues, we'll gladly file 
a patch.

Original comment by m.k.erma...@gmail.com on 28 Sep 2012 at 2:28

[GoogleCodeExporter]

Confirm: r2582 version successfully parsed jtsan files 

Original comment by tsite...@gmail.com on 29 Sep 2012 at 1:24