search

Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

Running lit tests under windows can fail due to race conditions #19789

Open Quuxplusone opened 10 years ago

Quuxplusone commented 10 years ago
Bugzilla Link PR19790
Status NEW
Importance P normal
Reported by Dan Liew (dan@su-root.co.uk)
Reported on 2014-05-19 11:08:08 -0700
Last modified on 2016-11-20 18:18:51 -0800
Version trunk
Hardware PC other
CC daniel@zuster.org, llvm-bugs@lists.llvm.org, modocache@gmail.com
Fixed by commit(s)
Attachments failing-lit-tests.txt (4281 bytes, text/plain)
Blocks
Blocked by
See also

Created attachment 12540 Example output from lit under Windows

OS: Windows 8.1 Python: Python 3.4

It seems that when running with the default --use-threads under Windows 8 with multiple threads tests can randomly fail. Which looks like a race condition.

What it looks like is happening is that commands in the test are being executed on files before those files are closed.

Steps to reproduce

Here is a randomly generated test suite that uses lit and which appears on Windows to randomly fail (the number of tests and which tests fail is not consistent) when using --use-threads and thread count is greater than 1.

git clone https://github.com/delcypher/lit-concurrency-windows-bug.git cd lit-concurrency-windows-bug lit --use-threads -vs .

The tests have the following format

RUN: python %s > %t

RUN: %diff %s.expect %t

%s.expect is a text file containing the expected output of running the first command.

When I run I randomly see messages like this...

Command 1: "fc" "/W" "C:\Users\Daniel\Desktop\lit-windows-concurrency-bug\testdi
r-1\test-11.py.expect" "C:\Users\Daniel\Desktop\lit-windows-concurrency-bug\test
dir-1\Output\test-11.py.tmp"
Command 1 Result: 1
Command 1 Output:
Comparing files C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\te
st-11.py.expect and C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-
1\OUTPUT\TEST-11.PY.TMP
Resync Failed.  Files are too different.
***** C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\test-11.py.e
xpect
FOO 0x44dcda6a797d76de
FOO 0x598b88dbaa99e079
FOO 0x61b339ff248174e5
FOO 0x7b87a9e25fefe911
...
...
FOO 0x4bca2f3e6eb5568b
FOO 0x258dc101eb3a8082
***** C:\USERS\DANIEL\DESKTOP\LIT-WINDOWS-CONCURRENCY-BUG\TESTDIR-1\OUTPUT\TEST-
11.PY.TMP
*****

Command 1 Stderr:

--

********************

The output shown above (truncated) from the fc tool (the Windows version of the diff tool) seems to suggest that the temporary file created by the first command in the tests (%t) is empty when fc was executed which suggests to me that the temporary file isn't closed properly before executing the next command.

It appears if I use the --use-processes flag instead then these random test failures do not occur. Attached is an example of the output seen when executing under Windows.

Quuxplusone commented 10 years ago

Attached failing-lit-tests.txt (4281 bytes, text/plain): Example output from lit under Windows

Quuxplusone commented 10 years ago

Oh I should probably add that there is no issue running under Linux, all the tests pass consistently.