utils/lit/tests/shtest-format.py fails on Windows

[Quuxplusone]

Bugzilla Link	PR33939
Status	NEW
Importance	P enhancement
Reported by	Brian Gesiak (modocache@gmail.com)
Reported on	2017-07-25 22:06:00 -0700
Last modified on	2017-08-18 10:45:16 -0700
Version	trunk
Hardware	PC All
CC	daniel@zuster.org, douglas_yung@playstation.sony.com, llvm-bugs@lists.llvm.org, rnk@google.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

As described in utils/lit/README.txt, lit has its own test suite, which can be run like so:

# From within your LLVM source directory.
utils/lit/lit.py \
    --path /path/to/your/llvm/build/bin \
    utils/lit/tests

However, when this command is run on a Windows host, the test utils/lit/tests/shtest-format.py fails.

[Quuxplusone]

This passes for me locally. I'm going to un-xfail it for now. Can you paste some logs to try to figure out if there are some environmental causes?

[Quuxplusone]

Here's the failure output on my Windows 10 desktop: https://gist.github.com/modocache/d29246751e92d6656ebb6594f15c5983. I have a potential fix for the problem on https://reviews.llvm.org/D27893, although I haven't yet implemented the solution suggested by the reviewers.

[Quuxplusone]

Oops, that last GitHub gist link contained some bad data. Here's a good link: https://gist.github.com/modocache/759cf2509f5519e8d5ce36792d3d99c1

[Quuxplusone]

We can't really expect the external_shell tests to pass without bash. They require things like '[.exe', or a shell that understands a builtin '[' command. I'm surprised you're seeing things liek "'true': command not found", though. Those are required for the LLVM suite, so surely you have a true.exe?

Anyway, maybe I can repro those failures by hacking lit.cfg so it doesn't find my gnu utilities.

I also think we should deprecate or remove the lit support for making .bat scripts and running them with cmd.exe. At this point, we're aiming to standardize on the internal shell anyway.

[Quuxplusone]

I just found this bug and wanted to add we were hitting this exact same problem (unable to find binaries for true, cat, etc.) on our internal build bot. I couldn't reproduce the failure on my own machine, so I had to disable the tests in our internal copy to unblock the bot.

I don't think it is a problem with the build bot since it runs all of the other clang/llvm lit tests fine which use those commands, but the lit test suite runs separately and seems to have problems getting the environment setup correctly.