lldb provides imprecise informations to reproduce problems

[llvmbot]

Bugzilla Link	35037
Version	unspecified
OS	Windows NT
Reporter	LLVM Bugzilla Contributor
CC	@adrian-prantl,@dwblaikie,@JDevlieghere,@lhames,@rnk

Extended Description

I haven't (yet) looked into how hard this is to fix, but the current state is definitely not ideal.

A failure will look like:

AssertionError: False is not True : Command 'expression auto $add = [](int a, int b) { return a + b; }' returns successfully Config=x86_64-/usr/bin/ccache Session info generated @ Mon Oct 23 09:28:31 2017 To rerun this test, issue the following command from the 'test' directory:

./dotest.py -A x86_64 -C /usr/bin/ccache -v -t -N dsym lang/cpp/lambdas

If you copy-paste the command you get an error, because of: dotest.py: error: unrecognized arguments: -N

Removing the -N bit from the invocation is not really enough, because by default dotest.py picks the first lldb in $PATH, which is not necessarily the correct one (it's actually, almost never the correct one in development).

$ python dotest.py -A x86_64 -C /usr/bin/ccache -v -t lang/cpp/lambdas ['dotest.py', '-A', 'x86_64', '-C', '/usr/bin/ccache', '-v', '-t', 'lang/cpp/lambdas'] LLDB library dir: /usr/bin LLDB import library dir: /usr/bin Traceback (most recent call last): File "", line 1, in ImportError: No module named lldb.embedded_interpreter lldb version 3.8.0 ( revision )

Last but not least, I'm currrently bulding with an in-tree clang, so I'm expecting tests to be run with that very clang. Either they're not (in which case this might be taken in consideration for fix) or the cmdline to reproduce lacks some relevant informations (again).

[llvmbot]

I'm looking at this now. This function needs to be extended. The whole lldbtest.py could use some love so I might do some cleanups while I'm there.

def getRunOptions(self):
    """Command line option for -A and -C to run this test again, called 
       from self.dumpSessionInfo()."""
    arch = self.getArchitecture()
    comp = self.getCompiler()
    if arch:
        option_str = "-A " + arch
    else:
        option_str = ""
    if comp:
        option_str += " -C " + comp
    return option_str

[llvmbot]

Definitely agree it is bad for external developers. This is something that has bothered me for quite some time.

Several things you might think about:

1) The command line we present to the users doesn't need to be the same as the command line the user ran. It only needs to say how to regenerate the test environment exactly. This is important because certain options can be "inferred" when run from specific directories, since cwd can affect search paths. So we're free to add (and indeed, we should add) as many options as necessary to ensure we guarantee full determinism on the manual re-run.

2) In particular this means you'll want to present the user with a command line involving the -C and -e options. It looks like we're already adding -C, but that it's wrong, since it should be pointing to your in-tree clang. -e should be easy to find as well. If it was specified on the command line, use that. Otherwise, at some point we add the directory of liblldb to sys.path, so use that to construct a path to lldb.exe and then use that absolte path to -e when generating rerun args.

[llvmbot]

FWIW, this is not necessarily a showstopper for people familiar/working on lldb(or LLVM), but I consider this very bad for external developers (e.g. people downstream packaging lldb)/users.

If a test fails and the cmdline isn't accurate enough to reproduce the issue, it's probably better not including it at all.