[IA] clang hangs with `-no-integrated-as -Wa,-`

[Quuxplusone]

Bugzilla Link	PR39410
Status	NEW
Importance	P enhancement
Reported by	Nick Desaulniers (ndesaulniers@google.com)
Reported on	2018-10-23 11:30:29 -0700
Last modified on	2018-10-24 08:58:39 -0700
Version	trunk
Hardware	PC Linux
CC	echristo@gmail.com, klimek@google.com, llozano@chromium.org, llvm-bugs@lists.llvm.org, llvm-dev@ndave.org, natechancellor@gmail.com, srhines@google.com
Fixed by commit(s)
Attachments	strace_clang_pipe_wa_noas.txt (74254 bytes, text/plain) strace_gcc_pipe_wa.txt (18757 bytes, text/plain)
Blocks
Blocked by
See also

From https://lkml.org/lkml/2018/10/23/136.

The linux kernel as of 4.20 for x86_64 uses the flag -Wa,- with -pipe. This causes the use of no-integrated-as to hang the build. I think -pipe is a red herring.

ie.

$ clang hello_world.c -no-integrated-as -Wa,-

can reproduce the issue. strace'ing with and without -Wa,-, I see:

with -Wa,-: ... access("/usr/bin/as", F_OK) = 0 ... wait4(167173,

w/o: -Wa,-: access("/usr/bin/as", F_OK) = 0 ... wait4(167996, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 167996 ... --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=167996, si_uid=366559, si_status=0, si_utime=0, si_stime=1} ---

so however clang is invoking the non-integrated-as, it's not receiving SIGCHLD in order to exit the wait4.

Looking at $ strace gcc hello_world.c -Wa,-

[Quuxplusone]

Attached strace_clang_pipe_wa_noas.txt (74254 bytes, text/plain): strace_clang_pipe_wa_noas.txt

[Quuxplusone]

Attached strace_gcc_pipe_wa.txt (18757 bytes, text/plain): strace_gcc_pipe_wa.txt

[Quuxplusone]

We run:

$ clang hello.c -pipe -Wa,- &
$ sudo cat /proc/`pgrep bin/as | tail -n1`/stack

[<0>] wait_woken+0x43/0x80
[<0>] n_tty_read+0x42f/0x880
[<0>] tty_read+0x7b/0xe0
[<0>] vfs_read+0x89/0x130
[<0>] ksys_read+0x52/0xc0
[<0>] do_syscall_64+0x55/0x110
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[<0>] 0xffffffffffffffff

so I assume the assembler is waiting maybe for an EOF to be written.  Let me
see if I can find that in the source of GAS.

[Quuxplusone]

Looks like EOF is meant for checking read values, but it itself is not writeable. It seems that closing a pipe is the better way to signal this?

I'm getting the feeling we should pick up -Wa,- in clang/lib/Driver/ToolChains/Gnu.cpp#tools::gnutools::Assembler::ConstructJob

and then somehow in llvm/lib/Support/Program.cpp#sys::ExecuteAndWait not do the wait. Or find a way to close the fd once we've finished all output?

[Quuxplusone]

It seems that clang will always write the assembly to a tempfile, then invoke the assembler on that temp file. So it seems that the only way for Clang to pipe the assembly into the assembler via stdout/stdin is if clang supported -pipe.

While Clang documents support for -pipe. https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-pipe Looks like Clang eats -pipe: https://github.com/llvm-mirror/clang/blob/391667a023f79287f9c40868f34f08c161555556/lib/Driver/Driver.cpp#L962 commit r110007 has the log: Driver: Start ripping out support for -pipe, which is worthless and complicates too many other things.

So maybe the right thing to do is to filter out -Wa,- from the CmdArgs for constructing the assembler job, maybe with a comment that that should be removed should -pipe ever be reimplemented?

[Quuxplusone]

Looks like -pipe -Wa,- doesn't buy us any speedup (maybe why -pipe was dropped from clang), so we'll just remove the use of them from the kernel. https://lkml.org/lkml/2018/10/24/111

Probably should still filter out -Wa,- and NOT pass it to the assembler.