separate single compile+link command to multiple steps

[GoogleCodeExporter]

distcc 3.2rc1 i686-pc-linux-gnu .

Running on up-to-date Arch Linux: 3.1.0-4-ARCH , gcc (GCC) 4.6.2 .

I was trying to transfer CPU load of the compilation process to more powerful 
host, called "letta":

$ distcc --show-hosts 
letta:3632

The remote host was running the same version of distcc[d]:

$ telnet letta 3632
Trying 193.219.43.130...
Connected to letta.
Escape character is '^]'.
get some stuff, ok?^M
Connection closed by foreign host.

Then I launched distcc, it built the file and returned 0.  Only it didn't pass 
compilation to the remote machine that ought to get it.

Compiler was run just on localhost, as shown here:

$ distccmon-text 

  5464  Compile     tg_007.c                              localhost[0]

Verbose log of the client:

$ DISTCC_VERBOSE=1 distcc cc -o tg_007.exe tg_007.c 
distcc[5464] (dcc_trace_version) distcc 3.2rc1 i686-pc-linux-gnu; built Nov  7 
2011 11:08:50
distcc[5464] (dcc_recursion_safeguard) safeguard level=0
distcc[5464] (main) compiler name is "distcc"
distcc[5464] (dcc_scan_args) scanning arguments: cc -o tg_007.exe tg_007.c
distcc[5464] (dcc_scan_args) found object/output file "tg_007.exe"
distcc[5464] (dcc_scan_args) found input file "tg_007.c"
distcc[5464] (dcc_scan_args) compiler apparently called not for compile
distcc[5464] (dcc_lock_host) got cpu lock on localhost slot 0 as fd3
distcc[5464] exec on localhost: cc -o tg_007.exe tg_007.c
distcc[5464] (dcc_note_state) note state 5, file "tg_007.c", host "localhost"
distcc[5464] (dcc_spawn_child) forking to execute: cc -o tg_007.exe tg_007.c
distcc[5464] (dcc_spawn_child) child started as pid5465
distcc[5465] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[5464] (dcc_collect_child) cc child 5465 terminated with status 0
distcc[5464] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 
minflt, 0 majflt
distcc[5464] compile tg_007.c on localhost completed ok
distcc[5464] (dcc_unlock) release lock fd3
distcc[5464] elapsed compilation time 0.400560s
distcc[5464] (dcc_exit) exit: code 0; self: 0.000000 user 0.003333 sys; 
children: 0.286647 user 0.053329 sys
distcc[5464] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files

Not a single packet was sent to the remote host.  I verified this via tcpdump.

There are nor error neither stdout messages generated during usual client run.

Original issue reported on code.google.com by sauli...@gmail.com on 13 Nov 2011 at 9:24

[GoogleCodeExporter]

This is "working as intended".

The relevant log message is this one:

   distcc[5464] (dcc_scan_args) compiler apparently called not for compile

Distcc only distributes compilation, not linking, but you have invoked the 
compiler in a mode in which it will do both compilation and linking. Distcc 
runs the command locally in such cases.

If you want to distribute the compilation, and run just the linking locally, 
then you need to split your command into two steps, one for compilation, and 
one for linking; then distcc will be able to distribute the compilation command 
(but will still run the linking command locally).

  # Compile
  DISTCC_VERBOSE=1 distcc cc -c tg_007.c 

  # Link
  DISTCC_VERBOSE=1 distcc cc -o tg_007.exe tg_007.o

Original comment by fergus.h...@gmail.com on 14 Nov 2011 at 7:50

• Changed state: Invalid

[GoogleCodeExporter]

Thank you for the detailed explanation, I missed the linking step due to my 
win32-based past where linker was standalone application:)

I guess some of these bits could go into docs.

Original comment by sauli...@gmail.com on 14 Nov 2011 at 10:01

[GoogleCodeExporter]

I'd say this would then be a feature request: I'd like distcc to be able to do 
that splitting of compile+link command to compile remotely and link locally 
automatically.
I have a quite low-powered device (OMAP3530) running distcc and a single 
high-powered "server" (AMD Phenom Quad Core) with a cross toolchain for the 
OMAP3530. Then I expected all compilation to be remote and said pump make -j6 
(building gcc). ... And the omap was running 6 conftest compilations locally.

Original comment by urja...@gmail.com on 13 Jun 2013 at 11:25

[GoogleCodeExporter]

irjaman writes:
> I'd like distcc to be able to do that splitting of
> compile+link command to compile remotely and link
> locally automatically.

Perhaps, although this is inferring quite a lot and distcc traditionally takes 
an appropriately conservative approach.  That is, distcc only does what the 
user asks, and single step compile and link is very different to multiple 
steps, as the single step may perform e.g. much more optimization.

Implicitly splitting such commands should only be done if it can be proved that 
the results would not significantly differ.  The whole point of distcc is to 
insert it in a toolchain without any impact other than distributing compile 
tasks.

For now, urjaman, I suggest you split your compiles as indicated in the 
previous comments.

I reopen the task to be investigated for impact and feasability.  The previous 
comments do not indicate anything about that.

Original comment by mand...@gmail.com on 14 Jun 2013 at 1:56

• Changed state: New
• Added labels: Type-Enhancement

[GoogleCodeExporter]

Original comment by mand...@gmail.com on 14 Jun 2013 at 1:57

• Changed title: separate single compile+link command to multiple steps