Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

[sylmio]

Hello,

I had problems updating Gentoo using distcc. Many packages had errors with relocation asking to add option -fPIC, for example:

/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: mpn/.libs/modexact_1c_odd.o: warning: relocation against `__gmp_modlimb_invert_table' in read-only section `.text'
/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: mpn/.libs/modexact_1c_odd.o: relocation R_X86_64_PC32 against symbol `__gmp_modlimb_invert_table' can not be used when making a shared object; recompile with -fPIC

One can effectively force this option, but it does not work for all packages.,I ended up finding the root cause is having verbose mode of distcc (that is DISTCC_VERBOSE=1) that perturbes configure script of autoconf. Configure found -fPIC not working (as well as -c and -o due to same problem):

checking for x86_64-pc-linux-gnu-gcc option to produce PIC... -fPIC -DPIC
checking if x86_64-pc-linux-gnu-gcc PIC flag -fPIC -DPIC works... no
checking if x86_64-pc-linux-gnu-gcc static flag -static works... no
checking if x86_64-pc-linux-gnu-gcc supports -c -o file.o... no
checking if x86_64-pc-linux-gnu-gcc supports -c -o file.o... (cached) no

This is due to configure comparing error output of some compilation test. But due to some outputs of distcc in verbose mode, there are differences interpreted as errors. First distcc output used in configure as reference :

distcc[12000] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Oct 13 2021 16:15:47
distcc[12000] (dcc_recursion_safeguard) safeguard level=0
distcc[12000] (main) compiler name is "x86_64-pc-linux-gnu-g++"
distcc[12000] (dcc_set_path) setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib64/opencascade-7.5.2/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin
distcc[12000] (dcc_scan_args) scanning arguments: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_scan_args) found input file "conftest.cpp"
distcc[12000] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp
distcc[12000] (dcc_get_hostlist) not reading /home/sylvain/.distcc/hosts: No such file or directory
distcc[12000] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[12000] (dcc_parse_hosts) found tcp token "10.42.0.2/24,cpp,lzo"
distcc[12000] (dcc_parse_options) got CPP option
distcc[12000] (dcc_parse_options) got LZO option
distcc[12000] (dcc_parse_hosts) found tcp token "10.42.0.1/12,cpp,lzo"
distcc[12000] (dcc_parse_options) got CPP option
distcc[12000] (dcc_parse_options) got LZO option
distcc[12000] (dcc_lock_host) got cpu lock on localhost slot 0 as fd3
distcc[12000] exec on localhost: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_note_state) note state 5, file "conftest.cpp", host "localhost"
distcc[12000] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_spawn_child) child started as pid12001
distcc[12001] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12000] (dcc_collect_child) cc child 12001 terminated with status 0
distcc[12000] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[12000] compile conftest.cpp on localhost completed ok
distcc[12000] (dcc_unlock) release lock fd3
distcc[12000] elapsed compilation time 0.009451s
distcc[12000] (dcc_exit) exit: code 0; self: 0.000721 user 0.000000 sys; children: 0.007215 user 0.002086 sys
distcc[12000] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files

Second output of distcc in configure to test -fPIC:

distcc[12114] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Oct 13 2021 16:15:47
distcc[12114] (dcc_recursion_safeguard) safeguard level=0
distcc[12114] (main) compiler name is "x86_64-pc-linux-gnu-g++"
distcc[12114] (dcc_set_path) setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib64/opencascade-7.5.2/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin
distcc[12114] (dcc_scan_args) scanning arguments: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_scan_args) found input file "conftest.cpp"
distcc[12114] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp
distcc[12114] (dcc_get_hostlist) not reading /home/sylvain/.distcc/hosts: No such file or directory
distcc[12114] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[12114] (dcc_parse_hosts) found tcp token "10.42.0.2/24,cpp,lzo"
distcc[12114] (dcc_parse_options) got CPP option
distcc[12114] (dcc_parse_options) got LZO option
distcc[12114] (dcc_parse_hosts) found tcp token "10.42.0.1/12,cpp,lzo"
distcc[12114] (dcc_parse_options) got CPP option
distcc[12114] (dcc_parse_options) got LZO option
distcc[12114] (dcc_lock_host) got cpu lock on localhost slot 0 as fd3
distcc[12114] exec on localhost: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_note_state) note state 5, file "conftest.cpp", host "localhost"
distcc[12114] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_spawn_child) child started as pid12115
distcc[12115] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12114] (dcc_collect_child) cc child 12115 terminated with status 0
distcc[12114] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[12114] compile conftest.cpp on localhost completed ok
distcc[12114] (dcc_unlock) release lock fd3
distcc[12114] elapsed compilation time 0.010154s
distcc[12114] (dcc_exit) exit: code 0; self: 0.000000 user 0.000742 sys; children: 0.005493 user 0.003699 sys
distcc[12114] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files

configure script expect same error output. If possible it would be better to output all this debug information of distcc somewhere else, as they are not errors. I am not familiar with dealing with different outputs, so I am not sure it is a root problem of distcc or autoconf.

Anyway I think it is important, as the only workaround is to remove verbose mode of distcc. And then it is not possible to know what happens with distcc for packages using autoconf.

Best regards,

Sylvain

[sylmio]

I forgot to say that I use: distcc 3.4 x86_64-pc-linux-gnu autoconf (GNU Autoconf) 2.69

[shawnl]

Those distcc logs have no errors.

distcc[12000] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp

I don't see the bug. Because you are a Gentoo user, I suspect the problem is due to ricing, and has nothing to do with distcc.

[sylmio]

Strictly speaking there is no bug in distcc. But the outputs in verbose mode perturbs autoconf. So I think there is a bug when combining autoconf+distcc with DISTCC_VERBOSE=1.

You can reproduce the problem by launching the configure script of a package and seeing that it finds -fPIC does not work, and then not using it when it should (for shared libraries for example). It is not due to Gentoo system, and I suppose it will happen when compiling a package whatever the linux distribution.

Since distcc detects it is called by autoconf, pb could be solved by reducing output to reproducible one during autoconf (removing instance number, computation time). Or more simply, remove all outputs during autoconf., and add an additional verbose level if ones need output during autoconf with DISTCC_VERBOSE=2 or DISTCC_VERBOSE_AUTOCONF=1.

[shawnl]

DISTCC_VERBOSE=1 is really verbose, and you shouldn't be setting it globally.

El vie., 15 oct. 2021 15:23, sylmio @.***> escribió:

Strictly speaking there is no bug in distcc. But the outputs in verbose mode perturbs autoconf. So I think there is a bug when combining autoconf+distcc with DISTCC_VERBOSE=1.

You can reproduce the problem by launching the configure script of a package and seeing that it finds -fPIC does not work, and then not using it when it should (for shared libraries for example). It is not due to Gentoo system, and I suppose it will happen when compiling a package whatever the linux distribution.

Since distcc detects it is called by autoconf, pb could be solved by reducing output to reproducible one during autoconf (removing instance number, computation time). Or more simply, remove all outputs during autoconf., and add an additional verbose level if ones need output during autoconf with DISTCC_VERBOSE=2 or DISTCC_VERBOSE_AUTOCONF=1.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/distcc/distcc/issues/441#issuecomment-944220933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4W4WF3FYPTWNUXYORCPLUHAFKLANCNFSM5F7IQ5QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sylmio]

Ok, I understand. So how ones have to get output without perturbing build process ?

[shawnl]

What problem are you trying to solve?

[sylmio]

I wanted to get some output of distcc to check if it was correctly dispatching the work to other computers. I found info with DISTCC_VERBOSE=1 ok for this. I did not found distccmon-text and distccmon-gui stisfying, as many dispatching did not appear, I suppose the time of compilation being too small compared to refresh rate.

[shawnl]

check out the log on the server