distcc/crossdev are ignoring my desktop

[ghost]

I followed the guide and it's not working.

• distcc is running on my desktop and my pi
• crossdev works on my desktop
• I don't have a firewall But there is no connection happening between my desktop and my pi. I've been trying to emerge qutebrowser for like a week now and my pi just locked up. Is there a way I can just compile it on my desktop, then toss it on the pi somehow?

[sakaki-]

Hi @AshtonHarding, sorry to hear you are having trouble with this.

Some quick questions:

• Does distcc work between PCs on your network?
• Can you ping your PC from your Pi, and vice versa?
• On your desktop PC, what do you have in /etc/conf.d/distccd (if your PC uses OpenRC) or /etc/systemd/system/distccd.service.d/00gentoo.conf (if it uses systemd)?
• What do you have in /etc/distcc/hosts on your Pi?
• What does netstat -vatn show on your PC?
• On your PC, does grep -i distccd /var/log/messages (if using OpenRC) or journalctl --boot --unit=distccd (if using systemd) show that the distccd server started up correctly?
• You may be running out of memory on your Pi, which is why it is locking up; how many simultaneous emerge jobs / threads are you running (emerge --info on your Pi)?
• Do you have the same version of gcc on your pi and desktop?

If you cannot get distcc to work, another option (which I use regularly for large compiles) is to chroot into the Pi's filesystem on the PC, using QEMU user-mode / binfmt_misc. See my notes here.

[ghost]

Does distcc work between PCs on your network?

Yes, I used it on another computer to install gentoo on.

Can you ping your PC from your Pi, and vice versa?

Yes.

On your desktop PC, what do you have in /etc/conf.d/distccd (if your PC uses OpenRC) or /etc/systemd/system/distccd.service.d/00gentoo.conf (if it uses systemd)?

# /etc/conf.d/distccd: config file for /etc/init.d/distccd

DISTCCD_OPTS=""

# this is the distccd executable 
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd.pid"

# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3632"

# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level info --log-file /var/log/distcc/distccd.log"

# SECURITY NOTICE:
# It is HIGHLY recommended that you use the --listen option
# for increased security. You can specify an IP to permit connections 
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.1.0/24"
#DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.1.101"

# set this for niceness
# Default is 15
DISTCCD_OPTS="${DISTCCD_OPTS} -N 20"

What do you have in /etc/distcc/hosts on your Pi?

# --- /etc/distcc/hosts -----------------------
# See the "Hosts Specification" section of
# "man distcc" for the format of this file.
#
# By default, just test that it works in loopback mode.
#127.0.0.1
# Add a distccd server host like this:
# (NB the /10 means maximum 10 jobs at once for this host; it has
# nothing to do with the IP address)
192.168.1.103/10,lzo,cpp

What does netstat -vatn show on your PC?

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:3632            0.0.0.0:*               LISTEN     
tcp        0      0 192.168.1.101:43480     178.32.120.169:443      TIME_WAIT  
tcp        0      0 192.168.1.101:51244     192.30.253.124:443      ESTABLISHED
tcp        0      0 192.168.1.101:55650     151.101.180.159:443     ESTABLISHED
tcp        0      0 192.168.1.101:39770     184.26.40.42:443        ESTABLISHED
tcp        0      0 192.168.1.101:38414     143.204.158.84:443      ESTABLISHED
tcp        0      0 192.168.1.101:35486     172.217.2.228:443       ESTABLISHED
tcp        0      0 192.168.1.101:39768     184.26.40.42:443        ESTABLISHED
tcp        0      0 192.168.1.101:38272     91.121.41.140:443       ESTABLISHED
tcp        0      0 192.168.1.101:47968     104.16.58.5:443         TIME_WAIT  
tcp        0      0 192.168.1.101:33298     104.244.42.8:443        ESTABLISHED
tcp        0      0 192.168.1.101:42454     104.97.201.113:443      ESTABLISHED
tcp        0      0 192.168.1.101:33100     104.244.42.129:443      ESTABLISHED
tcp        0      0 192.168.1.101:36674     198.252.206.25:443      ESTABLISHED
tcp        0      0 192.168.1.101:59352     184.51.54.66:443        ESTABLISHED
tcp        0      0 192.168.1.101:42456     104.97.201.113:443      ESTABLISHED
tcp        0      0 192.168.1.101:36646     198.252.206.25:443      ESTABLISHED
tcp        0      0 192.168.1.101:42438     104.244.42.195:443      ESTABLISHED
tcp        0      0 192.168.1.101:40548     23.207.243.7:443        ESTABLISHED
tcp        0      0 192.168.1.101:33372     192.30.253.113:443      ESTABLISHED
tcp        0      0 192.168.1.101:35254     13.33.112.179:443       ESTABLISHED
tcp        0      0 192.168.1.101:46884     192.168.1.104:22        ESTABLISHED
tcp        0      0 192.168.1.101:58332     104.16.59.37:443        ESTABLISHED
tcp6       0      0 :::22                   :::*                    LISTEN 

On your PC, does grep -i distccd /var/log/messages (if using OpenRC) or journalctl --boot --unit=distccd (if using systemd) show that the distccd server started up correctly?

grep: /var/log/messages: No such file or directory NOTE: /var/log/distcc/distccd.log

distccd[3335] (dcc_setup_daemon_path) daemon's PATH is /usr/x86_64-pc-linux-gnu/gcc-bin/7.3.0:/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/lib/llvm/6/bin:/opt/bin
distccd[3335] (dcc_listen_by_addr) listening on 0.0.0.0:3632
distccd[3335] (dcc_standalone_server) 16 CPUs online on this server
distccd[3335] (dcc_standalone_server) allowing up to 18 active jobs
distccd[3336] (dcc_log_daemon_started) preforking daemon started (3.2rc1 x86_64-pc-linux-gnu, built Feb 13 2019 11:23:35)

NOTE1: One of the things I haven't found out how to do was swap the PATH for the daemon, but I couldn't find any answers anywhere so I just decided it probably wasn't that big of a deal.

You may be running out of memory on your Pi, which is why it is locking up; how many simultaneous emerge jobs / threads are you running (emerge --info on your Pi)? EMERGE_DEFAULT_OPTS="--jobs=2 --load-average=1"

Do you have the same version of gcc on your pi and desktop?

Yes.

I may just use chroot to deal with this, it seems like more hassle than it's worth at this point. I need to buy an sd card reader for my desktop, so I won't be able to inform anyone of how that worked for awhile.

[sakaki-]

OK, that all looks sane enough. Per these notes, try creating a simple test file on your RPi3, main.c:

#include <stdio.h>

int main() {
    printf("Hello distcc!\n");
    return 0;
}

Make sure compiling this locally on your RPi3 with gcc works first.

Then, run tail -f /var/log/distcc/distccd.log on your PC, then run on your RPi3, as root:

pi64 ~ # export DISTCC_VERBOSE=1
pi64 ~ # distcc gcc -c main.c -o main.o

Does this work? If not, what gets printed out on the Pi side? On the PC side? I've just tried it here, and it appears to work fine (with gcc 8.2.0 on both sides).

[ghost]

I'm dropping the distcc idea for now, I have to get my pi working asap. So, I'm just doing it from the ground up since chroot has been having problems as well. I've got multiple pis, so when I have time again, which should be around fourth quarter this year, I'll try it again and make sure to add what I learned so others won't have the same problem. Thank you for the help!

[sakaki-]

OK, sorry to hear it didn't work out for you ><

I have since built out a test server following the wiki instructions, and used it successfully from a 'vanilla' 1.3.1 image, so not quite sure what the underlying issue was here; be interested to hear the outcome of your work (once time permits).

Best, sakaki