search

subgraph / subgraph-os-issues

Subgraph OS issues repository
73 stars 10 forks source link

Create test kernel patched w/wireguard #274

Open dma opened 6 years ago

kibal commented 6 years ago

We will want to follow the instructions for the experimental Debian GNU/Linux packages: https://www.wireguard.com/install/

kibal commented 6 years ago

Attempting to install wireguard with the Debian dkms package results in the following error:

apt-get install wireguard-dkms
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  wireguard-dkms
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/120 kB of archives.
After this operation, 785 kB of additional disk space will be used.
Selecting previously unselected package wireguard-dkms.
(Reading database ... 319261 files and directories currently installed.)
Preparing to unpack .../wireguard-dkms_0.0.20170810-1_all.deb ...
Unpacking wireguard-dkms (0.0.20170810-1) ...
Setting up wireguard-dkms (0.0.20170810-1) ...
Loading new wireguard-0.0.20170810-1 DKMS files...
Building for 4.9.28-subgraph
Building initial module for 4.9.28-subgraph
Error! Bad return status for module build on kernel: 4.9.28-subgraph (x86_64)
Consult /var/lib/dkms/wireguard/0.0.20170810-1/build/make.log for more information.

Corresponding log file:

DKMS make.log for wireguard-0.0.20170810-1 for kernel 4.9.28-subgraph (x86_64)
Tue Sep  5 14:14:58 UTC 2017
make: Entering directory '/usr/src/linux-headers-4.9.28-subgraph'
  LD      /var/lib/dkms/wireguard/0.0.20170810-1/build/built-in.o
  CC [M]  /var/lib/dkms/wireguard/0.0.20170810-1/build/main.o
  CC [M]  /var/lib/dkms/wireguard/0.0.20170810-1/build/noise.o
  CC [M]  /var/lib/dkms/wireguard/0.0.20170810-1/build/device.o
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/constify_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/colorize_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/rap_plugin/rap_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/kernexec_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/size_overflow_plugin/size_overflow_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
cc1: error: incompatible gcc/plugin versions
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/constify_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/constify_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/colorize_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/colorize_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/rap_plugin/rap_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/rap_plugin/rap_plugin.so
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/kernexec_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/kernexec_plugin.so
  CC [M]  /var/lib/dkms/wireguard/0.0.20170810-1/build/peer.o
scripts/Makefile.build:293: recipe for target '/var/lib/dkms/wireguard/0.0.20170810-1/build/main.o' failed
make[1]: *** [/var/lib/dkms/wireguard/0.0.20170810-1/build/main.o] Error 1
make[1]: *** Waiting for unfinished jobs....
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/size_overflow_plugin/size_overflow_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
cc1: error: incompatible gcc/plugin versions
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/size_overflow_plugin/size_overflow_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
cc1: error: incompatible gcc/plugin versions
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
scripts/Makefile.build:293: recipe for target '/var/lib/dkms/wireguard/0.0.20170810-1/build/device.o' failed
make[1]: *** [/var/lib/dkms/wireguard/0.0.20170810-1/build/device.o] Error 1
scripts/Makefile.build:293: recipe for target '/var/lib/dkms/wireguard/0.0.20170810-1/build/noise.o' failed
make[1]: *** [/var/lib/dkms/wireguard/0.0.20170810-1/build/noise.o] Error 1
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/constify_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/colorize_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/rap_plugin/rap_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/kernexec_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/size_overflow_plugin/size_overflow_plugin.so
cc1: error: incompatible gcc/plugin versions: need 6.3.0 20170425   but have 6.3.0 20170516  
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/latent_entropy_plugin.so
cc1: error: incompatible gcc/plugin versions
cc1: error: fail to initialize plugin ./scripts/gcc-plugins/randomize_layout_plugin.so
scripts/Makefile.build:293: recipe for target '/var/lib/dkms/wireguard/0.0.20170810-1/build/peer.o' failed
make[1]: *** [/var/lib/dkms/wireguard/0.0.20170810-1/build/peer.o] Error 1
Makefile:1495: recipe for target '_module_/var/lib/dkms/wireguard/0.0.20170810-1/build' failed
make: *** [_module_/var/lib/dkms/wireguard/0.0.20170810-1/build] Error 2
make: Leaving directory '/usr/src/linux-headers-4.9.28-subgraph'
kibal commented 6 years ago

The issue was easily solved by installing the 4.9.33 kernel package and wireguard dkms installs and builds as expected:

The following NEW packages will be installed:
  wireguard-dkms
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/120 kB of archives.
After this operation, 785 kB of additional disk space will be used.
Selecting previously unselected package wireguard-dkms.
(Reading database ... 321932 files and directories currently installed.)
Preparing to unpack .../wireguard-dkms_0.0.20170810-1_all.deb ...
Unpacking wireguard-dkms (0.0.20170810-1) ...
Setting up wireguard-dkms (0.0.20170810-1) ...
Loading new wireguard-0.0.20170810-1 DKMS files...
Building for 4.9.33-subgraph
Building initial module for 4.9.33-subgraph
Done.

wireguard:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.9.33-subgraph/kernel/net/

depmod...

DKMS: install completed.

A subsequent modprobe of the wireguard module succeeds and it should be added to /etc/modules to load on boot.

kibal commented 6 years ago

Install wireguard exactly as documented on the install page:

# Commands as documented on https://www.wireguard.com/install/
echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable-wireguard.list
printf 'Package: *\nPin: release a=unstable\nPin-Priority: 150\n' > /etc/apt/preferences.d/limit-unstable
apt update
apt install wireguard-dkms wireguard-tools

In the future we should include both of these packages in the subgraph repos and for now this experimental setup process should only be used by people willing to jump through hoops.

After install is finished and the wireguard kernel module is loaded, it is possible to create wireguard interfaces. Using wg-quick one may easily setup a wireguard interface (wg0).

Generate your wg0 private and public key:

wg genkey > /etc/wireguard/$HOSTNAME.private
wg pubkey < /etc/wireguard/$HOSTNAME.private > /etc/wireguard/$HOSTNAME.public

Share the public key (/etc/wireguard/$HOSTNAME.public) file with your VPN provider or relevant peer.

Make a configuration for your wg0 interface that contains your private key and at least one peer. The following example configuration will configure your wg0 interface to have an IP address of 10.3.2.6/24 and assumes that the VPN peer will forward packets and be reachable at 10.3.2.1/24 inside of the wg0 tunnel:

cat << EOF > /etc/wireguard/wg0.conf
[Interface]
FwMark: 0xca6c
ListenPort = 1234
PrivateKey = YOUR PRIVATE KEY FROM /etc/wireguard/$HOSTNAME.private
PreUp = ip route add $(ip route get YOUR_VPN_SERVER_PUBLIC_IP  | grep via)
PostUp = ip address add 10.3.2.6/24 dev wg0 && ip route add 0/1 dev wg0 via 10.3.2.1 && ip route add 128/1 dev wg0 via 10.3.2.1
PostDown = ip route del $(ip route get YOUR_VPN_SERVER_PUBLIC_IP | grep via)

[Peer]
Endpoint = YOUR_VPN_SERVER_PUBLIC_IP:SERVER_PORT
PublicKey = SERVERPUBLICKEY
AllowedIPs = 0.0.0.0/0
EOF

Fix the configuration file to correctly match the information of your peer. Run wg-quick to enable it:

wg-quick up wg0

To see the status of the connection, use the wg tool:

wg

The wg tool should output information about your current peers:

interface: wg0
  public key: sI+8Z2C/nVKSKSsJKpaL57muvpB4eJRHaSPm2qlKJlc=
  private key: (hidden)
  listening port: 1234
  fwmark: 0xca6c

peer: CWIZXq8BHMp1mhNBuRSrsxpllxscxhmt0aRbtFHeyCI=
  endpoint: [redacted]:1234
  allowed ips: 0.0.0.0/0

Similarly one may run wg-quick to disable the interface:

wg-quick down wg0

We will need to modify our firewall configurations to ensure this is useful as well as to consider details for oz integration.

kibal commented 6 years ago

To enable the use of wg-quick it is possible to add a configuration file for ferm in /etc/ferm/ferm.d/:

cat << EOF > /etc/ferm/ferm.d/40-wireguard.conf
# -*- mode: conf[space] -*-
#
#  Configuration file for ferm(1).
#

@def $WIREGUARD = (`test -f /usr/bin/wg && echo 1 || echo`); 
# This requires at one [Interface] defined with FwMark = 0xsomevalue
@def $WIREGUARDMARK = (`grep -i fwmark /etc/wireguard/*.conf|cut -d= -f2|| echo`);

@if $WIREGUARD {
    @if $WIREGUARDMARK {
        domain ip {
            table nat {
            chain OUTPUT {
                policy ACCEPT;
                mod mark mark $WIREGUARDMARK ACCEPT;
            }
            }

            table filter {
            chain INPUT {
                policy DROP;
                mod mark mark $WIREGUARDMARK ACCEPT;
                interface wg+ ACCEPT;
            }

            chain OUTPUT {
                policy DROP;
                mod mark mark $WIREGUARDMARK ACCEPT;
                outerface wg+ ACCEPT;
            }
            }
        }
    }
}
EOF
kibal commented 6 years ago

I have verified that the ferm configuration is functional. When the wg0 is interface is active, only local traffic (dhcp, lan exception, etc.) and wireguard packets encrypted to the configured peer(s) leave the system. When wg0 is not active, the system behaves as normal. If the wireguard tools are not installed, the policy for wireguard interfaces will not be loaded. This should be audited by @mckinney-subgraph, @xSmurf or @dma before we encourage others to use it.

The next step is to integrate wireguard with oz such that an oz container will bridge with the above wg0 interface.

DarkCoridor commented 6 years ago

I got this working on a basic Debian Stretch (non-SGOS) machine. The only detail left out that tripped me up is perms for wg0.conf file needed to be set to:

chmod 750 /etc/wireguard/wg0.conf
kibal commented 6 years ago

We could add native Wireguard support to OZ with only a few setup commands before oz container. An example proof of concept follows on this comment.

Reconfigure ferm for the following experiment:

/etc/init.d/ferm stop
cat << 'EOF' > /etc/ferm/ferm.d/40-wireguard.conf
# -*- mode: conf[space] -*-
#
#  Configuration file for ferm(1).
#

@def $WIREGUARDMARK = (0xca6c);
        domain ip {
            table nat {
            chain OUTPUT {
                policy ACCEPT;
                mod mark mark $WIREGUARDMARK ACCEPT;
            }
            }

            table filter {
            chain INPUT {
                policy DROP;
                mod mark mark $WIREGUARDMARK ACCEPT;
                interface wg+ ACCEPT;
            }

            chain OUTPUT {
                policy DROP;
                mod mark mark $WIREGUARDMARK ACCEPT;
                outerface wg+ ACCEPT;
            }
            }
        }
EOF

Restart ferm:

/etc/init.d/ferm restart

Modify the json profile for transmission to have networking type none and oz-init will create only lo at oz-init time:

{
"name": "transmission-gtk"
, "path": "/usr/bin/transmission-gtk"
, "xserver": {
    "enabled": true
    , "enable_tray": false
    , "tray_icon":"/usr/share/pixmaps/transmission.xpm"
    , "enable_notifications": true
}
, "networking":{
    "type":"none"
}
, "whitelist": [
    {"path":"${HOME}/.config/transmission-vpn/", "target":"${HOME}/.config/transmission/","can_create":true}
    ,{"path":"/var/lib/oz/cells.d/transmission-whitelist.seccomp", "read_only":true}
        ,{"path":"/var/lib/oz/openvpn/PIA/pia-resolv.conf", "target": "/run/resolvconf/resolv.conf", "force": true, "read_only":true}
]
, "blacklist": [
]
, "environment": [
]
, "shared_folders": [
        "${XDG_DOWNLOAD_DIR}"
        ,"${XDG_DESKTOP_DIR}"
]
, "seccomp": {
    "mode":"whitelist"
    , "whitelist":"/var/lib/oz/cells.d/transmission-whitelist.seccomp"
    , "enforce": true
    , "debug": false}
}

Launch transmission-gtk:

transmission-gtk

Create a wg0 interface and a namespace name associated with the corresponding oz-init pid from a normal root shell:

OZINITPID=`pidof oz-init`
ln -s  /proc/$OZINITPID/ns/net /var/run/netns/oz-torrent
ip link add wg0 type wireguard
ip link set wg0 netns oz-torrent

Inside of the oz sandbox for transmission-gtk there should now be lo and wg0 interfaces:

ip netns exec oz-torrent ip link show

Configure wg0 and set it as the default route:

ip netns exec oz-torrent wg setconf wg0 /etc/wireguard/wg0.conf
ip netns exec oz-torrent ip addr add 10.3.2.4/32 dev wg0
ip netns exec oz-torrent ip link set up dev wg0
ip netns exec oz-torrent ip route add default dev wg0

As root in the console it is possible to inspect the status with the usual Wireguard tools:

ip netns exec oz-torrent wg

To confirm your IP address as routed through the oz-torrent network namespace run the following dig command:

ip netns exec oz-torrent dig +short myip.opendns.com @208.67.222.222

This is functional and extremely stable. It would be simpler if OZ created a network namespace per sandbox and then executed ip netns exec oz-torrent $command at the oz-init time.