search

trailofbits / algo

Set up a personal VPN in the cloud
https://blog.trailofbits.com/2016/12/12/meet-algo-the-vpn-that-works/
GNU Affero General Public License v3.0
28.93k stars 2.32k forks source link

iptables rules.v6 has missing IPv6 egress IP #1734

Closed anna-is-cute closed 4 years ago

anna-is-cute commented 4 years ago

Describe the bug

When deploying algo to an existing Ubuntu 19.10 or 18.04 LTS fresh install on Linode, /etc/iptables/rules.v6 ends up containing this snippet:

# ...
:POSTROUTING ACCEPT [0:0]

# Allow traffic from the VPN network to the outside world, and replies
-A POSTROUTING -s fd9d:bc11:4020::/48,fd9d:bc11:4021::/48 -m policy --pol none --dir out -j SNAT --to

COMMIT

Notice how the egress IP is missing after --to. The box has working IPv6 connectivity.

To Reproduce

Steps to reproduce the behavior:

  1. Deploy a Linode using the nanode plan in their Canadian data centre.
  2. Attempt to deploy algo using option 11 (pre-existing server)
  3. Accept all defaults except turn on DNS ad block and SSH tunnelling.

Expected behavior

Successful deploy with /etc/iptables/rules.v6 containing an egress IP for --to.

Full log

(.env) ~/code/newalgo master (David-VII:jkcclemens)
% ./algo --private-key="$HOME"/.ssh/algo                                                                  ret: 2
 [WARNING]: Could not match supplied host pattern, ignoring: vpn-host

PLAY [localhost] ************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************
ok: [localhost]

TASK [Playbook dir stat] ****************************************************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ********************************************
ok: [localhost] => {
    "changed": false,
        "msg": "All assertions passed"
        }

TASK [Ensure the requirements installed] ************************************************************************
ok: [localhost]

TASK [Set required ansible version as a fact] *******************************************************************
ok: [localhost] => (item=ansible==2.8.3)

TASK [Verify Python meets Algo VPN requirements] ****************************************************************
ok: [localhost] => {
    "changed": false,
        "msg": "All assertions passed"
        }

TASK [Verify Ansible meets Algo VPN requirements] ***************************************************************
ok: [localhost] => {
    "changed": false,
        "msg": "All assertions passed"
        }

PLAY [Ask user for the input] ***********************************************************************************

TASK [Gathering Facts] ******************************************************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
    1. DigitalOcean
        2. Amazon Lightsail
            3. Amazon EC2
                4. Microsoft Azure
                    5. Google Compute Engine
                        6. Hetzner Cloud
                            7. Vultr
                                8. Scaleway
                                    9. OpenStack (DreamCompute optimised)
                                        10. CloudStack (Exoscale optimised)
                                            11. Install to existing Ubuntu 18.04 or 19.10 server (for more advanced users)

Enter the number of your desired provider
:

TASK [Cloud prompt] *********************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] *****************************************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:

TASK [Cellular On Demand prompt] ********************************************************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:

TASK [Wi-Fi On Demand prompt] ***********************************************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:

TASK [Retain the PKI prompt] ************************************************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:

TASK [DNS adblocking prompt] ************************************************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:

TASK [SSH tunneling prompt] *************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] *****************************************************************************
ok: [localhost]

PLAY [Provision the server] *************************************************************************************

TASK [Gathering Facts] ******************************************************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Fedora 31 (Thirty One)
Created from git clone. Last commit: 3f3138f Fix IPsec DNS when WireGuard uses port 53 (#1719)
Python 3.7.6
Runtime variables:
    algo_provider "local"
        algo_ondemand_cellular "False"
            algo_ondemand_wifi "False"
                algo_ondemand_wifi_exclude "X251bGw="
                    algo_dns_adblocking "True"
                        algo_ssh_tunneling "True"
                            wireguard_enabled "True"
                                dns_encryption "True"

TASK [Display the invocation environment] ***********************************************************************
changed: [localhost -> localhost]

TASK [Install the requirements] *********************************************************************************
ok: [localhost -> localhost]
[local : pause]
Enter the IP address of your server: (or use localhost for local installation):
[localhost]
:

TASK [local : pause] ********************************************************************************************
ok: [localhost]

TASK [local : Set the facts] ************************************************************************************
ok: [localhost]
[local : pause]
What user should we use to login on the server? (note: passwordless login required, or ignore if you're deploying to localhost)
[root]
:

TASK [local : pause] ********************************************************************************************
ok: [localhost]

TASK [local : Set the facts] ************************************************************************************
ok: [localhost]
[local : pause]
Enter the public IP address or domain name of your server: (IMPORTANT! This is used to verify the certificate)
[xxx.xxx.xxx.xxx]
:

TASK [local : pause] ********************************************************************************************
ok: [localhost]

TASK [local : Set the facts] ************************************************************************************
ok: [localhost]

TASK [Set subjectAltName as a fact] *****************************************************************************
ok: [localhost]

TASK [Add the server to an inventory group] *********************************************************************
changed: [localhost]

TASK [Wait until SSH becomes ready...] **************************************************************************
ok: [localhost]

TASK [Linux | set OS specific facts] ****************************************************************************
ok: [localhost]

TASK [Set config paths as facts] ********************************************************************************
ok: [localhost]

TASK [Update config paths] **************************************************************************************
changed: [localhost]

TASK [debug] ****************************************************************************************************
ok: [localhost] => {
    "IP_subject_alt_name": "redacted.hostname"
    }

TASK [Wait 600 seconds for target connection to become reachable/usable] ****************************************
ok: [localhost -> xxx.xxx.xxx.xxx] => (item=xxx.xxx.xxx.xxx)

PLAY [Configure the server and install required software] *******************************************************

TASK [Ensure the config directory exists] ***********************************************************************
ok: [xxx.xxx.xxx.xxx -> localhost]

TASK [Dump the ssh config] **************************************************************************************
ok: [xxx.xxx.xxx.xxx -> localhost]

TASK [common : Check the system] ********************************************************************************
ok: [xxx.xxx.xxx.xxx]
included: /home/jkcclemens/code/newalgo/roles/common/tasks/ubuntu.yml for xxx.xxx.xxx.xxx

TASK [common : Gather facts] ************************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Install unattended-upgrades] *********************************************************************
 [WARNING]: Could not find aptitude. Using apt-get instead

ok: [xxx.xxx.xxx.xxx]

TASK [common : Configure unattended-upgrades] *******************************************************************
changed: [xxx.xxx.xxx.xxx]

TASK [common : Periodic upgrades configured] ********************************************************************
changed: [xxx.xxx.xxx.xxx]

TASK [common : Disable MOTD on login and SSHD] ******************************************************************
changed: [xxx.xxx.xxx.xxx] => (item={'regexp': '^session.*optional.*pam_motd.so.*', 'line': '# MOTD DISABLED', 'file': '/etc/pam.d/login'})
changed: [xxx.xxx.xxx.xxx] => (item={'regexp': '^session.*optional.*pam_motd.so.*', 'line': '# MOTD DISABLED', 'file': '/etc/pam.d/sshd'})

TASK [common : Loopback for services configured] ****************************************************************
changed: [xxx.xxx.xxx.xxx]

TASK [common : systemd services enabled and started] ************************************************************
ok: [xxx.xxx.xxx.xxx] => (item=systemd-networkd)
ok: [xxx.xxx.xxx.xxx] => (item=systemd-resolved)

RUNNING HANDLER [common : restart systemd-networkd] *************************************************************
changed: [xxx.xxx.xxx.xxx]

TASK [common : Check apparmor support] **************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Set fact if apparmor enabled] ********************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Define facts] ************************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Set facts] ***************************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Set IPv6 support as a fact] **********************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Check size of MTU] *******************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Set OS specific facts] ***************************************************************************
ok: [xxx.xxx.xxx.xxx]

TASK [common : Install tools] ***********************************************************************************
changed: [xxx.xxx.xxx.xxx]

TASK [common : Install headers] *********************************************************************************
ok: [xxx.xxx.xxx.xxx]
included: /home/jkcclemens/code/newalgo/roles/common/tasks/iptables.yml for xxx.xxx.xxx.xxx

TASK [common : Iptables configured] *****************************************************************************
changed: [xxx.xxx.xxx.xxx] => (item={'src': 'rules.v4.j2', 'dest': '/etc/iptables/rules.v4'})

TASK [common : Iptables configured] *****************************************************************************
changed: [xxx.xxx.xxx.xxx] => (item={'src': 'rules.v6.j2', 'dest': '/etc/iptables/rules.v6'})

TASK [common : Sysctl tuning] ***********************************************************************************
changed: [xxx.xxx.xxx.xxx] => (item={'item': 'net.ipv4.ip_forward', 'value': 1})
changed: [xxx.xxx.xxx.xxx] => (item={'item': 'net.ipv4.conf.all.forwarding', 'value': 1})
changed: [xxx.xxx.xxx.xxx] => (item={'item': 'net.ipv6.conf.all.forwarding', 'value': 1})

RUNNING HANDLER [common : restart iptables] *********************************************************************
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"changed": false, "msg": "Unable to restart service netfilter-persistent: Job for netfilter-persistent.service failed because the control process exited with error code.\nSee \"systemctl status netfilter-persistent.service\" and \"journalctl -xe\" for details.\n"}
included: /home/jkcclemens/code/newalgo/playbooks/rescue.yml for xxx.xxx.xxx.xxx

TASK [debug] ****************************************************************************************************
ok: [xxx.xxx.xxx.xxx] => {
    "fail_hint": [
            "Sorry, but something went wrong!",
                    "Please check the troubleshooting guide.",
                            "https://trailofbits.github.io/algo/troubleshooting.html"
                                ]
                                }

TASK [Fail the installation] ************************************************************************************
fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP ******************************************************************************************************
xxx.xxx.xxx.xxx               : ok=27   changed=9    unreachable=0    failed=1    skipped=7    rescued=1    ignored=0
localhost                  : ok=33   changed=3    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0

(.env) ~/code/newalgo master (David-VII:jkcclemens)
davidemyers commented 4 years ago

I've seen this intermittently on other cloud providers as well.

joecool1029 commented 4 years ago

@davidemyers up to you how you want to do it, but I spent a lot of time figuring out the peculiarities of ipv6 on DO the past couple days. Maybe consider adding some of this comment should be added to troubleshooting, this area isn't my specialty but I'm certain people hit the issue. Commenting here because the first part is pertinent to this specific issue.

alternative_ingress_ip set to false leaves (for me) a broken ipv6 setup on DO. I have to switch the v6 rule from -j SNAT --to whatever to -j MASQUERADE to get working network through the server's ipv6. If this is set to true, things work as they should.

The resolution seems therefore to always use a MASQUERADE instead of SNAT even though (like NAT66) this is discouraged if the provider doesn't support this option or users on DO set/leave it to false. It would eliminate issues with providers that only give a single ipv6 address to their VPS's.

As for updating the troubleshooting page. I noticed under all circumstances my clients hard prefer ipv4 even when dual-stack ipv6 is working. There's a workaround for linux using glibc, but basically systems are hardcoded to realize a NAT66 setup usually means someone screwed up the network. This article had the best information I've seen on the issue: https://blog.apnic.net/2018/02/02/nat66-good-bad-ugly/

davidemyers commented 4 years ago

Algo used to use MASQUERADE for IPv6 before the alternative_ingress_ip feature was added. Jack has tagged this issue as a bug and I imagine he'll fix it by going back to using MASQUERADE when alternative_ingress_ip: false.