search

trailofbits / algo

Set up a personal VPN in the cloud
https://blog.trailofbits.com/2016/12/12/meet-algo-the-vpn-that-works/
GNU Affero General Public License v3.0
28.96k stars 2.32k forks source link

Error deploying to Digital Ocean using freebsd #1206

Closed fgsch closed 5 years ago

fgsch commented 5 years ago

Describe the bug

Failed to deploy to DO using image "freebsd-11-2-x64-zfs".

To Reproduce

Steps to reproduce the behavior:

  1. Use image freebsd-11-2-x64-zfs for digitalocean provider in config.cfg
  2. Try deploying to DO.

Expected behavior

Deployment succeeded.

Additional context

Looks like the strongswan user does not exist in FreeBSD.

Full log


PLAY [Ask user for the input] *****************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************
ok: [localhost]
[pause]
What provider would you like to use?
    1. DigitalOcean
    2. Amazon Lightsail
    3. Amazon EC2
    4. Vultr
    5. Microsoft Azure
    6. Google Compute Engine
    7. Scaleway
    8. OpenStack (DreamCompute optimised)
    9. Install to existing Ubuntu 18.04 server (Advanced)

Enter the number of your desired provider
:
1

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] ***********************************************************************************************************************************
ok: [localhost]
[pause]
Name the vpn server
[algo]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want the VPN to support Windows 10 or Linux Desktop clients? (enables compatible ciphers and key exchange, less secure)
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]
[pause]
Do you want to retain the CA key? (required to add users in the future, but less secure)
[y/N]
:

TASK [pause] **********************************************************************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] ***********************************************************************************************************************************
ok: [localhost]

PLAY [Provision the server] *******************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Alpine Linux v3.8 (Virtualized: docker)
ZIP file created: 2018-11-17 19:38:23.000000000
Python 2.7.15
Runtime variables:
    algo_provider "digitalocean"
    algo_ondemand_cellular "False"
    algo_ondemand_wifi "False"
    algo_ondemand_wifi_exclude "_null"
    algo_local_dns "False"
    algo_ssh_tunneling "False"
    algo_windows "False"
    wireguard_enabled "True"
    dns_encryption "True"

TASK [Display the invocation environment] *****************************************************************************************************************************
changed: [localhost -> localhost]

TASK [Generate the SSH private key] ***********************************************************************************************************************************
changed: [localhost]

TASK [Generate the SSH public key] ************************************************************************************************************************************
changed: [localhost]
[cloud-digitalocean : pause]
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
 (output is hidden):

TASK [cloud-digitalocean : pause] *************************************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Set the token as a fact] *******************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Get regions] *******************************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Set facts about thre regions] **************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Set default region] ************************************************************************************************************************
ok: [localhost]
[cloud-digitalocean : pause]
What region should the server be located in?
    1. ams3     Amsterdam 3
    2. blr1     Bangalore 1
    3. fra1     Frankfurt 1
    4. lon1     London 1
    5. nyc1     New York 1
    6. nyc3     New York 3
    7. sfo2     San Francisco 2
    8. sgp1     Singapore 1
    9. tor1     Toronto 1

Enter the number of your desired region
[6]
:
<REDACTED REGION>

TASK [cloud-digitalocean : pause] *************************************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Set additional facts] **********************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Delete the existing Algo SSH keys] *********************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Upload the SSH key] ************************************************************************************************************************
changed: [localhost]

TASK [cloud-digitalocean : Creating a droplet...] *********************************************************************************************************************
changed: [localhost]

TASK [cloud-digitalocean : set_fact] **********************************************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Tag the droplet] ***************************************************************************************************************************
changed: [localhost]
FAILED - RETRYING: Delete the new Algo SSH key (10 retries left).

TASK [cloud-digitalocean : Delete the new Algo SSH key] ***************************************************************************************************************
ok: [localhost]

TASK [Set subjectAltName as afact] ************************************************************************************************************************************
ok: [localhost]

TASK [Add the server to an inventory group] ***************************************************************************************************************************
changed: [localhost]

TASK [Additional variables for the server] ****************************************************************************************************************************
changed: [localhost]

TASK [Wait until SSH becomes ready...] ********************************************************************************************************************************
ok: [localhost]

TASK [debug] **********************************************************************************************************************************************************
ok: [localhost] => {
    "IP_subject_alt_name": "<REDACTED IP>"
}
Pausing for 20 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)

TASK [A short pause, in order to be sure the instance is ready] *******************************************************************************************************
ok: [localhost]

PLAY [Configure the server and install required software] *************************************************************************************************************

TASK [common : Check the system] **************************************************************************************************************************************
changed: [<REDACTED IP>]

TASK [common : include_tasks] *****************************************************************************************************************************************
included: /algo/roles/common/tasks/freebsd.yml for <REDACTED IP>

TASK [common : set_fact] **********************************************************************************************************************************************
ok: [<REDACTED IP>]

TASK [common : setup] *************************************************************************************************************************************************
ok: [<REDACTED IP>]
changed: [<REDACTED IP>] => (item=git)
changed: [<REDACTED IP>] => (item=subversion)
changed: [<REDACTED IP>] => (item=screen)
changed: [<REDACTED IP>] => (item=coreutils)
changed: [<REDACTED IP>] => (item=openssl)
changed: [<REDACTED IP>] => (item=bash)
changed: [<REDACTED IP>] => (item=wget)

TASK [common : Install tools] *****************************************************************************************************************************************

TASK [common : Loopback included into the rc config] ******************************************************************************************************************
changed: [<REDACTED IP>]
changed: [<REDACTED IP>] => (item={u'param': u'firewall_enable', u'value': u'"YES"'})
changed: [<REDACTED IP>] => (item={u'param': u'firewall_type', u'value': u'"open"'})
changed: [<REDACTED IP>] => (item={u'param': u'gateway_enable', u'value': u'"YES"'})
changed: [<REDACTED IP>] => (item={u'param': u'natd_enable', u'value': u'"YES"'})
changed: [<REDACTED IP>] => (item={u'param': u'natd_interface', u'value': u'"vtnet0"'})
changed: [<REDACTED IP>] => (item={u'param': u'natd_flags', u'value': u'"-dynamic -m"'})

TASK [common : Enable the gateway features] ***************************************************************************************************************************

TASK [common : FreeBSD | Activate IPFW] *******************************************************************************************************************************
changed: [<REDACTED IP>]

RUNNING HANDLER [common : restart ipfw] *******************************************************************************************************************************
changed: [<REDACTED IP>]

RUNNING HANDLER [common : restart loopback bsd] ***********************************************************************************************************************
changed: [<REDACTED IP>]

TASK [common : Generate password for the CA key] **********************************************************************************************************************
changed: [<REDACTED IP> -> localhost]

TASK [common : Generate p12 export password] **************************************************************************************************************************
changed: [<REDACTED IP> -> localhost]

TASK [common : Define facts] ******************************************************************************************************************************************
ok: [<REDACTED IP>]

TASK [common : set_fact] **********************************************************************************************************************************************
ok: [<REDACTED IP>]

TASK [common : Set IPv6 support as a fact] ****************************************************************************************************************************
ok: [<REDACTED IP>]
changed: [<REDACTED IP>] => (item={u'item': u'net.inet.ip.forwarding', u'value': 1})
changed: [<REDACTED IP>] => (item={u'item': u'net.inet6.ip6.forwarding', u'value': 1})

TASK [common : Sysctl tuning] *****************************************************************************************************************************************

TASK [dns_encryption : Include tasks for FreeBSD] *********************************************************************************************************************
included: /algo/roles/dns_encryption/tasks/freebsd.yml for <REDACTED IP>

TASK [dns_encryption : Install dnscrypt-proxy] ************************************************************************************************************************
changed: [<REDACTED IP>]

TASK [dns_encryption : Enable mac_portacl] ****************************************************************************************************************************
changed: [<REDACTED IP>]

TASK [dns_encryption : dnscrypt-proxy ip-blacklist configured] ********************************************************************************************************
changed: [<REDACTED IP>]

TASK [dns_encryption : dnscrypt-proxy configured] *********************************************************************************************************************
changed: [<REDACTED IP>]

TASK [dns_encryption : dnscrypt-proxy enabled and started] ************************************************************************************************************
changed: [<REDACTED IP>]

RUNNING HANDLER [dns_encryption : restart dnscrypt-proxy] *************************************************************************************************************
changed: [<REDACTED IP> -> localhost] => (item=private)
changed: [<REDACTED IP> -> localhost] => (item=public)

TASK [wireguard : Ensure the required directories exist] **************************************************************************************************************

TASK [wireguard : Include tasks for FreeBSD] **************************************************************************************************************************
included: /algo/roles/wireguard/tasks/freebsd.yml for <REDACTED IP>

TASK [wireguard : BSD | WireGuard installed] **************************************************************************************************************************
changed: [<REDACTED IP>]

TASK [wireguard : set_fact] *******************************************************************************************************************************************
ok: [<REDACTED IP>]

TASK [wireguard : BSD | Configure rc script] **************************************************************************************************************************
changed: [<REDACTED IP>]
changed: [<REDACTED IP>] => (item=<REDACTED USER>)
changed: [<REDACTED IP>] => (item=<REDACTED USER>)
changed: [<REDACTED IP>] => (item=<REDACTED IP>)

TASK [wireguard : Generate private keys] ******************************************************************************************************************************
changed: [<REDACTED IP>] => (item=None)
changed: [<REDACTED IP>] => (item=None)
changed: [<REDACTED IP>] => (item=None)

TASK [wireguard : Save private keys] **********************************************************************************************************************************
changed: [<REDACTED IP>] => (item=<REDACTED USER>)
changed: [<REDACTED IP>] => (item=<REDACTED USER>)
changed: [<REDACTED IP>] => (item=<REDACTED IP>)

TASK [wireguard : Touch the lock file] ********************************************************************************************************************************
ok: [<REDACTED IP>] => (item=<REDACTED USER>)
ok: [<REDACTED IP>] => (item=<REDACTED USER>)
ok: [<REDACTED IP>] => (item=<REDACTED IP>)

TASK [wireguard : Generate public keys] *******************************************************************************************************************************
changed: [<REDACTED IP>] => (item=None)
changed: [<REDACTED IP>] => (item=None)
changed: [<REDACTED IP>] => (item=None)

TASK [wireguard : Save public keys] ***********************************************************************************************************************************
changed: [<REDACTED IP> -> localhost] => (item=<REDACTED USER>)
changed: [<REDACTED IP> -> localhost] => (item=<REDACTED USER>)

TASK [wireguard : WireGuard user list updated] ************************************************************************************************************************

TASK [wireguard : set_fact] *******************************************************************************************************************************************
ok: [<REDACTED IP> -> localhost]
changed: [<REDACTED IP> -> localhost] => (item=(0, u'<REDACTED USER>'))
changed: [<REDACTED IP> -> localhost] => (item=(1, u'<REDACTED USER>'))

TASK [wireguard : WireGuard users config generated] *******************************************************************************************************************
ok: [<REDACTED IP> -> localhost] => (item=(0, u'<REDACTED USER>'))
ok: [<REDACTED IP> -> localhost] => (item=(1, u'<REDACTED USER>'))

TASK [wireguard : Generate QR codes] **********************************************************************************************************************************

TASK [wireguard : WireGuard configured] *******************************************************************************************************************************
changed: [<REDACTED IP>]

TASK [wireguard : WireGuard enabled and started] **********************************************************************************************************************
changed: [<REDACTED IP>]

RUNNING HANDLER [wireguard : restart wireguard] ***********************************************************************************************************************
changed: [<REDACTED IP>]

TASK [vpn : Install strongSwan] ***************************************************************************************************************************************
changed: [<REDACTED IP>]
changed: [<REDACTED IP>] => (item={u'dest': u'/usr/local/etc/strongswan.conf', u'src': u'strongswan.conf.j2', u'group': u'wheel', u'mode': u'0644', u'owner': u'root'})
changed: [<REDACTED IP>] => (item={u'dest': u'/usr/local/etc/ipsec.conf', u'src': u'ipsec.conf.j2', u'group': u'wheel', u'mode': u'0644', u'owner': u'root'})
failed: [<REDACTED IP>] (item={u'dest': u'/usr/local/etc/ipsec.secrets', u'src': u'ipsec.secrets.j2', u'group': u'wheel', u'mode': u'0600', u'owner': u'strongswan'}) => {"changed": false, "checksum": "6f637290359ab0c08b84270cd886c2a8d1ed539e", "dest": "/usr/local/etc/ipsec.secrets", "gid": 0, "group": "wheel", "item": {"dest": "/usr/local/etc/ipsec.secrets", "group": "wheel", "mode": "0600", "owner": "strongswan", "src": "ipsec.secrets.j2"}, "mode": "0600", "msg": "chown failed: failed to look up user strongswan", "owner": "root", "path": "/usr/local/etc/ipsec.secrets", "size": 27, "state": "file", "uid": 0}

TASK [vpn : Setup the config files from our templates] ****************************************************************************************************************

TASK [vpn : debug] ****************************************************************************************************************************************************
ok: [<REDACTED IP>] => {
    "fail_hint": [
        "Sorry, but something went wrong!",
        "Please check the troubleshooting guide.",
        "https://trailofbits.github.io/algo/troubleshooting.html"
    ]
}

TASK [vpn : fail] *****************************************************************************************************************************************************
fatal: [<REDACTED IP>]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP ************************************************************************************************************************************************************
<REDACTED IP>             : ok=41   changed=28   unreachable=0    failed=2
localhost                  : ok=34   changed=8    unreachable=0    failed=0```
fgsch commented 5 years ago

FreeBSD:

FreeBSD algo 11.2-RELEASE FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

Packages installed:

apr-1.6.3.1.6.1_1              Apache Portability Library
bash-4.4.23                    GNU Project's Bourne Again SHell
ca_root_nss-3.38               Root certificate bundle from the Mozilla Project
coreutils-8.30                 Free Software Foundation core utilities
curl-7.60.0                    Command line tool and library for transferring data with URLs
cvsps-2.1_2                    Create patchset information from CVS
db5-5.3.28_7                   Oracle Berkeley DB, revision 5.3
dmidecode-3.1_1                Tool for dumping DMI (SMBIOS) contents in human-readable format
dnscrypt-proxy2-2.0.16_1       Flexible DNS proxy with support for encrypted protocols
e2fsprogs-1.44.2               Utilities & library to manipulate ext2/3/4 filesystems
e2fsprogs-libblkid-1.44.2      Blkid library from e2fsprogs package
e2fsprogs-libss-1.44.2         Command-line interface parsing library from e2fsprogs
e2fsprogs-libuuid-1.44.2       UUID library from e2fsprogs package
expat-2.2.6_1                  XML 1.0 parser written in C
gdbm-1.13_1                    GNU database manager
gettext-runtime-0.19.8.1_1     GNU gettext runtime libraries and programs
git-2.19.1                     Distributed source code management tool
gpart-0.1h_2                   Tries to recover lost partition tables and file systems
indexinfo-0.3.1                Utility to regenerate the GNU info page index
jq-1.5_3                       Lightweight and flexible command-line JSON processor
libffi-3.2.1_2                 Foreign Function Interface
libiconv-1.14_11               Character set conversion library
libidn2-2.0.5                  Implementation of IDNA2008 internationalized domain names
liblz4-1.8.3,1                 LZ4 compression library, lossless and very fast
libnghttp2-1.31.1              HTTP/2.0 C Library
libunistring-0.9.10            Unicode string library
oniguruma-6.8.1                Regular expressions library compatible with POSIX/GNU/Perl
openssl-1.0.2p_1,1             SSL and crypto library
p5-Authen-SASL-2.16_1          Perl5 module for SASL authentication
p5-CGI-4.40                    Handle Common Gateway Interface requests and responses
p5-Digest-HMAC-1.03_1          Perl5 interface to HMAC Message-Digest Algorithms
p5-Error-0.17026               Error/exception handling in object-oriented programming style
p5-GSSAPI-0.28_1               Perl extension providing access to the GSSAPIv2 library
p5-HTML-Parser-3.72            Perl5 module for parsing HTML documents
p5-HTML-Tagset-3.20_1          Some useful data table in parsing HTML
p5-IO-Socket-INET6-2.72_1      Perl module with object interface to AF_INET6 domain sockets
p5-IO-Socket-SSL-2.059         Perl5 interface to SSL sockets
p5-Mozilla-CA-20180117         Perl extension for Mozilla CA cert bundle in PEM format
p5-Net-SSLeay-1.85             Perl5 interface to SSL
p5-Socket6-0.29                IPv6 related part of the C socket.h defines and structure manipulators
pcre-8.42                      Perl Compatible Regular Expressions library
perl5-5.26.2                   Practical Extraction and Report Language
pkg-1.10.5_5                   Package manager
py27-Babel-2.5.1               Collection of tools for internationalizing Python applications
py27-Jinja2-2.10               Fast and easy to use stand-alone template engine
py27-MarkupSafe-1.0            Implements XML/HTML/XHTML Markup safe string for Python
py27-asn1crypto-0.22.0         ASN.1 library with a focus on performance and a pythonic API
py27-boto-2.48.0               Python interface to Amazon Web Services
py27-certifi-2018.1.18         Mozilla SSL certificates
py27-cffi-1.11.2               Foreign Function Interface for Python calling C code
py27-chardet-3.0.4             Universal encoding detector for Python 2 and 3
py27-cheetah-2.4.4_1           HTML template engine for Python
py27-cloud-init-0.7.6_1        Init scripts for use on cloud images
py27-configobj-5.0.6_1         Simple but powerful config file reader and writer
py27-cryptography-2.1.4        Cryptographic recipes and primitives for Python developers
py27-enum34-1.1.6              Python 3.4 Enum backported to 3.3, 3.2, 3.1, 2.7
py27-idna-2.6                  Internationalized Domain Names in Applications (IDNA)
py27-ipaddress-1.0.19          Port of Python 3.3+ ipaddress module to 2.7
py27-jsonpatch-1.21            Apply JSON-Patches (RFC 6902)
py27-jsonpointer-1.9_1         Identify specific nodes in a JSON document (RFC 6901)
py27-markdown-2.6.8            Python implementation of Markdown
py27-oauth-1.0.1_2             Python Library for OAuth
py27-openssl-17.5.0_1          Python interface to the OpenSSL library
py27-prettytable-0.7.2_2       Library for displaying tabular data in an ASCII table format
py27-pycparser-2.18            C parser in Python
py27-pysocks-1.6.8             Python SOCKS module
py27-pytz-2018.3,1             World Timezone Definitions for Python
py27-requests-2.18.4           HTTP library written in Python for human beings
py27-serial-3.4                Serial port encapsulation library for Python
py27-setuptools-39.0.1         Python packages installer
py27-six-1.11.0                Python 2 and 3 compatibility utilities
py27-urllib3-1.22              HTTP library with thread-safe connection pooling, file post, and more
py27-yaml-3.12                 Python YAML parser
python27-2.7.15                Interpreted object-oriented programming language
readline-7.0.3_1               Library for editing command lines as they are typed
rsync-3.1.3                    Network file distribution/synchronization utility
screen-4.6.2                   Multi-screen window manager
serf-1.3.9_3                   Serf HTTP client library
sqlite3-3.25.1                 SQL database engine in a C library
strongswan-5.7.1               Open Source IKEv2 IPsec-based VPN solution
subversion-1.10.2_1            Version control system
sudo-1.8.22                    Allow others to run commands as root
utf8proc-2.1.0                 UTF-8 processing library
vim-console-8.0.1638           Improved version of the vi editor (console only)
wget-1.19.5                    Retrieve files from the Net via HTTP(S) and FTP
wireguard-0.0.20180925         Fast, modern and secure VPN Tunnel
wireguard-go-0.0.20180613      WireGuard implementation in Go

Check:

root@algo:~ # grep strongswan /etc/passwd ; echo $?
1
davidemyers commented 5 years ago

According to the documentation you need to create the FreeBSD server first, make sure the kernel meets the requirements, and then perform a local install by invoking Ansible directly as shown.

fgsch commented 5 years ago

Apparently broken in bcba9055474ea99ead92786729266f1b3d186e19

fgsch commented 5 years ago

Based on the comment on #1127:

From now StrongSwan runs under the default user space strongswan:nogroup instead of custom strongswan:strongswan

The commit above incorrectly removed:

- name: Ensure that the strongswan user exist
 user: name=strongswan group=strongswan state=present
fgsch commented 5 years ago

It also looks like this also broke Linux systems where the strongswan user is not created by the package (e.g. alpine)

jackivanov commented 5 years ago

Apparently broken in bcba905

Yes, that's right. I'll revert it back later today.

It also looks like this also broke Linux systems where the strongswan user is not created by the package (e.g. alpine)

We support Ubuntu 18 only, so nothing else is broken