trailofbits / algo

Set up a personal VPN in the cloud
https://blog.trailofbits.com/2016/12/12/meet-algo-the-vpn-that-works/
GNU Affero General Public License v3.0
28.93k stars 2.32k forks source link

Error on Google Cloud install: gpg: keyserver receive failed: No data #1014

Closed slowtokyo closed 6 years ago

slowtokyo commented 6 years ago

OS / Environment (where do you run Algo on)

Darwin MBP 17.6.0 Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64 x86_64

Cloud Provider (where do you deploy Algo to)

Google Compute Engine

Summary of the problem

When installing the latest version of algo (version available 26 June 2018), the task [dns_encryption : Add the repository] fails with the following error:

fatal: [35.189.137.142]: FAILED! => {"changed": false, "cmd": "apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7", "msg": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: No data"], "stdout": "Executing: /tmp/apt-key-gpghome.AFo1sVboxo/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.AFo1sVboxo/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7"]}

Steps to reproduce the behavior

  1. Install algo using Google Cloud
  2. Let the script run until the task [dns_encryption : Add the repository]

Full log

(env) ➜  algo-master ./algo

  What provider would you like to use?
    1. DigitalOcean
    2. Amazon EC2
    3. Microsoft Azure
    4. Google Compute Engine
    5. Scaleway
    6. OpenStack (DreamCompute optimised)
    7. Install to existing Ubuntu 16.04 server (Advanced)

Enter the number of your desired provider
: 4

Enter the local path to your credentials JSON file (https://support.google.com/cloud/answer/6158849?hl=en&ref_topic=6262490#serviceaccounts):
:[REDACTED]

Name the vpn server:
[algo]: [REDACTED]

  What zone should the server be located in?
    1. Eastern Canada          (Montreal A)
    2. Eastern Canada          (Montreal B)
    3. Eastern Canada          (Montreal C)
    4. Central US              (Iowa A)
    5. Central US              (Iowa B)
    6. Central US              (Iowa C)
    7. Central US              (Iowa F)
    8. Western US              (Oregon A)
    9. Western US              (Oregon B)
    10. Western US             (Oregon C)
    11. Eastern US             (Northern Virginia A)
    12. Eastern US             (Northern Virginia B)
    13. Eastern US             (Northern Virginia C)
    14. Eastern US             (South Carolina B)
    15. Eastern US             (South Carolina C)
    16. Eastern US             (South Carolina D)
    17. South America East     (São Paulo A)
    18. South America East     (São Paulo B)
    19. South America East     (São Paulo C)
    20. Northern Europe        (Hamina A)
    21. Northern Europe        (Hamina B)
    22. Northern Europe        (Hamina C)
    23. Western Europe         (Belgium B)
    24. Western Europe         (Belgium C)
    25. Western Europe         (Belgium D)
    26. Western Europe         (London A)
    27. Western Europe         (London B)
    28. Western Europe         (London C)
    29. Western Europe         (Frankfurt A)
    30. Western Europe         (Frankfurt B)
    31. Western Europe         (Frankfurt C)
    32. Western Europe         (Netherlands A)
    33. Western Europe         (Netherlands B)
    34. Western Europe         (Netherlands C)
    35. South Asia             (Mumbai A)
    36. South Asia             (Mumbai B)
    37. South Asia             (Mumbai C)
    38. Southeast Asia         (Singapore A)
    39. Southeast Asia         (Singapore B)
    40. Southeast Asia         (Singapore C)
    41. East Asia              (Taiwan A)
    42. East Asia              (Taiwan B)
    43. East Asia              (Taiwan C)
    44. Northeast Asia         (Tokyo A)
    45. Northeast Asia         (Tokyo B)
    46. Northeast Asia         (Tokyo C)
    47. Australia              (Sydney A)
    48. Australia              (Sydney B)
    49. Australia              (Sydney C)

Please choose the number of your zone. Press enter for default (#20) zone.
[20]: 44

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
[y/N]: 

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
[y/N]: 

Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
[y/N]: 

Do you want each user to have their own account for SSH tunneling?
[y/N]: 

Do you want the VPN to support Windows 10 or Linux Desktop clients? (enables compatible ciphers and key exchange, less secure)
[y/N]: 

Do you want to retain the CA key? (required to add users in the future, but less secure)
[y/N]: 

PLAY [Configure the server] ********************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]

TASK [Local pre-tasks] *************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/local.yml for localhost

TASK [Generate the SSH private key] ************************************************************************************
ok: [localhost]

TASK [Generate the SSH public key] *************************************************************************************
ok: [localhost]

TASK [Change mode for the SSH private key] *****************************************************************************
ok: [localhost]

TASK [Ensure the dynamic inventory exists] *****************************************************************************
ok: [localhost]

TASK [cloud-gce : set_fact] ********************************************************************************************
ok: [localhost]

TASK [cloud-gce : set_fact] ********************************************************************************************
ok: [localhost]

TASK [cloud-gce : set_fact] ********************************************************************************************
ok: [localhost]

TASK [cloud-gce : Network configured] **********************************************************************************
changed: [localhost]

TASK [cloud-gce : Creating a new instance...] **************************************************************************
changed: [localhost]

TASK [cloud-gce : Add the instance to an inventory group] **************************************************************
changed: [localhost]

TASK [cloud-gce : set_fact] ********************************************************************************************
ok: [localhost]

TASK [cloud-gce : Ensure the group gce exists in the dynamic inventory file] *******************************************
changed: [localhost]

TASK [cloud-gce : Populate the dynamic inventory] **********************************************************************
changed: [localhost]

TASK [Local post-tasks] ************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/post.yml for localhost

TASK [Wait until SSH becomes ready...] *********************************************************************************
ok: [localhost]

TASK [A short pause, in order to be sure the instance is ready] ********************************************************
Pausing for 20 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [localhost]

TASK [include_tasks] ***************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/local_ssh.yml for localhost

TASK [Ensure the local ssh directory is exist] *************************************************************************
ok: [localhost]

TASK [Copy the algo ssh key to the local ssh directory] ****************************************************************
changed: [localhost]

PLAY [Configure the server and install required software] **************************************************************

TASK [Common pre-tasks] ************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/common.yml for 35.189.137.142

TASK [Check the system] ************************************************************************************************
changed: [35.189.137.142]

TASK [Ubuntu pre-tasks] ************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/ubuntu.yml for 35.189.137.142

TASK [Ubuntu | Install prerequisites] **********************************************************************************
changed: [35.189.137.142] => (item=sleep 10)
changed: [35.189.137.142] => (item=apt-get update -qq)
changed: [35.189.137.142] => (item=apt-get install -qq -y python2.7 sudo)

TASK [Ubuntu | Configure defaults] *************************************************************************************
changed: [35.189.137.142]

TASK [FreeBSD pre-tasks] ***********************************************************************************************
skipping: [35.189.137.142]

TASK [include_tasks] ***************************************************************************************************
included: /Users/user/Documents/code/algo-master/playbooks/facts/main.yml for 35.189.137.142

TASK [Gather Facts] ****************************************************************************************************
ok: [35.189.137.142]

TASK [Ensure the algo ssh key exist on the server] *********************************************************************
ok: [35.189.137.142]

TASK [Check if IPv6 configured] ****************************************************************************************
ok: [35.189.137.142]

TASK [Set facts if the deployment in a cloud] **************************************************************************
ok: [35.189.137.142]

TASK [Generate password for the CA key] ********************************************************************************
changed: [35.189.137.142 -> localhost]

TASK [Generate p12 export password] ************************************************************************************
changed: [35.189.137.142 -> localhost]

TASK [Define password facts] *******************************************************************************************
ok: [35.189.137.142]

TASK [Define the commonName] *******************************************************************************************
ok: [35.189.137.142]

TASK [common : Install tools] ******************************************************************************************

TASK [common : Sysctl tuning] ******************************************************************************************

TASK [common : Install tools] ******************************************************************************************

TASK [common : Sysctl tuning] ******************************************************************************************

TASK [common : Install tools] ******************************************************************************************

TASK [common : Sysctl tuning] ******************************************************************************************

TASK [common : include_tasks] ******************************************************************************************
included: /Users/user/Documents/code/algo-master/roles/common/tasks/ubuntu.yml for 35.189.137.142

TASK [common : Install software updates] *******************************************************************************
changed: [35.189.137.142]

TASK [common : Upgrade the ca certificates] ****************************************************************************
ok: [35.189.137.142]

TASK [common : Check if reboot is required] ****************************************************************************
changed: [35.189.137.142]

TASK [common : Reboot] *************************************************************************************************
changed: [35.189.137.142]

TASK [common : Wait until SSH becomes ready...] ************************************************************************
ok: [35.189.137.142 -> localhost]

TASK [common : Include unatteded upgrades configuration] ***************************************************************
included: /Users/user/Documents/code/algo-master/roles/common/tasks/unattended-upgrades.yml for 35.189.137.142

TASK [common : Install unattended-upgrades] ****************************************************************************
ok: [35.189.137.142]

TASK [common : Configure unattended-upgrades] **************************************************************************
changed: [35.189.137.142]

TASK [common : Periodic upgrades configured] ***************************************************************************
changed: [35.189.137.142]

TASK [common : Disable MOTD on login and SSHD] *************************************************************************
changed: [35.189.137.142] => (item={u'regexp': u'^session.*optional.*pam_motd.so.*', u'line': u'# MOTD DISABLED', u'file': u'/etc/pam.d/login'})
changed: [35.189.137.142] => (item={u'regexp': u'^session.*optional.*pam_motd.so.*', u'line': u'# MOTD DISABLED', u'file': u'/etc/pam.d/sshd'})

TASK [common : Loopback for services configured] ***********************************************************************
changed: [35.189.137.142]

TASK [common : systemd services enabled and started] *******************************************************************
ok: [35.189.137.142] => (item=systemd-networkd)
ok: [35.189.137.142] => (item=systemd-resolved)

RUNNING HANDLER [common : restart systemd-networkd] ********************************************************************
changed: [35.189.137.142]

TASK [common : Check apparmor support] *********************************************************************************
changed: [35.189.137.142]

TASK [common : set_fact] ***********************************************************************************************
ok: [35.189.137.142]

TASK [common : set_fact] ***********************************************************************************************
ok: [35.189.137.142]

TASK [common : include_tasks] ******************************************************************************************
skipping: [35.189.137.142]

TASK [common : Install tools] ******************************************************************************************
ok: [35.189.137.142] => (item=git)
ok: [35.189.137.142] => (item=screen)
changed: [35.189.137.142] => (item=apparmor-utils)
ok: [35.189.137.142] => (item=uuid-runtime)
ok: [35.189.137.142] => (item=coreutils)
changed: [35.189.137.142] => (item=iptables-persistent)
changed: [35.189.137.142] => (item=cgroup-tools)
ok: [35.189.137.142] => (item=openssl,linux-headers-4.15.0-1009-gcp)

TASK [common : Sysctl tuning] ******************************************************************************************
changed: [35.189.137.142] => (item={u'item': u'net.ipv4.ip_forward', u'value': 1})
changed: [35.189.137.142] => (item={u'item': u'net.ipv4.conf.all.forwarding', u'value': 1})
changed: [35.189.137.142] => (item={u'item': u'net.ipv6.conf.all.forwarding', u'value': 1})

TASK [wireguard : WireGuard repository configured] *********************************************************************
changed: [35.189.137.142]

TASK [wireguard : WireGuard installed] *********************************************************************************
changed: [35.189.137.142]

TASK [wireguard : Ensure the required directories exist] ***************************************************************
changed: [35.189.137.142 -> localhost] => (item=private)
changed: [35.189.137.142 -> localhost] => (item=public)

TASK [wireguard : Delete the lock files] *******************************************************************************
skipping: [35.189.137.142] => (item=user1) 
skipping: [35.189.137.142] => (item=user2) 
skipping: [35.189.137.142] => (item=35.189.137.142) 

TASK [wireguard : Generate private keys] *******************************************************************************
changed: [35.189.137.142] => (item=user1)
changed: [35.189.137.142] => (item=user2)
changed: [35.189.137.142] => (item=35.189.137.142)
 [WARNING]: As of Ansible 2.4, the parameter 'executable' is no longer supported with the 'command' module. Not using
'bash'.

TASK [wireguard : Save private keys] ***********************************************************************************
changed: [35.189.137.142] => (item=None)
changed: [35.189.137.142] => (item=None)
changed: [35.189.137.142] => (item=None)

TASK [wireguard : Touch the lock file] *********************************************************************************
changed: [35.189.137.142] => (item=user1)
changed: [35.189.137.142] => (item=user2)
changed: [35.189.137.142] => (item=35.189.137.142)

TASK [wireguard : Generate public keys] ********************************************************************************
ok: [35.189.137.142] => (item=user1)
ok: [35.189.137.142] => (item=user2)
ok: [35.189.137.142] => (item=35.189.137.142)

TASK [wireguard : Save public keys] ************************************************************************************
changed: [35.189.137.142] => (item=None)
changed: [35.189.137.142] => (item=None)
changed: [35.189.137.142] => (item=None)

TASK [wireguard : WireGuard configured] ********************************************************************************
changed: [35.189.137.142]

TASK [wireguard : WireGuard reload-module-on-update] *******************************************************************
changed: [35.189.137.142]

TASK [wireguard : WireGuard users config generated] ********************************************************************
changed: [35.189.137.142 -> localhost] => (item=(0, u'user1'))
changed: [35.189.137.142 -> localhost] => (item=(1, u'user2'))

TASK [wireguard : WireGuard enabled and started] ***********************************************************************
changed: [35.189.137.142]

RUNNING HANDLER [wireguard : restart wireguard] ************************************************************************
changed: [35.189.137.142]

TASK [dns_encryption : Include tasks for Ubuntu] ***********************************************************************
included: /Users/user/Documents/code/algo-master/roles/dns_encryption/tasks/ubuntu.yml for 35.189.137.142

TASK [dns_encryption : Add the repository] *****************************************************************************
fatal: [35.189.137.142]: FAILED! => {"changed": false, "cmd": "apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7", "msg": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: No data"], "stdout": "Executing: /tmp/apt-key-gpghome.AFo1sVboxo/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.AFo1sVboxo/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7"]}

PLAY RECAP *************************************************************************************************************
35.189.137.142             : ok=47   changed=28   unreachable=0    failed=1   
localhost                  : ok=21   changed=6    unreachable=0    failed=0  
slowtokyo commented 6 years ago

This seems to have been a temporary issue and did not happen on a subsequent install so I can't replicate it. Feel free to close. However I'll leave it open for now in case someone wants to investigate.

dguido commented 6 years ago

Thanks!

TC1977 commented 6 years ago

Just ran into the same error while trying to test out the ansible2.5 branch on an AWS EC2 instance using the local install option. Here's the output from ./algo -vvv:


TASK [dns_encryption : Add the repository] *************************************
task path: /home/ubuntu/algo/roles/dns_encryption/tasks/ubuntu.yml:2
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "cmd": "apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7", 
    "invocation": {
        "module_args": {
            "codename": "artful", 
            "filename": null, 
            "install_python_apt": true, 
            "mode": null, 
            "repo": "ppa:shevchuk/dnscrypt-proxy", 
            "state": "present", 
            "update_cache": true, 
            "validate_certs": true
        }
    }, 
    "msg": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data", 
    "rc": 2, 
    "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data\n", 
    "stderr_lines": [
        "Warning: apt-key output should not be parsed (stdout is not a terminal)", 
        "gpg: keyserver receive failed: No data"
    ], 
    "stdout": "Executing: /tmp/apt-key-gpghome.smruv9dzG8/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7\n", 
    "stdout_lines": [
        "Executing: /tmp/apt-key-gpghome.smruv9dzG8/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 EEE1EE950DF627C6879627B56C46343FA0B41DB7"
    ]
}

I'll start over and see if I can replicate.

TC1977 commented 6 years ago

Couldn't replicate it, even doing a local install of the ansible2.5 branch on the same EC2 instance.

bmoquist commented 6 years ago

I experienced a similar issue today using the current distribution to deploy to AWS EC2 -- the error at a different part of the process during Wireguard repository configuration. I made a second attempt starting from scratch, and the install went fine.

Error:

TASK [wireguard : WireGuard repository configured] ***************************************************************
fatal: [13.58.93.167]: FAILED! => {"changed": false, "cmd": "apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 E1B39B6EF6DDB96564797591AE33835F504A1A25", "msg": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: No data"], "stdout": "Executing: /tmp/apt-key-gpghome.XiSO86hQrl/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 E1B39B6EF6DDB96564797591AE33835F504A1A25\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.XiSO86hQrl/gpg.1.sh --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 E1B39B6EF6DDB96564797591AE33835F504A1A25"]}
jackivanov commented 6 years ago

Confirmed. The issue not really depends on us, but I'll make a workaround