trailofbits / algo

Set up a personal VPN in the cloud
https://blog.trailofbits.com/2016/12/12/meet-algo-the-vpn-that-works/
GNU Affero General Public License v3.0
28.65k stars 2.31k forks source link

Installation error on azure with adblocking enabled. #14568

Closed az-pz closed 11 months ago

az-pz commented 1 year ago

Describe the bug When you choose adblocking as true while installing algo on azure, the installation fails.

Relevant section: TASK [dns : Adblock script added to cron] **** fatal: [20.198.72.84]: FAILED! => {"changed": false, "msg": "Failed to find required executable \"crontab\" in paths: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"}

To Reproduce

Steps to reproduce the behavior:

  1. Start installation, choose azure as cloud provider and choose adblocker option as yes. You can choose anything else for other options.
  2. VM is created successfully. But adblocker is not installed because cron is not installed. Expected behavior Installation should succeed.

Workaround

  1. ssh to the created VM using ssh -F configs/<vm_ip>/ssh_config <vm_ip>
  2. Install cron using sudo apt install cron.
  3. Exit ssh : exit.
  4. Restart the installation process again using ./algo and choose all the other options selected before. Additional context NA Full log

PLAY [localhost] *****************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************
ok: [localhost]

TASK [Playbook dir stat] *********************************************************************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] *************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
[DEPRECATION WARNING]: Use 'ansible.utils.ipaddr' module instead. This feature will be removed from ansible.netcommon in a
release after 2024-01-01. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: The value '' is not a valid IP address or network, passing this value to ipaddr filter might result in breaking change
in future.

TASK [Ensure the requirements installed] *****************************************************************************************
ok: [localhost]

TASK [Set required ansible version as a fact] ************************************************************************************
ok: [localhost] => (item=ansible==6.1.0)

TASK [Just get the list from default pip] ****************************************************************************************
ok: [localhost]

TASK [Verify Python meets Algo VPN requirements] *********************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] ********************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
[WARNING]: Found variable using reserved name: no_log

PLAY [Ask user for the input] ****************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
    1. DigitalOcean
    2. Amazon Lightsail
    3. Amazon EC2
    4. Microsoft Azure
    5. Google Compute Engine
    6. Hetzner Cloud
    7. Vultr
    8. Scaleway
    9. OpenStack (DreamCompute optimised)
    10. CloudStack (Exoscale optimised)
    11. Linode
    12. Install to existing Ubuntu 18.04 or 20.04 server (for more advanced users)

Enter the number of your desired provider
:
4^M
TASK [Cloud prompt] **************************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] **********************************************************************************************
ok: [localhost]
[VPN server name prompt]
Name the vpn server
[algo]
:
algo-vpn-india^M
TASK [VPN server name prompt] ****************************************************************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:
^M
TASK [Cellular On Demand prompt] *************************************************************************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:
^M
TASK [Wi-Fi On Demand prompt] ****************************************************************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:
y^M
TASK [Retain the PKI prompt] *****************************************************************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:
y^M
TASK [DNS adblocking prompt] *****************************************************************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:
^M
TASK [SSH tunneling prompt] ******************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] **********************************************************************************************
ok: [localhost]

PLAY [Provision the server] ******************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Ubuntu 22.04.1 LTS (Virtualized: wsl)
Created from git clone. Last commit: 651f949 Update cloud-hetzner.md (#14450)
Python 3.10.6
Runtime variables:
    algo_provider "azure"
    algo_ondemand_cellular "False"
    algo_ondemand_wifi "False"
    algo_ondemand_wifi_exclude "X251bGw="
    algo_dns_adblocking "True"
    algo_ssh_tunneling "False"
    wireguard_enabled "True"
    dns_encryption "True"

TASK [Display the invocation environment] ****************************************************************************************
changed: [localhost]

TASK [Install the requirements] **************************************************************************************************
ok: [localhost]

TASK [Generate the SSH private key] **********************************************************************************************
ok: [localhost]

TASK [Generate the SSH public key] ***********************************************************************************************
ok: [localhost]

TASK [Copy the private SSH key to /tmp] ******************************************************************************************
ok: [localhost]

TASK [Include a provisioning role] ***********************************************************************************************

TASK [cloud-azure : Install requirements] ****************************************************************************************
ok: [localhost]

TASK [cloud-azure : set_fact] ****************************************************************************************************
ok: [localhost]

TASK [cloud-azure : Set the default region] **************************************************************************************
ok: [localhost]
[cloud-azure : pause]
What region should the server be located in?
    1. Asia
    2. Asia Pacific
    3. Australia
    4. (Asia Pacific) Australia Central
    5. (Asia Pacific) Australia Central 2
    6. (Asia Pacific) Australia East
    7. (Asia Pacific) Australia Southeast
    8. Brazil
    9. (South America) Brazil South
    10. (South America) Brazil Southeast
    11. Canada
    12. (Canada) Canada Central
    13. (Canada) Canada East
    14. (Asia Pacific) Central India
    15. (US) Central US
    16. (US) Central US EUAP
    17. (US) Central US (Stage)
    18. (Asia Pacific) East Asia
    19. (Asia Pacific) East Asia (Stage)
    20. (US) East US
    21. (US) East US 2
    22. (US) East US 2 EUAP
    23. (US) East US 2 (Stage)
    24. (US) East US (Stage)
    25. Europe
    26. (Europe) France Central
    27. (Europe) France South
    28. (Europe) Germany North
    29. (Europe) Germany West Central
    30. Global
    31. India
    32. Japan
    33. (Asia Pacific) Japan East
    34. (Asia Pacific) Japan West
    35. (Asia Pacific) Jio India Central
    36. (Asia Pacific) Jio India West
    37. (Asia Pacific) Korea Central
    38. (Asia Pacific) Korea South
    39. (US) North Central US
    40. (US) North Central US (Stage)
    41. (Europe) North Europe
    42. (Europe) Norway East
    43. (Europe) Norway West
    44. (Europe) Qatar Central
    45. (Africa) South Africa North
    46. (Africa) South Africa West
    47. (US) South Central US
    48. (US) South Central US (Stage)
    49. (Asia Pacific) Southeast Asia
    50. (Asia Pacific) Southeast Asia (Stage)
    51. (Asia Pacific) South India
    52. (Europe) Sweden Central
    53. (Europe) Sweden South
    54. (Europe) Switzerland North
    55. (Europe) Switzerland West
    56. (Middle East) UAE Central
    57. (Middle East) UAE North
    58. United Kingdom
    59. (Europe) UK South
    60. (Europe) UK West
    61. United States
    62. (US) West Central US
    63. (Europe) West Europe
    64. (Asia Pacific) West India
    65. (US) West US
    66. (US) West US 2
    67. (US) West US 2 (Stage)
    68. (US) West US 3
    69. (US) West US (Stage)

Enter the number of your desired region
[20]
:
14^M
TASK [cloud-azure : pause] *******************************************************************************************************
ok: [localhost]

TASK [cloud-azure : set_fact] ****************************************************************************************************
ok: [localhost]

TASK [cloud-azure : Create AlgoVPN Server] ***************************************************************************************
changed: [localhost]

TASK [cloud-azure : set_fact] ****************************************************************************************************
ok: [localhost]

TASK [Set subjectAltName as a fact] **********************************************************************************************
ok: [localhost]

TASK [Add the server to an inventory group] **************************************************************************************
changed: [localhost]

TASK [Additional variables for the server] ***************************************************************************************
changed: [localhost]

TASK [Wait until SSH becomes ready...] *******************************************************************************************
ok: [localhost]

TASK [debug] *********************************************************************************************************************
ok: [localhost] => {
    "IP_subject_alt_name": "20.198.72.84"
}

TASK [Wait 600 seconds for target connection to become reachable/usable] *********************************************************
ok: [localhost -> 20.198.72.84] => (item=20.198.72.84)

PLAY [Configure the server and install required software] ************************************************************************

TASK [Wait until the cloud-init completed] ***************************************************************************************
ok: [20.198.72.84]

TASK [Ensure the config directory exists] ****************************************************************************************
changed: [20.198.72.84 -> localhost]

TASK [Dump the ssh config] *******************************************************************************************************
changed: [20.198.72.84 -> localhost]

TASK [common : Check the system] *************************************************************************************************
ok: [20.198.72.84]

TASK [common : include_tasks] ****************************************************************************************************
included: /home/<username>/algo/roles/common/tasks/ubuntu.yml for 20.198.72.84

TASK [common : Gather facts] *****************************************************************************************************
ok: [20.198.72.84]

TASK [common : Install software updates] *****************************************************************************************
ok: [20.198.72.84]

TASK [common : Check if reboot is required] **************************************************************************************
changed: [20.198.72.84]

TASK [common : Install unattended-upgrades] **************************************************************************************
ok: [20.198.72.84]

TASK [common : Configure unattended-upgrades] ************************************************************************************
changed: [20.198.72.84]

TASK [common : Periodic upgrades configured] *************************************************************************************
changed: [20.198.72.84]

TASK [common : Disable MOTD on login and SSHD] ***********************************************************************************
changed: [20.198.72.84] => (item={'regexp': '^session.*optional.*pam_motd.so.*', 'line': '# MOTD DISABLED', 'file': '/etc/pam.d/login'})
changed: [20.198.72.84] => (item={'regexp': '^session.*optional.*pam_motd.so.*', 'line': '# MOTD DISABLED', 'file': '/etc/pam.d/sshd'})
[WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when
running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually

TASK [common : Ensure fallback resolvers are set] ********************************************************************************
changed: [20.198.72.84]
[DEPRECATION WARNING]: Use 'ansible.utils.ipmath' module instead. This feature will be removed from ansible.netcommon in a
release after 2024-01-01. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

TASK [common : Loopback for services configured] *********************************************************************************
changed: [20.198.72.84]

TASK [common : systemd services enabled and started] *****************************************************************************
ok: [20.198.72.84] => (item=systemd-networkd)
ok: [20.198.72.84] => (item=systemd-resolved)

RUNNING HANDLER [common : restart systemd-networkd] ******************************************************************************
changed: [20.198.72.84]

RUNNING HANDLER [common : restart systemd-resolved] ******************************************************************************
changed: [20.198.72.84]

TASK [common : Check apparmor support] *******************************************************************************************
ok: [20.198.72.84]

TASK [common : Set fact if apparmor enabled] *************************************************************************************
ok: [20.198.72.84]

TASK [common : Define facts] *****************************************************************************************************
ok: [20.198.72.84]

TASK [common : Set facts] ********************************************************************************************************
ok: [20.198.72.84]

TASK [common : Set IPv6 support as a fact] ***************************************************************************************
ok: [20.198.72.84]

TASK [common : Check size of MTU] ************************************************************************************************
ok: [20.198.72.84]

TASK [common : Set OS specific facts] ********************************************************************************************
ok: [20.198.72.84]

TASK [common : Install tools] ****************************************************************************************************
changed: [20.198.72.84]

TASK [common : include_tasks] ****************************************************************************************************
included: /home/<username>/algo/roles/common/tasks/iptables.yml for 20.198.72.84
[DEPRECATION WARNING]: Use 'ansible.utils.ipmath' module instead. This feature will be removed from ansible.netcommon in a
release after 2024-01-01. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

TASK [common : Iptables configured] **********************************************************************************************
changed: [20.198.72.84] => (item={'src': 'rules.v4.j2', 'dest': '/etc/iptables/rules.v4'})

TASK [common : Sysctl tuning] ****************************************************************************************************
changed: [20.198.72.84] => (item={'item': 'net.ipv4.ip_forward', 'value': 1})
changed: [20.198.72.84] => (item={'item': 'net.ipv4.conf.all.forwarding', 'value': 1})

RUNNING HANDLER [common : restart iptables] **************************************************************************************
changed: [20.198.72.84]

TASK [dns : Include tasks for Ubuntu] ********************************************************************************************
included: /home/<username>/algo/roles/dns/tasks/ubuntu.yml for 20.198.72.84

TASK [dns : Install dnscrypt-proxy] **********************************************************************************************
changed: [20.198.72.84]

TASK [dns : Ubuntu | Configure AppArmor policy for dnscrypt-proxy] ***************************************************************
changed: [20.198.72.84]

TASK [dns : Ubuntu | Enforce the dnscrypt-proxy AppArmor policy] *****************************************************************
ok: [20.198.72.84]

TASK [dns : Ubuntu | Ensure that the dnscrypt-proxy service directory exist] *****************************************************
changed: [20.198.72.84]

TASK [dns : Ubuntu | Add custom requirements to successfully start the unit] *****************************************************
changed: [20.198.72.84]

TASK [dns : dnscrypt-proxy ip-blacklist configured] ******************************************************************************
changed: [20.198.72.84]
[DEPRECATION WARNING]: Use 'ansible.utils.ipmath' module instead. This feature will be removed from ansible.netcommon in a
release after 2024-01-01. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

TASK [dns : dnscrypt-proxy configured] *******************************************************************************************
changed: [20.198.72.84]

TASK [dns : Adblock script created] **********************************************************************************************
changed: [20.198.72.84]

TASK [dns : Adblock script added to cron] ****************************************************************************************
fatal: [20.198.72.84]: FAILED! => {"changed": false, "msg": "Failed to find required executable \"crontab\" in paths: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"}

TASK [include_tasks] *************************************************************************************************************
included: /home/<username>/algo/playbooks/rescue.yml for 20.198.72.84

TASK [debug] *********************************************************************************************************************
ok: [20.198.72.84] => {
PLAY RECAP ***********************************************************************************************************************
20.198.72.84               : ok=40   changed=21   unreachable=0    failed=1    skipped=10   rescued=1    ignored=0
localhost                  : ok=37   changed=4    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0
DanielSzentimrey-Harrach commented 1 year ago

Is this issue prioritized for a fix? Based on the proposed workaround the solution should be as simple as installing cron on the target VM as part of the process.

Also, I tried the workaround but it didn't work for me, even after I logged in to Azure before executing ./algo. Any advice?

PLAY [localhost] *******************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]

TASK [Playbook dir stat] ***********************************************************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ***************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
[DEPRECATION WARNING]: Use 'ansible.utils.ipaddr' module instead. This feature will be removed from ansible.netcommon
in a release after 2024-01-01. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[WARNING]: The value '' is not a valid IP address or network, passing this value to ipaddr filter might result in
breaking change in future.

TASK [Ensure the requirements installed] *******************************************************************************
ok: [localhost]

TASK [Set required ansible version as a fact] **************************************************************************
ok: [localhost] => (item=ansible==6.1.0)

TASK [Just get the list from default pip] ******************************************************************************
ok: [localhost]

TASK [Verify Python meets Algo VPN requirements] ***********************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] **********************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
[WARNING]: Found variable using reserved name: no_log

PLAY [Ask user for the input] ******************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
    1. DigitalOcean
    2. Amazon Lightsail
    3. Amazon EC2
    4. Microsoft Azure
    5. Google Compute Engine
    6. Hetzner Cloud
    7. Vultr
    8. Scaleway
    9. OpenStack (DreamCompute optimised)
    10. CloudStack (Exoscale optimised)
    11. Linode
    12. Install to existing Ubuntu latest LTS server (for more advanced users)

Enter the number of your desired provider
:
4^M
TASK [Cloud prompt] ****************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] ************************************************************************************
ok: [localhost]
[VPN server name prompt]
Name the vpn server
[algo]
:
<<servername>>^M
TASK [VPN server name prompt] ******************************************************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:
N^M
TASK [Cellular On Demand prompt] ***************************************************************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:
N^M
TASK [Wi-Fi On Demand prompt] ******************************************************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:
N^M
TASK [Retain the PKI prompt] *******************************************************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:
y^M
TASK [DNS adblocking prompt] *******************************************************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:
N^M
TASK [SSH tunneling prompt] ********************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] ************************************************************************************
ok: [localhost]

PLAY [Provision the server] ********************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Ubuntu 20.04.5 LTS (Virtualized: microsoft)
Created from git fork. Last commit: 75cfeab Ubuntu 22.04 support (#14579)
Python 3.8.10
Runtime variables:
    algo_provider "azure"
    algo_ondemand_cellular "False"
    algo_ondemand_wifi "False"
    algo_ondemand_wifi_exclude "X251bGw="
    algo_dns_adblocking "True"
    algo_ssh_tunneling "False"
    wireguard_enabled "True"
    dns_encryption "True"

TASK [Display the invocation environment] ******************************************************************************
changed: [localhost]

TASK [Install the requirements] ****************************************************************************************
ok: [localhost]

TASK [Generate the SSH private key] ************************************************************************************
ok: [localhost]

TASK [Generate the SSH public key] *************************************************************************************
ok: [localhost]

TASK [Copy the private SSH key to /tmp] ********************************************************************************
ok: [localhost]

TASK [Include a provisioning role] *************************************************************************************

TASK [cloud-azure : Install requirements] ******************************************************************************
ok: [localhost]

TASK [cloud-azure : set_fact] ******************************************************************************************
ok: [localhost]

TASK [cloud-azure : Set the default region] ****************************************************************************
ok: [localhost]
[cloud-azure : pause]
What region should the server be located in?
    1. Asia
    2. Asia Pacific
    3. Australia
    4. (Asia Pacific) Australia Central
    5. (Asia Pacific) Australia Central 2
    6. (Asia Pacific) Australia East
    7. (Asia Pacific) Australia Southeast
    8. Brazil
    9. (South America) Brazil South
    10. (South America) Brazil Southeast
    11. Canada
    12. (Canada) Canada Central
    13. (Canada) Canada East
    14. (Asia Pacific) Central India
    15. (US) Central US
    16. (US) Central US EUAP
    17. (US) Central US (Stage)
    18. (Asia Pacific) East Asia
    19. (Asia Pacific) East Asia (Stage)
    20. (US) East US
    21. (US) East US 2
    22. (US) East US 2 EUAP
    23. (US) East US 2 (Stage)
    24. (US) East US (Stage)
    25. Europe
    26. (Europe) France Central
    27. (Europe) France South
    28. (Europe) Germany North
    29. (Europe) Germany West Central
    30. Global
    31. India
    32. Japan
    33. (Asia Pacific) Japan East
    34. (Asia Pacific) Japan West
    35. (Asia Pacific) Jio India Central
    36. (Asia Pacific) Jio India West
    37. (Asia Pacific) Korea Central
    38. (Asia Pacific) Korea South
    39. (US) North Central US
    40. (US) North Central US (Stage)
    41. (Europe) North Europe
    42. (Europe) Norway East
    43. (Europe) Norway West
    44. (Europe) Qatar Central
    45. (Africa) South Africa North
    46. (Africa) South Africa West
    47. (US) South Central US
    48. (US) South Central US (Stage)
    49. (Asia Pacific) Southeast Asia
    50. (Asia Pacific) Southeast Asia (Stage)
    51. (Asia Pacific) South India
    52. (Europe) Sweden Central
    53. (Europe) Sweden South
    54. (Europe) Switzerland North
    55. (Europe) Switzerland West
    56. (Middle East) UAE Central
    57. (Middle East) UAE North
    58. United Kingdom
    59. (Europe) UK South
    60. (Europe) UK West
    61. United States
    62. (US) West Central US
    63. (Europe) West Europe
    64. (Asia Pacific) West India
    65. (US) West US
    66. (US) West US 2
    67. (US) West US 2 (Stage)
    68. (US) West US 3
    69. (US) West US (Stage)

Enter the number of your desired region
[20]
:
21^M
TASK [cloud-azure : pause] *********************************************************************************************
ok: [localhost]

TASK [cloud-azure : set_fact] ******************************************************************************************
ok: [localhost]

TASK [cloud-azure : Create AlgoVPN Server] *****************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or log in with Azure CLI (`az login`)."}

TASK [include_tasks] ***************************************************************************************************
included: /home/<<username>>/algo/playbooks/rescue.yml for localhost

TASK [debug] ***********************************************************************************************************
ok: [localhost] => {
    "fail_hint": [
        "Sorry, but something went wrong!",
        "Please check the troubleshooting guide.",
        "https://trailofbits.github.io/algo/troubleshooting.html"
    ]
}

TASK [Fail the installation] *******************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP *************************************************************************************************************
localhost                  : ok=31   changed=1    unreachable=0    failed=1    skipped=1    rescued=1    ignored=0
DanielSzentimrey-Harrach commented 11 months ago

Can confirm it now works as expected.