Closed benmontour closed 6 years ago
Can you make the full log more readable? Can't reproduce the issue and never had it before.
Did the best I could to make the log more readable. Hope that helps. I'm going to try redownloading and redoing the install from scratch again to see if it happens again. I ran it a few times yesterday, but it was from the same folder without redownloading and starting from scratch.
Just did fresh download and install. Fails in the exact same place. Something isn't right with the Install dnscrypt-proxy step.
Everything seems OK, can't reproduce. I'm still waiting for the full log here.
What full log do you need? I have the error on "Install dnscrypt-proxy" included in original post. Happy to pull whatever logs you need.
The whole log of the installation process
PLAY [Ask user for the input] *******************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************
ok: [localhost]
[pause]
What provider would you like to use?
1. DigitalOcean
2. Amazon EC2
3. Vultr
4. Microsoft Azure
5. Google Compute Engine
6. Scaleway
7. OpenStack (DreamCompute optimised)
8. Install to existing Ubuntu 18.04 server (Advanced)
Enter the number of your desired provider
:
1
TASK [pause] ************************************************************************************************************************
ok: [localhost]
TASK [Set facts based on the input] *************************************************************************************************
ok: [localhost]
[pause]
Name the vpn server
[algo]
:
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
[y/N]
:
y
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
[y/N]
:
y
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
List the names of trusted Wi-Fi networks (if any) that macOS/iOS clients exclude from using the VPN
(e.g., your home network. Comma-separated value, e.g., HomeNet,OfficeWifi,AlgoWiFi)
:
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
[y/N]
:
y
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want the VPN to support Windows 10 or Linux Desktop clients? (enables compatible ciphers and key exchange, less secure)
[y/N]
:
TASK [pause] ************************************************************************************************************************
ok: [localhost]
[pause]
Do you want to retain the CA key? (required to add users in the future, but less secure)
[y/N]
:
TASK [pause] ************************************************************************************************************************
ok: [localhost]
TASK [Set facts based on the input] *************************************************************************************************
ok: [localhost]
PLAY [Provision the server] *********************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************
ok: [localhost]
TASK [Generate the SSH private key] *************************************************************************************************
changed: [localhost]
TASK [Generate the SSH public key] **************************************************************************************************
ok: [localhost]
[cloud-digitalocean : pause]
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
(output is hidden):
TASK [cloud-digitalocean : pause] ***************************************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Set the token as a fact] *********************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Get regions] *********************************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Set facts about thre regions] ****************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Set default region] **************************************************************************************
ok: [localhost]
[cloud-digitalocean : pause]
What region should the server be located in?
1. ams3 Amsterdam 3
2. blr1 Bangalore 1
3. fra1 Frankfurt 1
4. lon1 London 1
5. nyc1 New York 1
6. nyc3 New York 3
7. sfo2 San Francisco 2
8. sgp1 Singapore 1
9. tor1 Toronto 1
Enter the number of your desired region
[6]
:
6
TASK [cloud-digitalocean : pause] ***************************************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Set additional facts] ************************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Delete the existing Algo SSH keys] ***********************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Upload the SSH key] **************************************************************************************
changed: [localhost]
TASK [cloud-digitalocean : Creating a droplet...] ***********************************************************************************
changed: [localhost]
TASK [cloud-digitalocean : set_fact] ************************************************************************************************
ok: [localhost]
TASK [cloud-digitalocean : Tag the droplet] *****************************************************************************************
changed: [localhost]
FAILED - RETRYING: Delete the new Algo SSH key (10 retries left).
TASK [cloud-digitalocean : Delete the new Algo SSH key] *****************************************************************************
ok: [localhost]
TASK [Set subjectAltName as afact] **************************************************************************************************
ok: [localhost]
TASK [Add the server to an inventory group] *****************************************************************************************
changed: [localhost]
TASK [Additional variables for the server] ******************************************************************************************
changed: [localhost]
TASK [Wait until SSH becomes ready...] **********************************************************************************************
ok: [localhost]
TASK [debug] ************************************************************************************************************************
ok: [localhost] => {
"IP_subject_alt_name": "REDACTED IP"
}
Pausing for 20 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
TASK [A short pause, in order to be sure the instance is ready] *********************************************************************
ok: [localhost]
PLAY [Configure the server and install required software] ***************************************************************************
TASK [common : Check the system] ****************************************************************************************************
changed: [REDACTED IP]
TASK [common : include_tasks] *******************************************************************************************************
included: /Users/ben.montour/Applications/algo/roles/common/tasks/ubuntu.yml for REDACTED
changed: [REDACTED IP] => (item=[u'python2.7', u'sudo'])
TASK [common : Ubuntu | Install prerequisites] **************************************************************************************
TASK [common : Ubuntu | Configure defaults] *****************************************************************************************
changed: [REDACTED IP]
TASK [common : Gather facts] ********************************************************************************************************
ok: [REDACTED IP]
TASK [common : Install software updates] ********************************************************************************************
changed: [REDACTED IP]
TASK [common : Check if reboot is required] *****************************************************************************************
changed: [REDACTED IP]
TASK [common : Reboot] **************************************************************************************************************
changed: [REDACTED IP]
TASK [common : Wait until SSH becomes ready...] *************************************************************************************
ok: [REDACTED -> localhost]
TASK [common : Install unattended-upgrades] *****************************************************************************************
ok: [REDACTED IP]
TASK [common : Configure unattended-upgrades] ***************************************************************************************
changed: [REDACTED IP]
TASK [common : Periodic upgrades configured] ****************************************************************************************
changed: [REDACTED IP]
changed: [REDACTED IP] => (item={u'regexp': u'^session.*optional.*pam_motd.so.*', u'line': u'# MOTD DISABLED', u'file': u'/etc/pam.d/login'})
changed: [REDACTED IP] => (item={u'regexp': u'^session.*optional.*pam_motd.so.*', u'line': u'# MOTD DISABLED', u'file': u'/etc/pam.d/sshd'})
TASK [common : Disable MOTD on login and SSHD] **************************************************************************************
TASK [common : Loopback for services configured] ************************************************************************************
changed: [REDACTED IP]
ok: [REDACTED IP] => (item=systemd-networkd)
ok: [REDACTED IP] => (item=systemd-resolved)
TASK [common : systemd services enabled and started] ********************************************************************************
RUNNING HANDLER [common : restart systemd-networkd] *********************************************************************************
changed: [REDACTED IP]
TASK [common : Check apparmor support] **********************************************************************************************
changed: [REDACTED IP]
TASK [common : set_fact] ************************************************************************************************************
ok: [REDACTED IP]
TASK [common : set_fact] ************************************************************************************************************
ok: [REDACTED IP]
ok: [REDACTED IP] => (item=git)
ok: [REDACTED IP] => (item=screen)
changed: [REDACTED IP] => (item=apparmor-utils)
ok: [REDACTED IP] => (item=uuid-runtime)
ok: [REDACTED IP] => (item=coreutils)
changed: [REDACTED IP] => (item=iptables-persistent)
changed: [REDACTED IP] => (item=cgroup-tools)
ok: [REDACTED IP] => (item=openssl,linux-headers-4.15.0-30-generic)
TASK [common : Install tools] *******************************************************************************************************
TASK [common : Generate password for the CA key] ************************************************************************************
changed: [REDACTED IP -> localhost]
TASK [common : Generate p12 export password] ****************************************************************************************
changed: [REDACTED IP -> localhost]
TASK [common : Define facts] ********************************************************************************************************
ok: [REDACTED IP]
TASK [common : set_fact] ************************************************************************************************************
ok: [REDACTED IP]
changed: [REDACTED IP] => (item={u'item': u'net.ipv4.ip_forward', u'value': 1})
changed: [REDACTED IP] => (item={u'item': u'net.ipv4.conf.all.forwarding', u'value': 1})
changed: [REDACTED IP] => (item={u'item': u'net.ipv6.conf.all.forwarding', u'value': 1})
TASK [common : Sysctl tuning] *******************************************************************************************************
TASK [dns_adblocking : Dnsmasq installed] *******************************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : The dnsmasq directory created] *******************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : include_tasks] ***********************************************************************************************
included: /Users/ben.montour/Applications/algo/roles/dns_adblocking/tasks/ubuntu.yml for REDACTED IP
TASK [dns_adblocking : Ubuntu | Dnsmasq profile for apparmor configured] ************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Ubuntu | Enforce the dnsmasq AppArmor policy] ****************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Ubuntu | Ensure that the dnsmasq service directory exist] ****************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Ubuntu | Setup the cgroup limitations for the ipsec daemon] **************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Dnsmasq configured] ******************************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Adblock script created] **************************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Adblock script added to cron] ********************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Update adblock hosts] ****************************************************************************************
changed: [REDACTED IP]
RUNNING HANDLER [dns_adblocking : restart dnsmasq] **********************************************************************************
changed: [REDACTED IP]
RUNNING HANDLER [vpn : daemon-reload] ***********************************************************************************************
changed: [REDACTED IP]
TASK [dns_adblocking : Dnsmasq enabled and started] *********************************************************************************
ok: [REDACTED IP]
TASK [dns_encryption : Include tasks for Ubuntu] ************************************************************************************
included: /Users/ben.montour/Applications/algo/roles/dns_encryption/tasks/ubuntu.yml for REDACTED IP
TASK [dns_encryption : Add the repository] ******************************************************************************************
changed: [REDACTED IP]
TASK [dns_encryption : Install dnscrypt-proxy] **************************************************************************************
fatal: [REDACTED IP]: FAILED! => {"cache_update_time": 1535551799, "cache_updated": true, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'dnscrypt-proxy'' failed: E: Sub-process /usr/bin/dpkg returned an error code (1)\n", "rc": 100, "stderr": "E: Sub-process /usr/bin/dpkg returned an error code (1)\n", "stderr_lines": ["E: Sub-process /usr/bin/dpkg returned an error code (1)"], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following package was automatically installed and is no longer required:\n grub-pc-bin\nUse 'apt autoremove' to remove it.\nThe following NEW packages will be installed:\n dnscrypt-proxy\n0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.\nNeed to get 2140 kB of archives.\nAfter this operation, 7990 kB of additional disk space will be used.\nGet:1 http://ppa.launchpad.net/shevchuk/dnscrypt-proxy/ubuntu bionic/main amd64 dnscrypt-proxy amd64 2.0.16~ppa1~bionic [2140 kB]\nFetched 2140 kB in 5s (443 kB/s)\nSelecting previously unselected package dnscrypt-proxy.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 91587 files and directories currently installed.)\r\nPreparing to unpack .../dnscrypt-proxy_2.0.16~ppa1~bionic_amd64.deb ...\r\nUnpacking dnscrypt-proxy (2.0.16~ppa1~bionic) ...\r\nSetting up dnscrypt-proxy (2.0.16~ppa1~bionic) ...\r\nJob for dnsmasq.service failed because a fatal signal was delivered to the control process.\r\nSee \"systemctl status dnsmasq.service\" and \"journalctl -xe\" for details.\r\ninvoke-rc.d: initscript dnsmasq, action \"restart\" failed.\r\n\u001b[0;1;31m*\u001b[0m dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server\r\n Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)\r\n Drop-In: /etc/systemd/system/dnsmasq.service.d\r\n `-100-CustomLimitations.conf\r\n Active: \u001b[0;1;31mfailed\u001b[0m (Result: signal) since Wed 2018-08-29 14:10:10 UTC; 8ms ago\r\n Process: 3950 ExecStop=/etc/init.d/dnsmasq systemd-stop-resolvconf (code=exited, status=0/SUCCESS)\r\n Process: 3957 ExecStartPre=/usr/sbin/dnsmasq --test \u001b[0;1;31m(code=killed, signal=KILL)\u001b[0m\r\n Main PID: 2944 (code=exited, status=0/SUCCESS)\r\n CPU: 217ms\r\n\r\nAug 29 14:10:06 algo systemd[1]: Stopped dnsmasq - A lightweight DHCP and caching DNS server.\r\nAug 29 14:10:06 algo systemd[1]: dnsmasq.service: Consumed 445ms CPU time\r\nAug 29 14:10:06 algo systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...\r\nAug 29 14:10:10 algo systemd[1]: \u001b[0;1;39m\u001b[0;1;31m\u001b[0;1;39mdnsmasq.service: Control process exited, code=killed status=9\u001b[0m\r\nAug 29 14:10:10 algo systemd[1]: \u001b[0;1;39m\u001b[0;1;31m\u001b[0;1;39mdnsmasq.service: Failed with result 'signal'.\u001b[0m\r\nAug 29 14:10:10 algo systemd[1]: \u001b[0;1;31m\u001b[0;1;39m\u001b[0;1;31mFailed to start dnsmasq - A lightweight DHCP and caching DNS server.\u001b[0m\r\nAug 29 14:10:10 algo systemd[1]: dnsmasq.service: Consumed 217ms CPU time\r\ndpkg: error processing package dnscrypt-proxy (--configure):\r\n installed dnscrypt-proxy package post-installation script subprocess returned error exit status 1\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following package was automatically installed and is no longer required:", " grub-pc-bin", "Use 'apt autoremove' to remove it.", "The following NEW packages will be installed:", " dnscrypt-proxy", "0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.", "Need to get 2140 kB of archives.", "After this operation, 7990 kB of additional disk space will be used.", "Get:1 http://ppa.launchpad.net/shevchuk/dnscrypt-proxy/ubuntu bionic/main amd64 dnscrypt-proxy amd64 2.0.16~ppa1~bionic [2140 kB]", "Fetched 2140 kB in 5s (443 kB/s)", "Selecting previously unselected package dnscrypt-proxy.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 91587 files and directories currently installed.)", "Preparing to unpack .../dnscrypt-proxy_2.0.16~ppa1~bionic_amd64.deb ...", "Unpacking dnscrypt-proxy (2.0.16~ppa1~bionic) ...", "Setting up dnscrypt-proxy (2.0.16~ppa1~bionic) ...", "Job for dnsmasq.service failed because a fatal signal was delivered to the control process.", "See \"systemctl status dnsmasq.service\" and \"journalctl -xe\" for details.", "invoke-rc.d: initscript dnsmasq, action \"restart\" failed.", "\u001b[0;1;31m*\u001b[0m dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server", " Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)", " Drop-In: /etc/systemd/system/dnsmasq.service.d", " `-100-CustomLimitations.conf", " Active: \u001b[0;1;31mfailed\u001b[0m (Result: signal) since Wed 2018-08-29 14:10:10 UTC; 8ms ago", " Process: 3950 ExecStop=/etc/init.d/dnsmasq systemd-stop-resolvconf (code=exited, status=0/SUCCESS)", " Process: 3957 ExecStartPre=/usr/sbin/dnsmasq --test \u001b[0;1;31m(code=killed, signal=KILL)\u001b[0m", " Main PID: 2944 (code=exited, status=0/SUCCESS)", " CPU: 217ms", "", "Aug 29 14:10:06 algo systemd[1]: Stopped dnsmasq - A lightweight DHCP and caching DNS server.", "Aug 29 14:10:06 algo systemd[1]: dnsmasq.service: Consumed 445ms CPU time", "Aug 29 14:10:06 algo systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...", "Aug 29 14:10:10 algo systemd[1]: \u001b[0;1;39m\u001b[0;1;31m\u001b[0;1;39mdnsmasq.service: Control process exited, code=killed status=9\u001b[0m", "Aug 29 14:10:10 algo systemd[1]: \u001b[0;1;39m\u001b[0;1;31m\u001b[0;1;39mdnsmasq.service: Failed with result 'signal'.\u001b[0m", "Aug 29 14:10:10 algo systemd[1]: \u001b[0;1;31m\u001b[0;1;39m\u001b[0;1;31mFailed to start dnsmasq - A lightweight DHCP and caching DNS server.\u001b[0m", "Aug 29 14:10:10 algo systemd[1]: dnsmasq.service: Consumed 217ms CPU time", "dpkg: error processing package dnscrypt-proxy (--configure):", " installed dnscrypt-proxy package post-installation script subprocess returned error exit status 1"]}
PLAY RECAP **************************************************************************************************************************
REDACTED IP : ok=41 changed=29 unreachable=0 failed=1
localhost : ok=34 changed=6 unreachable=0 failed=0
Is that the log you were looking for? If not, let me know where I can find it.
Ok, so on more digging, it's related to the dnsmasq memory limit you have set to 16384 kB. in 100-CustomLimitations.conf.j2. I added a custom host file which I guess it to large to fit within this constraint. That's my bad for not mentioning that. As this appeared to be related to dnscrypt-proxy instead of dnsmasq. Can this limit be increased or are there negative consequences to that?
I'd still suspect dnscrypt-proxy based on that log. If you run the Algo install without the custom host file, does it succeed? And then maybe you can plug in the host file later?
I can try that, what I did as a test was to double the MemoryLimit in /roles/dns_adblocking/templates/100-CustomLimitations.conf.j2 and everything installed just fine.
I added a custom host file which I guess it to large to fit within this constraint. That's my bad for not mentioning that
Any modifications on your own.
I can't reproduce the issue with default settings, so, closing this for now
OS / Environment (where do you run Algo on)
Cloud Provider (where do you deploy Algo to)
Summary of the problem
Running ./algo to initiate automated deploy to DigitalOcean
Steps to reproduce the behavior
Full log