CERTIFICATE_VERIFY_FAILED error despite running "install certificate" commands in macOS 10.15.1

[gnorcie]

Describe the bug

Despite installing python3 and running the install certificates command per the documentation (downloaded python3 per docs + ran the install certs command + commands for prereqs)

To Reproduce

Attempt to install algo on macos 10.15.1

Expected behavior

Algo installs without error

Additional context

I've previously installed algo w/ zero issues on this machine.

I am not using macports or brew, I attempted to run brew install curl-ca-bundle but no package was found.

(I do not have macports on this machine and have had issues in the past when I mix brew and macports on same machine, but open to installing it and trying it if that may fix issue, but hoping to get to the root cause since it's my understanding running the python cert install should have solved the issue)

Thanks for the help!

Full log

(.env) $ ./algo
 [WARNING]: Could not match supplied host pattern, ignoring: vpn-host

PLAY [localhost] ********************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************
ok: [localhost]

TASK [Playbook dir stat] ************************************************************************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ****************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Ensure the requirements installed] ********************************************************************************************
ok: [localhost]

TASK [Set required ansible version as a fact] ***************************************************************************************
ok: [localhost] => (item=ansible==2.8.3)

TASK [Verify Python meets Algo VPN requirements] ************************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] ***********************************************************************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY [Ask user for the input] *******************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
    1. DigitalOcean
    2. Amazon Lightsail
    3. Amazon EC2
    4. Microsoft Azure
    5. Google Compute Engine
    6. Hetzner Cloud
    7. Vultr
    8. Scaleway
    9. OpenStack (DreamCompute optimised)
    10. CloudStack (Exoscale optimised)
    11. Install to existing Ubuntu 18.04

Enter the number of your desired provider
:

TASK [Cloud prompt] *****************************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] *************************************************************************************************
ok: [localhost]
[VPN server name prompt]
Name the vpn server
[algo]
:

TASK [VPN server name prompt] *******************************************************************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS IPsec clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:

TASK [Cellular On Demand prompt] ****************************************************************************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS IPsec clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:

TASK [Wi-Fi On Demand prompt] *******************************************************************************************************
ok: [localhost]
[Trusted Wi-Fi networks prompt]
List the names of any trusted Wi-Fi networks where macOS/iOS IPsec clients should not use "Connect On Demand"
(e.g., your home network. Comma-separated value, e.g., HomeNet,OfficeWifi,AlgoWiFi)
:

TASK [Trusted Wi-Fi networks prompt] ************************************************************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:

TASK [Retain the PKI prompt] ********************************************************************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:

TASK [DNS adblocking prompt] ********************************************************************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:

TASK [SSH tunneling prompt] *********************************************************************************************************
ok: [localhost]

TASK [Set facts based on the input] *************************************************************************************************
ok: [localhost]

PLAY [Provision the server] *********************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Mac OS X 10.15.1
ZIP file created: Nov  6 02:25:35 2019
Python 3.7.3
Runtime variables:
    algo_provider "digitalocean"
    algo_ondemand_cellular "True"
    algo_ondemand_wifi "True"
    algo_ondemand_wifi_exclude "X251bGw="
    algo_dns_adblocking "True"
    algo_ssh_tunneling "False"
    wireguard_enabled "True"
    dns_encryption "True"

TASK [Display the invocation environment] *******************************************************************************************
changed: [localhost -> localhost]

TASK [Install the requirements] *****************************************************************************************************
ok: [localhost -> localhost]

TASK [Generate the SSH private key] *************************************************************************************************
ok: [localhost]

TASK [Generate the SSH public key] **************************************************************************************************
ok: [localhost]

TASK [Copy the private SSH key to /tmp] *********************************************************************************************
ok: [localhost -> localhost]
[cloud-digitalocean : pause]
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
 (output is hidden):

TASK [cloud-digitalocean : pause] ***************************************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Set the token as a fact] *********************************************************************************
ok: [localhost]

TASK [cloud-digitalocean : Get regions] *********************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "content": "", "elapsed": 0, "msg": "Status code was -1 and not [200]: Request failed: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>", "redirected": false, "status": -1, "url": "https://api.digitalocean.com/v2/regions"}
included: /Users/[redacted]/Downloads/algo-master/playbooks/rescue.yml for localhost

TASK [debug] ************************************************************************************************************************
ok: [localhost] => {
    "fail_hint": [
        "Sorry, but something went wrong!",
        "Please check the troubleshooting guide.",
        "https://trailofbits.github.io/algo/troubleshooting.html"
    ]
}

TASK [Fail the installation] ********************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP **************************************************************************************************************************
localhost                  : ok=28   changed=1    unreachable=0    failed=1    skipped=0    rescued=1    ignored=0

[davidemyers]

Please post the output of:

ls -l $(which python3)

Try running Install Certificates again. Do you see output from the command? I've seen the command do nothing when not using the default shell in Terminal.

If you just installed Python 3 from python.org, can I ask why you chose version 3.7.3 and not a more recent version of 3.7? (3.8 will not work at the moment)

[gnorcie]

Maybe that's the issue - to be honest I was in a rush when downloading and installing from the site:

$ ls -l $(which python3)
lrwxr-xr-x  1 root  admin  9 Nov 18 20:53 /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 -> python3.8

So this may be a user error problem? I could DL 3.7 then rerun the certs command if that may fix...

[davidemyers]

You downloaded 3.8 from python.org but the log you posted above says 3.7.3, so you apparently have multiple versions installed. You should try to find and remove the old version so it doesn't cause problems in the future.

See my post here on how to remove a python.org package from macOS.

If you still have Brew installed then brew install python3 should be all you need once you've cleaned out any other Python 3 packages.

[gnorcie]

Thanks, I appreciate the help. I think you're correct about the root cause, but my system is especially borked, so it'll take some time to get all the old python purged.

[adhamnm]

I've had the same issue as well on my mac. Not sure about the versions thing though as I'm using python 3.7.3. Also I found a similar issue with Ansible regarding the certificate verification on python. I tried another install using python 3.6.5 and got the same output. https://github.com/ansible/ansible/issues/14089

[davidemyers]

@adhamnm Please post the output from the following commands:

ls -l $(which python3)
grep -c CERT /Library/Frameworks/Python.framework/Versions/3.7/etc/openssl/cert.pem

[adhamnm]

I get "No such file or directory" I was initially trying to deploy on DigitalOcean when I got the error above. However, after I tried on GCloud it worked without issues. Thank you guys for the amazing library.

[davidemyers]

I get "No such file or directory"

For both commands?

Did you get your Python 3.7 from python.org and run the "Install Certificates" command?

[adhamnm]

No for the first command i get: -rwxrwxr-x 1 staff 8632 Nov 30 20:45 //.env/bin/python3

btw I also tried python 3.5.2 after trying 3.7.3 and 3.6.5 with DigitalOcean and got the same response for the 3 of them. For some reason urlopen was not able to verify the certificate from DO

[davidemyers]

@gnorcie So apparently after announcing they would not be including new versions of Python in Catalina, Apple now includes Python 3.7.3 as part of the Xcode Command Line Tools. Like git, a stub version of python3 is installed in /usr/bin and will prompt you to install the tools the first time you run it. I'm not running Catalina yet so I totally missed this.

Does this sound like what you see on your system? I wonder if Apple's version is incapable of verifying certificates. I'm afraid I can't test this yet.