Add OCSP checks

[felddy]

Add OCSP checks to OpenVPN. Here is a little ditty that we were using on pfSense:

#!/usr/bin/env bash

#set -x

# Perform OCSP queries with OpenSSL
# given a certificate serial number.

# Full documentation and examples:
# http://www.openssl.org/docs/apps/ocsp.html

# OCSP responder URL (mandatory)
ocsp_url="http://ocsp.dimc.dhs.gov"

home="/root/ocsp"
#home="."

# Path to issuer certificate (mandatory)
# YOU MUST SET THIS TO THE PATH TO THE CA CERTIFICATE
issuer_CAs=("${home}/DHS_CA4_2021.pem" "${home}/DHS_CA4_2025.pem")

# use a nonce in the query, set to "-no_nonce" to not use it
nonce="-no_nonce"

# Verify the response
# YOU MUST SET THIS TO THE PATH TO THE RESPONSE VERIFICATION CERT
verify="${home}/dhsca_fullpath.pem"

# Depth in the certificate chain where the cert to verify is.
# Set to -1 to run the verification at every level (NOTE that
# in that case you need a more complex script as the various
# parameters for the query will likely be different at each level)
# "0" is the usual value here, where the client certificate is
check_depth=0

cur_depth=$1     # this is the *CURRENT* depth
common_name=$2   # CN in case you need it

# minimal sanity checks

err=0
for issuer in ${issuer_CAs[@]}; do
  if [ -z "$issuer" ] || [ ! -e "$issuer" ]; then
    echo "Error: issuer certificate undefined or not found!" >&2
    err=1
  fi
  # build up part of the openssl ocsp command
  issuers="${issuers} -issuer $issuer "
done

if [ -z "$verify" ] || [ ! -e "$verify" ]; then
  echo "Error: verification certificate undefined or not found!" >&2
  err=1
fi

if [ -z "$ocsp_url" ]; then
  echo "Error: OCSP server URL not defined!" >&2
  err=1
fi

if [ $err -eq 1 ]; then
  echo "Did you forget to customize the variables in the script?" >&2
  exit 1
fi

# begin
if [ $check_depth -eq -1 ] || [ $cur_depth -eq $check_depth ]; then

  eval serial="\$tls_serial_${cur_depth}"

  # To successfully complete, the following must happen:
  #
  # - The serial number must not be empty
  # - The exit status of "openssl ocsp" must be zero
  # - The output of the above command must contain the line
  #   "${serial}: good"
  #
  # Everything else fails with exit status 1.

  if [ -n "$serial" ]; then

    # This is only an example; you are encouraged to run this command (without
    # redirections) manually against your or your CA's OCSP server to see how
    # it responds, and adapt accordingly.
    # Sample output that is assumed here:
    #
    # Response verify OK
    # 4287405: good
    #      This Update: Apr 24 19:38:49 2010 GMT
    #      Next Update: May  2 14:23:42 2010 GMT
    #
    # NOTE: It is needed to check the exit code of OpenSSL explicitly.  OpenSSL
    #       can in some circumstances give a "good" result if it could not
    #       reach the the OSCP server.  In this case, the exit code will indicate
    #       if OpenSSL itself failed or not.  If OpenSSL's exit code is not 0,
    #       don't trust the OpenSSL status.

    status=$(openssl ocsp $issuers \
                    "$nonce" \
                    -CAfile "$verify" \
                    -url "$ocsp_url" \
                    -serial "${serial}" 2>&1)

    if [ $? -eq 0 ]; then
      # check if ocsp didn't report any errors
      if echo "$status" | grep -Eq "(error|fail)"; then
          exit 1
      fi
      # check that the reported status of certificate is ok
      if echo "$status" | grep -Eq "^${serial}: good"; then
        # check if signature on the OCSP response verified correctly
        if echo "$status" | grep -Eq "^Response verify OK"; then
            echo "Recieved good OCSP verification for certificate serial: ${serial}"
            exit 0
        fi
      fi
    fi
  fi
  # if we get here, something was wrong
  exit 1
fi

[michaelsaki]

@felddy I am trying to get the script to run on some of the Molecule Docker containers locally. One of the portions of the configurations require a path to a response verification certificate verify="${home}/dhsca_fullpath.pem". When I run sudo find / -type f -name "*.pem" from inside one of the containers I can't seem to find any files with that specific naming convention. Here is my output from the find command:

sudo find / -type f -name "* pem"
/usr/lib/python3/dist-packages/pip/_vendor/certifi/cacert.pem
/usr/lib/python3/dist-packages/botocore/cacert.pem
/usr/lib/python3/dist-packages/slapdtest/certs/client.pem
/usr/lib/python3/dist-packages/slapdtest/certs/server.pem
/usr/lib/python3/dist-packages/slapdtest/certs/ca.pem
/usr/lib/python3/dist-packages/certifi/cacert.pem
/usr/share/doc/openvpn/examples/sample-keys/dh2048.pem
/usr/share/gnupg/sks-keyservers.netCA.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_SSP_CA_2029.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_SSP_CA_2030.pem
/etc/openvpn/server/certs/DHS_CA4_2033.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_SSP_CA_2025.pem
/etc/openvpn/server/certs/DHS_CA4_2029.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_Root_CA_2030.pem
/etc/openvpn/server/certs/DHS_CA4_2025.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_Root_CA_2025.pem
/etc/openvpn/server/certs/US_Treasury_Root_CA_2026.pem
/etc/openvpn/server/certs/Entrust_Managed_Services_Root_CA_2029.pem
/etc/openvpn/server/certs/Federal_Common_Policy_CA_G2_2040.pem
/etc/openvpn/server/certs/US_Treasury_Root_CA_2025.pem

[felddy]

@michaelsaki , you'll have to install the DHS certificate authority certificates. It might make sense to abstract out the installation of DHS CA certs into a new role that can be used by both FreeIPA and OpenVPN.

Here is a link to the code in the FreeIPA server role:

• https://github.com/cisagov/ansible-role-freeipa-server/blob/94d909cfc34de150fb1328271949c12ce22d9e9d/tasks/main.yml#L15-L27

- name: Copy DHS CA certs
  ansible.builtin.get_url:
    dest: /usr/local/share
    mode: 0600
    url: https://pki.treas.gov/dhsca_fullpath.p7b
- name: Convert P7B to PEM
  ansible.builtin.command:
    cmd: >
      openssl pkcs7 -print_certs
      -in /usr/local/share/dhsca_fullpath.p7b
      -inform DER
      -out /usr/local/share/dhsca_fullpath.pem
    creates: /usr/local/share/dhsca_fullpath.pem

[jsf9k]

It might make sense to abstract out the installation of DHS CA certs into a new role that can be used by both FreeIPA and OpenVPN.

I believe such a role already exists!

[michaelsaki]

@jsf9k and @felddy so I would just need to add the ansible code to ansible-role-dhs-certificates?

[jsf9k]

@jsf9k and @felddy so I would just need to add the ansible code to ansible-role-dhs-certificates?

You can possibly just use add ansible-role-dhs-certificates as a dependency of this role, in meta/main.yml, and then remove the DHS certificate Ansible code in this role.

You'll have to compare what ansible-role-dhs-certificates does versus what the DHS certificate jazz in this role does. If this role does some extra stuff then you may have to leave that piece of it in.

[michaelsaki]

@felddy Will the ansible-role-dhs-certificates accomplish the same utility as the fetch_user_ca_certs.sh? I am comparing the two right now and it seems like there are decent amount of differences between them. Thought you might have some insight in what the script does vs the ansible role.

[felddy]

Yes. These are pretty different. Probably different enough to remain separate. The OpenVPN implementation is actually gathering more CAs since we had folks with credentials from signed by the DOE CA. The installation path of the certs is also different.

It's always nice to DRY things out when we can, but I don't think it makes sense here given the myriad differences.

Thanks for looking into it.

[michaelsaki]

@felddy that is more or less the conclusion I was coming to. So maybe I should approach it with your original suggestion where I make a new role? Like below:

@michaelsaki , you'll have to install the DHS certificate authority certificates. It might make sense to abstract out the installation of DHS CA certs into a new role that can be used by both FreeIPA and OpenVPN.

Here is a link to the code in the FreeIPA server role:

• https://github.com/cisagov/ansible-role-freeipa-server/blob/94d909cfc34de150fb1328271949c12ce22d9e9d/tasks/main.yml#L15-L27

- name: Copy DHS CA certs
  ansible.builtin.get_url:
    dest: /usr/local/share
    mode: 0600
    url: https://pki.treas.gov/dhsca_fullpath.p7b
- name: Convert P7B to PEM
  ansible.builtin.command:
    cmd: >
      openssl pkcs7 -print_certs
      -in /usr/local/share/dhsca_fullpath.p7b
      -inform DER
      -out /usr/local/share/dhsca_fullpath.pem
    creates: /usr/local/share/dhsca_fullpath.pem

[michaelsaki]

Update

I am now able to get the script running in the Molecule Docker containers but I am still having some issues.

I keep running into some errors when I run the openssl-ocsp check. error: unable to get local issuer certificate and error: unable to get issue certificate I will leave a screenshot below. It seems that the serial number is coming up as "good" but when I researched why I was getting this error I saw some people suggest that maybe I am not passing in the full cert chain into the openssl command. That would make sense why it isn't able to get the issue certificate. I suspect I am just giving it the intermediate_cert.pem and the root_cert.pem and not the client or server cert. This is just speculation though. Wanted to see if anyone has seen a similar error. Also just FYI I am using the Entrust_Managed_Services_SSP_CA_2025.pem and Entrust_Managed_Services_Root_CA_2025.pem

[michaelsaki]

Update

So after having some conversations during one of the team syncs I have been trying to run the OCSP check in the verify-cn.py script instead of a separate shell script. However I am a bit confused as to how the script is used in the ansible role. I do understand that it is used to verify freeipa along with kerberos too. But I am trying to understand how I can witness it in "action" so to speak. I am not seeing it being executed when I runmolecule converge or molecule test. Is it suppose to run or are we just installing the python script?

[dav3r]

Is it suppose to run or are we just installing the python script?

@michaelsaki You're correct that verify-cn.py does not get run by this Ansible role, it only installs it. That script gets run by OpenVPN when a user connects to the VPN. See here for the OpenVPN config file template where this is specified: https://github.com/cisagov/openvpn-server-tf-module/blob/a809a60a0a94edc111eebb302e1822ace0d491e1/cloudinit/openvpn-config.tpl.yml#L140-L143

[michaelsaki]

The latest branch I was working on was: https://github.com/cisagov/ansible-role-openvpn/tree/improvement/add_OCSP_checks

I was able to get a mocked version of the verify-cn.py script running locally that performed the OCSP check. When it was loaded onto the staging OpenVPN servers it was discovered that one of the dependencies to perform the check, the Cryptography library, on the servers was 3.3.2. The success that I had running locally was using 41.0.7. This was an issue because the OCSP check is trying to process multiple SINGLERESP's, 20 total, and the outdated version of the Cryptography library only supports doing 1 at a time. This caused a block on the development until we resolved upgrading the OpenVPN servers from an AMI based on Debian Bullseye to Debian Bookworm which would have the upgraded Cryptography library.