Internal CA certificate cannot be verified

[taikedz]

Hello

I'm trying to use volumerize (docker) with an internal server, which has a company internal CA-signed ccertificate.

I'm trying to pass the certificate in via environment variable, but the duplicity process is not picking this up, or not using this mechanism.

What is the correct way to have volumerize/duplicity trust the internal CA certificate ?

Volumerize command:

 docker run
    --rm
    --name volumerize-s3_minio.company.lan_volumerize.mybuntu-3.appgrafana
    -v appgrafana_grafana:/source/appgrafana_grafana:ro
    -v appgrafana_opentsdb:/source/appgrafana_opentsdb:ro
    -v appgrafana_whisper:/source/appgrafana_whisper:ro
    -v /home/git/CA-Deployer/root-certificates/rnd-ca.cer:/etc/pki/server-cert.cer
    -e SSL_CERT_FILE=/etc/pki/server-cert.cer
    -e SSL_CERT_DIR=/etc/pki
    -v volumerize_s3_minio.company.lan_volumerize.mybuntu-3.appgrafana_cache:/volumerize_cache
    -v volumerize_s3_minio.company.lan_volumerize.mybuntu-3.appgrafana_credentials:/credentials
    -e VOLUMERIZE_SOURCE=/source
    -e VOLUMERIZE_TARGET=s3://minio.company.lan/volumerize.mybuntu-3.appgrafana
    -e AWS_ACCESS_KEY_ID=$KEY
    -e AWS_SECRET_ACCESS_KEY=$SECRET
    -e VOLUMERIZE_FULL_IF_OLDER_THAN=6D
    --env-file /tmp/tmp.tr6tIIyrjK blacklabelops/volumerize:1.1.0 backup

The output I get is

- name: VolumerizeBackupJob
  cmd: /etc/volumerize/periodicBackup
  time: '0 0 4 * * *'
  onError: Continue
  notifyOnError: false
  notifyOnFailure: false

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1546, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1540, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1375, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/site-packages/duplicity/commandline.py", line 1126, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib/python2.7/site-packages/duplicity/commandline.py", line 1015, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/site-packages/duplicity/backends/_boto_single.py", line 166, in __init__
    self.resetConnection()
  File "/usr/lib/python2.7/site-packages/duplicity/backends/_boto_single.py", line 191, in resetConnection
    location=self.my_location)
  File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line 616, in create_bucket
    data=data)
  File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line 668, in make_request
    retry_handler=retry_handler
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1030, in _mexe
    raise ex
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)

I have confirmed that the docker environment's python does see the certificate

$> docker exec -it volumerize-s3_minio.company.lan_volumerize.mybuntu-3.appgrafana bash
bash-4.3# ps aux
PID   USER     TIME   COMMAND
    1 root       0:00 /bin/tini -- /opt/volumerize/docker-entrypoint.sh backup
    6 root       0:00 {backup} /bin/bash /etc/volumerize/backup
   29 root       0:02 python2 /usr/bin/duplicity --allow-source-mismatch --arch
   34 root       0:00 bash
   40 root       0:00 ps aux
bash-4.3# python -c "import ssl; print(ssl.get_default_verify_paths())" | sed 's/,/\n/g'
DefaultVerifyPaths(cafile='/etc/pki/server-cert.cer'
 capath='/etc/ssl/certs'
 openssl_cafile_env='SSL_CERT_FILE'
 openssl_cafile='/etc/ssl/cert.pem'
 openssl_capath_env='SSL_CERT_DIR'
 openssl_capath='/etc/ssl/certs')

[blacklabelops]

Duplicity documentation on certificate verification: http://duplicity.nongnu.org/duplicity.1.html#toc24

You need duplicity parameter --ssl-cacert-path.

Duplicity parameters are passed like this: https://github.com/blacklabelops/volumerize#duplicity-parameters

[taikedz]

Thanks for the response.

Alas my requirement is for s3:// connections, but duplicity 0.7 series only supports overriding certificates for WebDAV and LFTP (reason being, underlying boto 2 which services s3:// connections does not support ca bundle overriding). This remains the case for duplicity in Alpine 3.8 (still uses duplicity 0.7)

I'll have to figure out some other way to work around this.

Thanks