thelastpickle / cassandra-medusa

Apache Cassandra Backup and Restore Tool
Apache License 2.0
264 stars 142 forks source link

Issue on DISA-STIG RHEL7 with FIPS encryption mode #230

Closed bkrajmalnik1 closed 2 years ago

bkrajmalnik1 commented 3 years ago

I just installed medusa on 2 test environments. One is a regular RHEL 7 deployment, whereas the second is a hardened deployment incorporating FIPS encryption. While performing a backup on the hardened environment, the schema is being exported to the local backup location, but at that point the system halts for an extended amount of time and then exits. After the initial failure, I set selinux to permissive mode to rule out anything cause by a SELinux policy issue.

The stack trace for this is shown below.

[root@rh7-stig medusa]# /opt/rh/rh-python36/root/bin/medusa backup [2020-11-18 18:01:20,698] INFO: Monitoring provider is noop [2020-11-18 18:01:20,794] WARNING: is ccm : 0 [2020-11-18 18:01:21,092] INFO: Creating snapshot [2020-11-18 18:01:21,092] INFO: Saving tokenmap and schema [2020-11-18 18:09:41,541] ERROR: This error happened during the backup: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips Traceback (most recent call last): File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/cassandra_medusa-0.8.0.dev0-py3.6.egg/medusa/backup.py", line 190, in main node_backup.schema = schema File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/cassandra_medusa-0.8.0.dev0-py3.6.egg/medusa/storage/node_backup.py", line 137, in schema self._storage.storage_driver.upload_blob_from_string(self.schema_path, schema) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/retrying-1.3.3-py3.6.egg/retrying.py", line 49, in wrapped_f return Retrying(_dargs, _dkw).call(f, _args, _kw) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/retrying-1.3.3-py3.6.egg/retrying.py", line 212, in call raise attempt.get() File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/retrying-1.3.3-py3.6.egg/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/six.py", line 703, in reraise raise value File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/retrying-1.3.3-py3.6.egg/retrying.py", line 200, in call attempt = Attempt(fn(_args, *_kwargs), attempt_number, False) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/cassandra_medusa-0.8.0.dev0-py3.6.egg/medusa/storage/abstract_storage.py", line 65, in upload_blob_from_string object_name=str(path) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/libcloud/storage/drivers/local.py", line 537, in upload_object_via_stream return self._make_object(container, object_name) File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/libcloud/storage/drivers/local.py", line 178, in _make_object data_hash = self._get_hash_function() File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/libcloud/storage/base.py", line 907, in _get_hash_function func = getattr(hashlib, self.hash_type)() ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

Is there any additional information I could possibly provide to assist in the troubleshooting?

┆Issue is synchronized with this Jira Task by Unito ┆Issue Number: K8SSAND-313

bkrajmalnik1 commented 3 years ago

Hi Alexander,

I did some additional research and believe I have found the issue. On systems running FIPS, md5 is not considered a safe algorithm and mostly disallowed. I found that hashlib.md5 can accept a flag which will allow the usage of MD5 in FIPS environments, using the form hashlib.md5(usedforsecurity=False)

I will try to look for all the instances where this is being called to see if I can fix the problem using md5. Another option, I suppose, would be to use sha256 instead of md5, but I do not know what effects this can have on the overall functionality. I am only using local storage, so I suppose it should be safe, but if md5 is used as part of the transfer protocol with some of the other storage drivers, this may break.

bkrajmalnik1 commented 2 years ago

I found the solution. The fix is outside of the medusa codebase, and rather in Apache Libcloud. libcloud/storage/base.py

The below function needs to have the usedforsecurity=False argument passed, as shown below

def _get_hash_function(self):
    """
    Return instantiated hash function for the hash type supported by
    the provider.
    """
    try:
        func = getattr(hashlib, self.hash_type)(usedforsecurity=False)
    except AttributeError:
        raise RuntimeError('Invalid or unsupported hash type: %s' %
                           (self.hash_type))

    return func

That being said, I recommend that in the medusa codebase the hashlib.md5() calls be modified to pass the usedforsecurity=False argument, which allow the MD5 to be generated by OpenSSL.