CloudFront URL signing broken in Python 3

[mitchellrj]

Passing a filename as the private key causes it to be opened in text mode. When the rsa module looks for b'-----BEGIN RSA PRIVATE KEY-----' it cannot find it when parsing the private key file.

Opening the file in binary mode and passing the handle as the argument works around this.

Then, the _sign_string method of Distribution passes a str of message where the rsa module expects bytes.

boto==2.34.0
rsa==3.1.4

[mitchellrj]

Monkey-patch to fix Python 3 only:

import base64

from boto.cloudfront.distribution import Distribution
import six

def _sign_string(message, private_key_file=None, private_key_string=None):
    """
    Signs a string for use with Amazon CloudFront.
    Requires the rsa library be installed.
    """
    try:
        import rsa
    except ImportError:
        raise NotImplementedError("Boto depends on the python rsa "
                                  "library to generate signed URLs for "
                                  "CloudFront")
    # Make sure only one of private_key_file and private_key_string is set
    if private_key_file and private_key_string:
        raise ValueError("Only specify the private_key_file or the private_key_string not both")
    if not private_key_file and not private_key_string:
        raise ValueError("You must specify one of private_key_file or private_key_string")
    # If private_key_file is a file name, open it and read it
    if private_key_string is None:
        if isinstance(private_key_file, six.string_types):
            with open(private_key_file, 'rb') as file_handle:
                private_key_string = file_handle.read()
        # Otherwise, treat it like a file
        else:
            private_key_string = private_key_file.read()

    # Sign it!
    private_key = rsa.PrivateKey.load_pkcs1(private_key_string)
    signature = rsa.sign(message.encode('ascii'), private_key, 'SHA-1')
    return signature

def _url_base64_encode(msg):
    """
    Base64 encodes a string using the URL-safe characters specified by
    Amazon.
    """
    msg_base64 = base64.b64encode(msg)
    msg_base64 = msg_base64.replace(b'+', b'-')
    msg_base64 = msg_base64.replace(b'=', b'_')
    msg_base64 = msg_base64.replace(b'/', b'~')
    return msg_base64.decode('ascii')

Distribution._sign_string = staticmethod(_sign_string)
Distribution._url_base64_encode = staticmethod(_url_base64_encode)

[ddemid]

+1

[cilia]

Is this going to be fixed, or is it going to left like this to be used with monkey patching?

[cilia]

Just to add that if we want to generate signed URL with wildcard using 'policy_url' argument, then we also need to patch _create_signing_params method to fix the following line:

encoded_policy = self._url_base64_encode(policy.encode('ascii'))

[mekza]

Another way to be able to sign URL without patching:

from boto.cloudfront.distribution import Distribution
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
import base64

class RSADistribution(Distribution):
    def sign_rsa(self, message):
        private_key = serialization.load_pem_private_key(self.keyfile, password=None,
                            backend=default_backend())
        signer = private_key.signer(padding.PKCS1v15(), hashes.SHA1())
        message = message.encode('utf-8')
        signer.update(message)
        return signer.finalize()

    def _sign_string(self, message, private_key_file=None, private_key_string=None):
        if private_key_file:
            self.keyfile = open(private_key_file, 'rb').read()
        return self.sign_rsa(message)

    @staticmethod
    def _url_base64_encode(msg):
        """
        Base64 encodes a string using the URL-safe characters specified by
        Amazon.
        """
        msg_base64 = base64.b64encode(msg).decode('utf-8')
        msg_base64 = msg_base64.replace('+', '-')
        msg_base64 = msg_base64.replace('=', '_')
        msg_base64 = msg_base64.replace('/', '~')
        return msg_base64

[Aameer]

Thanks to @mekza I was able to get cloudfront signed urls working. I am using boto 2.37. For anyone who is facing this issue on python3.4 I made a detailed post here. Hope it will help someone

[rayluo]

Yes, in Python 3, RSA signer (also the standard base64 encoder) requires bytes as input/output. We now implement the sign feature in a Python 2/3 compatible way in Botocore. You can take a look into its implementation.

Botocore is the underlying library of Boto3. Boto3, the next version of Boto, is now stable and recommended for general use. It can be used side-by-side with Boto in the same project, so it is easy to start using Boto3 in your existing projects as well as new projects. Going forward, API updates and all new feature work will be focused on Boto3.

[mekza]

@rayluo Finally :clap: :clap: you can also add signed cookies:

def create_signed_cookies(self, url, private_key_file=None, keypair_id=None,
            expires_at=20, secure=True):
        """
        Generate the Cloudfront distirbution signed cookies
        """

        policy = self._custom_policy(
            url,
            expires_at
        )

        encoded_policy = self._url_base64_encode(policy.encode('utf-8'))
        signature = self.generate_signature(
            policy, private_key_file=private_key_file
        )
        cookies = {
            "CloudFront-Policy": encoded_policy,
            "CloudFront-Signature": signature,
            "CloudFront-Key-Pair-Id": keypair_id
        }
        return cookies

[Webinator2129]

@rayluo how should this be called now after the changes?