search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
341 stars 95 forks source link

Support Signed, Secure and Expiring URLs with CloudFront/S3 #138

Open ghost opened 11 years ago

ghost commented 11 years ago

This pull request is mainly a solution to Issue #133. By default instantiation, S3BotoStorage automatically sets parameters so that files getting placed on S3/CloudFront require SSL, be signed by account that uploads them and sets an expiration of 3600 seconds.

This obviously was a problem for us and @yusufhm as we use this in conjunction with the CDN module on Drupal. I have made attempts to make this as clean as possible but my python experience is extremely limited. The following configuration options are available for the S3 and CloudFront transports:

Cloudfront:

  <servers>
    <server name="cloudfront example" transporter="cf">
      <bucket_name>mybucket</bucket_name>
      <access_key_id>myaccesskey</access_key_id>
      <secret_access_key>mysecretaccesskey</secret_access_key>
      <distro_domain_name>CNAME or cloudfront.net address</distro_domain_name>
      <!-- only required if not using *.cloudfront.net in distro_domain_name -->
     <secure_distro_domain_name>example.cloudfront.net<secure_distro_domain>
      <use_signed_urls>false</use_signed_urls>
      <use_secure_urls>false</use_secure_urls>
      <!-- expiration is stored in seconds -->
      <set_url_expiration>0</set_url_expiration>
    </server>
  </servers>

S3:

  <servers>
    <server name="s3 example" transporter="s3">
      <bucket_name>mybucket</bucket_name>
      <access_key_id>myaccesskey</access_key_id>
      <secret_access_key>mysecretaccesskey</secret_access_key>
      <use_signed_urls>false</use_signed_urls>
      <use_secure_urls>false</use_secure_urls>
      <!-- expiration is stored in seconds -->
      <set_url_expiration>0</set_url_expiration>
    </server>
  </servers>

Let me know what you think, I'm sure it probably needs some polishing but I wanted to do what I could to help out.

yusufhm commented 11 years ago

Man, that's great! I'll test it later tonight and let you know how it goes!

ghost commented 11 years ago

Yeah let me know if you have any problems, we're already using it in a production environment and it seems to work without a hitch so far.