search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
340 stars 95 forks source link

Error: 'S3ResponseError: 403 Forbidden #124

Closed joshstrange closed 11 years ago

joshstrange commented 12 years ago

I am attempting to use S3 with file conveyor after failing at getting Rackspace cloud files working. It makes it a lot further however I am getting these errors when it tries to upload:

2012-07-16 16:08:01,192 - Arbitrator.Transporter - ERROR - The transporter 'S3' has failed while transporting the file '/var/www/www.uhaps.com/sites/default/files/css/cdn_css_http_db398cdf5ac878a29ce3d386e13876c9_admin_menu.css' (action: 1). Error: 'S3ResponseError: 403 Forbidden

I think this might be a bucket policy issue from my searching online but I couldn't get any of the examples I found online to work. Can anyone who is using S3 tell me if they had to do anything on AWS dashboard to get S3 working with fileconveyor? Thanks!

mrcwinn commented 12 years ago

I have the same issue.

The transporter 'S3' has failed while transporting the file 'tmp/somefile.jpg' (action: 1). Error: 'S3ResponseError: 403 Forbidden

However, I can use the same credentials as found in config.xml in Transmit and clearly see my bucket and have access to PUT files. I also opened up permissions to the world (temporarily) and added a bucket policy. Still the same result.

mrcwinn commented 12 years ago

I have narrowed this down. I was only able to connect by adding my keys to ~/.boto — but successive file uploads failed. This means, to me, that when FC calls S3BotoStorage, it is not passing properly the access keys, and so the upload fails.

Getting Fileconveyor to run reliably is quite a job in itself... I'm considering abandoning it in production.

andyshinn commented 11 years ago

I was running into this issue and fixed it by changing some options for S3BotoStorage in the transporter_s3.py file. I posted a copy of my file at https://gist.github.com/3715452

yusufhm commented 11 years ago

I got the same error and andyshinn's modification worked for me!

wimleers commented 11 years ago

@andyshinn or @yusufhm Could you maybe create a pull request based on that gist? Then I'll merge it in!

yusufhm commented 11 years ago

OK, I'll have to read a bit on how to do that first; I've never used github for contributing before, let alone a gist.

andyshinn commented 11 years ago

To note: I am no longer using S3 for storage so while this fix did work for me at the time, it has been a while since I used it. Should S3 change or break I won't be likely to help make future changes.

wimleers commented 11 years ago

@andyshinn Thanks for mentioning that, but since @yusufhm confirmed that this fixed it for him, I'm assuming this will most likely apply to everybody. If not, that's a hell of a coincidence :)

(And my god, what a pain it is to get Unicode handling right in Python!)