search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
341 stars 95 forks source link

Can't seem to get arbitrator.py to run with S3 #107

Open paperscissors opened 12 years ago

paperscissors commented 12 years ago

I'm getting a pretty nondescript error when I try to run arbitrator.py, aside from some deprecated warnings. Not sure if it's something I'm doing with my config.xml or there's some bug I don't know about. Here's the errors:

/sbin/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated from sets import Set, ImmutableSet 2012-02-26 21:46:19,619 - Arbitrator - WARNING - File Conveyor is initializing. 2012-02-26 21:46:19,623 - Arbitrator - WARNING - Loaded config file. /usr/lib/python2.6/site-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

Here's my (slightly modified) config.xml:

(uploaded to cloudapp, this strips out XML tags: http://cl.ly/2Y2d2D2r3s2M1z3H2g3N

Any ideas? Thanks.

paperscissors commented 12 years ago

it looks like this is a combination of a malformed config.xml and a python depreciation error. i can't make too much of this error message, other than it is tanking on creating the transporter.

2012-03-15 22:36:13,773 - Arbitrator - WARNING - File Conveyor is initializing. 2012-03-15 22:36:13,777 - Arbitrator - WARNING - Loaded config file. Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

mnestor commented 12 years ago

Fixed in https://github.com/mnestor/fileconveyor

paperscissors commented 12 years ago

thanks, i knew i wasn't crazy.

paperscissors commented 12 years ago

I'm actually still seeing errors, but most are different, particularly the SECRET_KEY one. not sure if this refers to the "secret_access_key" value in the S3 server transporter group, but otherwise there's no secret_key value on config.xml that I know of.

python arbitrator.py /opt/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated from sets import Set, ImmutableSet 2012-03-27 02:51:17,326 - Arbitrator - WARNING - File Conveyor is initializing. 2012-03-27 02:51:17,329 - Arbitrator - WARNING - Loaded config file. /usr/lib/python2.6/site-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO /usr/lib/python2.6/site-packages/django/conf/init.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty. warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning) Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

mrcwinn commented 12 years ago

This is exactly what I see. I fixed the django SECRET_KEY error by editing my django global settings file (this is confusing but it is different than the SECRET_ACCESS_KEY required by S3). I'm still getting a slew of errors, however:

/usr/local/lib/python2.6/dist-packages/boto/utils.py:44: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2, os, StringIO Traceback (most recent call last): File "arbitrator.py", line 1183, in run_file_conveyor() File "arbitrator.py", line 1166, in run_file_conveyor arbitrator = Arbitrator(os.path.join(FILE_CONVEYOR_PATH, "config.xml"), restart) File "arbitrator.py", line 151, in init transporter = self.__create_transporter(server) File "arbitrator.py", line 927, in __create_transporter except ConnectionError, e: NameError: global name 'ConnectionError' is not defined

chrisivens commented 12 years ago

Looks like ConnectionError is not included in the file from transporters.transporter