File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
I'm having trouble getting fileconveyor to sync files from my server to cloudfiles. The installation and setup seemed to work fine, I run the arbitrator.py and it does not throw any errors. it just does not find any files to sync. I've read the post about fileconveyor design, and I've tried making many alteration to the config XML file, but nothing I do seems to work.
My output is as follows
/usr/local/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated
from sets import Set, ImmutableSet
2013-02-22 19:11:04,815 - Arbitrator - WARNING - File Conveyor is initializing.
2013-02-22 19:11:04,817 - Arbitrator - WARNING - Loaded config file.
/usr/local/lib/python2.6/dist-packages/django/conf/init.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)
2013-02-22 13:11:04,912 - Arbitrator - WARNING - Created 'cloudfiles' transporter for the 'mybin' server.
2013-02-22 13:11:04,912 - Arbitrator - WARNING - Server connection tests succesful!
2013-02-22 13:11:04,913 - Arbitrator - WARNING - Setup: created transporter pool for the 'mybin' server.
2013-02-22 13:11:04,914 - Arbitrator - WARNING - Setup: initialized 'pipeline' persistent queue, contains 0 items.
2013-02-22 13:11:04,915 - Arbitrator - WARNING - Setup: initialized 'files_in_pipeline' persistent list, contains 0 items.
2013-02-22 13:11:04,916 - Arbitrator - WARNING - Setup: initialized 'failed_files' persistent list, contains 0 items.
2013-02-22 13:11:04,916 - Arbitrator - WARNING - Setup: initialized 'files_to_delete' persistent list, contains 0 items.
2013-02-22 13:11:04,917 - Arbitrator - WARNING - Setup: moved 0 items from the 'files_in_pipeline' persistent list into the 'pipeline' persistent queue.
2013-02-22 13:11:04,917 - Arbitrator - WARNING - Setup: connected to the synced files DB. Contains metadata for 0 previously synced files.
2013-02-22 13:11:04,994 - Arbitrator - WARNING - Setup: initialized FSMonitor.
2013-02-22 13:11:04,996 - Arbitrator - WARNING - Fully up and running now.
I'm having trouble getting fileconveyor to sync files from my server to cloudfiles. The installation and setup seemed to work fine, I run the arbitrator.py and it does not throw any errors. it just does not find any files to sync. I've read the post about fileconveyor design, and I've tried making many alteration to the config XML file, but nothing I do seems to work.
My output is as follows
/usr/local/src/fileconveyor/fileconveyor/filter.py:10: DeprecationWarning: the sets module is deprecated from sets import Set, ImmutableSet 2013-02-22 19:11:04,815 - Arbitrator - WARNING - File Conveyor is initializing. 2013-02-22 19:11:04,817 - Arbitrator - WARNING - Loaded config file. /usr/local/lib/python2.6/dist-packages/django/conf/init.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty. warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning) 2013-02-22 13:11:04,912 - Arbitrator - WARNING - Created 'cloudfiles' transporter for the 'mybin' server. 2013-02-22 13:11:04,912 - Arbitrator - WARNING - Server connection tests succesful! 2013-02-22 13:11:04,913 - Arbitrator - WARNING - Setup: created transporter pool for the 'mybin' server. 2013-02-22 13:11:04,914 - Arbitrator - WARNING - Setup: initialized 'pipeline' persistent queue, contains 0 items. 2013-02-22 13:11:04,915 - Arbitrator - WARNING - Setup: initialized 'files_in_pipeline' persistent list, contains 0 items. 2013-02-22 13:11:04,916 - Arbitrator - WARNING - Setup: initialized 'failed_files' persistent list, contains 0 items. 2013-02-22 13:11:04,916 - Arbitrator - WARNING - Setup: initialized 'files_to_delete' persistent list, contains 0 items. 2013-02-22 13:11:04,917 - Arbitrator - WARNING - Setup: moved 0 items from the 'files_in_pipeline' persistent list into the 'pipeline' persistent queue. 2013-02-22 13:11:04,917 - Arbitrator - WARNING - Setup: connected to the synced files DB. Contains metadata for 0 previously synced files. 2013-02-22 13:11:04,994 - Arbitrator - WARNING - Setup: initialized FSMonitor. 2013-02-22 13:11:04,996 - Arbitrator - WARNING - Fully up and running now.