File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
I'm currently getting the UnicodeDecodeError everytime I start up my fileconveyor system..
screen python src/fileconveyor/fileconveyor/arbitrator.py 2013-10-01 06:32:01,192 - Arbitrator - WARNING - File Conveyor is initializing. 2013-10-01 06:32:01,194 - Arbitrator - WARNING - Loaded config file. 2013-10-01 06:32:01,213 - Arbitrator - WARNING - Created 'cloudfiles' transporter for the 'Rackspace Cloud Files' server. 2013-10-01 06:32:01,213 - Arbitrator - WARNING - Server connection tests succesful! 2013-10-01 06:32:01,214 - Arbitrator - WARNING - Setup: created transporter pool for the 'Rackspace Cloud Files' server. 2013-10-01 06:32:01,215 - Arbitrator - WARNING - Setup: initialized 'pipeline' persistent queue, contains 0 items. 2013-10-01 06:32:01,216 - Arbitrator - WARNING - Setup: initialized 'files_in_pipeline' persistent list, contains 0 items. 2013-10-01 06:32:01,216 - Arbitrator - WARNING - Setup: initialized 'failed_files' persistent list, contains 0 items. 2013-10-01 06:32:01,217 - Arbitrator - WARNING - Setup: initialized 'files_to_delete' persistent list, contains 0 items. 2013-10-01 06:32:01,218 - Arbitrator - WARNING - Setup: moved 0 items from the 'files_in_pipeline' persistent list into the 'pipeline' persistent queue. 2013-10-01 06:32:01,221 - Arbitrator - WARNING - Setup: connected to the synced files DB. Contains metadata for 4970 previously synced files. 2013-10-01 06:32:01,226 - Arbitrator - WARNING - Setup: initialized FSMonitor. 2013-10-01 06:32:01,226 - Arbitrator - WARNING - Fully up and running now. Exception in thread FSMonitorThread: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in bootstrap_inner self.run() File "/root/Oterai/src/fileconveyor/fileconveyor/fsmonitor_polling.py", line 68, in run self.process_queues() File "/root/Oterai/src/fileconveyor/fileconveyor/fsmonitor_polling.py", line 99, in process_queues self.__add_dir(path, event_mask) File "/root/Oterai/src/fileconveyor/fileconveyor/fsmonitor_polling.py", line 45, in add_dir FSMonitor.generate_missed_events(self, path) File "/root/Oterai/src/fileconveyor/fileconveyor/fsmonitor.py", line 130, in generate_missed_events for event_path, result in self.pathscanner.scan_tree(path): File "/root/Oterai/src/fileconveyor/fileconveyor/pathscanner.py", line 239, in scan_tree for subpath, subresult in self.scan_tree(os.path.join(path, filename)): File "/root/Oterai/src/fileconveyor/fileconveyor/pathscanner.py", line 239, in scan_tree for subpath, subresult in self.scan_tree(os.path.join(path, filename)): File "/root/Oterai/src/fileconveyor/fileconveyor/pathscanner.py", line 226, in scan_tree result = self.scan(path) File "/root/Oterai/src/fileconveyor/fileconveyor/pathscanner.py", line 190, in scan for path, filename, mtime, is_dir in self.listdir(path): File "/root/Oterai/src/fileconveyor/fileconveyor/pathscanner.py", line 77, in listdir path_to_file = os.path.join(path, filename) File "/usr/lib/python2.7/posixpath.py", line 80, in join path += '/' + b UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 3: ordinal not in range(128)