search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
340 stars 95 forks source link

Threading and the sftp transporter #88

Open kalkwarf opened 12 years ago

kalkwarf commented 12 years ago

I have a trivial configuration file which contains one source, no filters, no processors, and one sftp destination.

The server portion of my config file looks like this:

<servers>
   <server name="server.local" transporter="sftp">
     <host>server.local</host>
     <username>kalkwarf</username>
     <password>(my password here)</password>
     <url>http://server.local/Shared Items/testo/</url>
   </server>
 </servers>

When I try to start the arbitrator process, an exception is raised, causing the process to terminate:

Traceback (most recent call last):
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/arbitrator.py", line 1058, in <module>
   run_file_conveyor()
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/arbitrator.py", line 1041, in run_file_conveyor
   arbitrator = Arbitrator(os.path.join(sys.path[0], "config.xml"), restart)
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/arbitrator.py", line 168, in __init__
   transporter = self.__create_transporter(server)
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/arbitrator.py", line 872, in __create_transporter
   transporter = transporter_class(settings, self.transporter_callback, self.transporter_error_callback, "Arbitrator")
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/transporters/transporter_sftp.py", line 36, in __init__
   self.storage._start_connection()
 File "/Volumes/Spinning/RandomSource/fileconveyor/code/dependencies/storages/SFTPStorage.py", line 109, in _start_connection
   t.connect(username=self._config['user'], password=self._config['passwd']) #,hostkey=hostkey
 File "build/bdist.macosx-10.7-intel/egg/paramiko/transport.py", line 989, in connect
 File "build/bdist.macosx-10.7-intel/egg/paramiko/transport.py", line 454, in start_client
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 467, in start
   raise RuntimeError("threads can only be started once")
RuntimeError: threads can only be started once

The exception is raised here:

#Use password auth
t.connect(username=self._config['user'], password=self._config['passwd']) #,hostkey=hostkey

which makes me wonder if its a problem restricted to username:password authentication, or if it's a more general problem.

wimleers commented 12 years ago

I'm going to try to reproduce this tomorrow. I can't promise that though.