search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
341 stars 95 forks source link

ERROR - The transporter 'cumulus' has failed while transporting the file #132

Closed sobi3ch closed 11 years ago

sobi3ch commented 11 years ago

I've install filecoveyor using this article and with pip , because 

$ python26 setup.py install

didn't work for me (there was an error with setuptools)

I've use touch command to change file and fileConveyor detect it fine but couldn't upload it (errro) Error during uploading one file (debug on) : http://pastebin.com/kCtKHUsg

anyone have a clue why is happening??

sobi3ch commented 11 years ago

I follow this guidelines http://jamestombs.co.uk/2012-11-12/rackspace-cloud-files-cdn-fileconveyor-drupal-6-not-easy-it-seems

sobi3ch commented 11 years ago

can anyone help? please

chrisivens commented 11 years ago

Is that for a UK cloudfiles account?

sobi3ch commented 11 years ago

yes - I just follow tut from http://jamestombs.co.uk/2012-11-12/rackspace-cloud-files-cdn-fileconveyor-drupal-6-not-easy-it-seems where guy suggest to use yours branch 'no-delete' to work with UK rackspace

If you need anything more let me know... thx

=========== MORE DEBUG messages ===========

2012-12-03 04:15:48,673 - Arbitrator                - WARNING  - Fully up and running now.
2012-12-03 04:15:48,811 - Arbitrator                - INFO     - Pipeline queue -> filter queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:15:48,812 - Arbitrator                - INFO     - Filtering: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' matches the 'JS, CSS, Images' rule for the 'test' source!
2012-12-03 04:15:48,812 - Arbitrator                - INFO     - Filter queue -> process queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' (rule: 'JS, CSS, Images').
2012-12-03 04:15:48,823 - Arbitrator                - INFO     - Process queue -> transport queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:15:48,824 - Arbitrator                - DEBUG    - Process queue: started the '' processor chain for the file '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:15:48,825 - Arbitrator                - WARNING  - Created 'cumulus' transporter for the 'rackspace' server.
2012-12-03 04:15:48,832 - Arbitrator                - INFO     - Transport queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' to transfer to server 'rackspace' with transporter #0 (of 1), place 1 in the queue.
2012-12-03 04:15:49,334 - Arbitrator.Transporter    - DEBUG    - Running the transporter 'cumulus' to sync '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:15:49,876 - Arbitrator.Transporter    - ERROR    - The transporter 'cumulus' has failed while transporting the file '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' (action: 1). Error: ''.
2012-12-03 04:15:50,510 - Arbitrator                - WARNING  - Retry queue -> 'failed_files' persistent list: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'. Retrying later.
Exception in thread FSMonitorThread:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/piotrek/fileconveyor/fileconveyor/fsmonitor_inotify.py", line 122, in run
    self.__process_queues()
  File "/home/piotrek/fileconveyor/fileconveyor/fsmonitor_inotify.py", line 155, in __process_queues
    self.__add_dir(path, event_mask)
  File "/home/piotrek/fileconveyor/fileconveyor/fsmonitor_inotify.py", line 76, in __add_dir
    raise FSMonitorError, "Could not monitor '%s', reason: %s" % (path, e)
FSMonitorError: Could not monitor '/var/www/phoenix', reason: add_watch: cannot watch /var/www/phoenix/sites/default/files/imagefield_thumbs/otd/ticket_description/60_minute_flight WD=-1, Errno=No space left on device (ENOSPC)

2012-12-03 04:16:10,916 - Arbitrator                - WARNING  - Signaling to stop.
2012-12-03 04:16:11,031 - Arbitrator                - WARNING  - Stopping.
chrisivens commented 11 years ago

In that case, there isn't a way in config.xml yet to tell it to use the UK url so the error you are getting is an auth one but there's no proper feedback that that is the case.

Try adding this to your settings file:

CUMULUS = {
  'AUTH_URL' : 'uk_authurl'
}
jamestombs commented 11 years ago

That's my tutorial and how I got it working after this issue - https://github.com/wimleers/fileconveyor/issues/128

@chrisivens your django_settings.py file from the no-delete branch already includes the:

CUMULUS = {
    'AUTH_URL' : 'uk_authurl'
}```
chrisivens commented 11 years ago

I honestly haven't got enough time to take a look at this today at work but I'll try and get to look at it this evening. My changes are long overdue I'd say. I need to test it all to oblivion.

sobi3ch commented 11 years ago

same problem

my settings.py file http://pastebin.com/iwHax5fY

as you can see below error is empty > '' What should I do to see some more info whats happening?

2012-12-03 04:54:27,719 - Arbitrator                - INFO     - Transport queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' to transfer to server 'rackspace' with transporter #1 (of 1), place 1 in the queue.
2012-12-03 04:54:27,713 - Arbitrator                - INFO     - Process queue -> transport queue: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:54:27,931 - Arbitrator.Transporter    - DEBUG    - Running the transporter 'cumulus' to sync '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'.
2012-12-03 04:54:28,463 - Arbitrator.Transporter    - ERROR    - The transporter 'cumulus' has failed while transporting the file '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg' (action: 1). Error: ''.
2012-12-03 04:54:28,528 - Arbitrator                - WARNING  - Retry queue -> 'failed_files' persistent list: '/var/www/phoenix/sites/default/files/brightcove_thumbnail/96980657001_207566934001_titmouse-thumbnail.jpg'. Retrying later.
2012-12-03 04:54:35,902 - Arbitrator                - WARNING  - Signaling to stop.
2012-12-03 04:54:35,940 - Arbitrator                - WARNING  - Stopping.
sobi3ch commented 11 years ago

@chrisivens fine, waiting for evening.. thanks

sobi3ch commented 11 years ago

I forget to tell you I couldn't run $ pythong26 setup.py install because of missing setuptools package. When I try it to install via yum install python-setuptools there was a message I already have setuptools installed. I think it was problem of python version 2.6, yum was checking package for original python witch is 2.4 in my redhat box. Anyway I've try again run $ pythong26 setup.py install to copy&paste error message to issue on rackspace. And then suddenly everything just start to work! (??). I've made so many changes that I'm unable to explane this but hopefully this comment can help someone..

Finally I've install file conveyor by pip (pip install -e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor), and of course this doesn't work..

Thanks for all guys

jamestombs commented 11 years ago

Just to be certain, you are running the command python26 setup.py install not pythong26 setup.py install?

wimleers commented 11 years ago

Good catch, @tombscomputers :)

sobi3ch commented 11 years ago

@tombscomputers yes, yes just mistype this. I type it from my head..