search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
341 stars 95 forks source link

Database integrity error #79

Closed cerw closed 11 years ago

cerw commented 13 years ago

Why do i get this when i do 1st Sync?

2011-08-22 17:22:27,318 - Arbitrator - WARNING - Synced: '/var/www/beachsafe.org.au/www/jquery-mobile/themes/valencia/images/texture_05.png' (CREATED). 2011-08-22 17:22:27,318 - Arbitrator - CRITICAL - Database integrity error: columns input_file, server are not unique. Duplicate key: input_file = '/var/www/beachsafe.org.au/www/jquery-mobile/themes/valencia/images/icon-search-black.png', server = 'scp push cdn'.

wimleers commented 13 years ago

Did you update the code and were you using File Conveyor before the recent (August) commits?

Then you'll need to run the new upgrade.py script. This has been documented in README.txt, too.

Any suggestions on how to communicate this better?

cerw commented 13 years ago

My fault, i did not see that command, i always just delete the *.db when I add new source is that right? Or upgrade.py will do that for me?

Also S3 transporter is not wokring for me at all, and on SCP after while it dies and i get

raise RuntimeError("threads can only be started once")

RuntimeError: threads can only be started once

Any ideas? Otherwise great project/idea!

Petr

wimleers commented 13 years ago

Very strange, in that case. If you deleted your .db files, then you should not be able to see this error. Are you sure you deleted all of them?

Also, are you sure that you really updated all code? Maybe you accidentally updated a subset of the files?

cerw commented 13 years ago

I think it's bit confusing, I tough the upgrade will update my *.db file, this is my 1st installation. And looks like when i run it in screen I mostly get error from SCP transporter. Also fileconveyor only does one source and then it stops? Nothing in log file .. I must be missing something?

cerw commented 13 years ago

http://pastebin.com/VaRGD7Cq config.xml

wimleers commented 13 years ago

The SFTP/SCP support was only recently contributed, maybe you're running into an unknown bug (there is no known bug with it). Please try without that destination for a minute.

And no, File Conveyor does 1) never stop, 2) definitely not stop after one source.

To track down the cause of this problem, you could change your settings.py file to configure the logging to the DEBUG level. (Also see the README.txt for that.)

j0rd commented 12 years ago

I'm also getting this database corruption error with the latest code installed yesterday with the link from the fileconveyor.com website.

wimleers commented 12 years ago

@j0rd Can you reproduce this without using any processors and using only the symlink_or_copy transporter?

wimleers commented 11 years ago

More than a year without information to solve this. Closing.