search

wimleers / fileconveyor

File Conveyor is a daemon written in Python to detect, process and sync files. In particular, it's designed to sync files to CDNs. Amazon S3 and Rackspace Cloud Files, as well as any Origin Pull or (S)FTP Push CDN, are supported. Originally written for my bachelor thesis at Hasselt University in Belgium.
https://wimleers.com/fileconveyor
The Unlicense
341 stars 95 forks source link

Not syncing new files on CentOS 6.3 #148

Open software2 opened 11 years ago

software2 commented 11 years ago

I have set up fileconveyor on a CentOS6.3 box which I intend to use to host several Magento commerce platforms.

FileConveyor is configured and works on startup (correctly syncing all the changed files) but does not seem to be picking up file changes while running.. See this simple test, where I:

magetest1 .... magetest2

Any advice would be appreciated Ryan

edmooney commented 11 years ago

By chance are the files you're watching on a mounted windows share? If so it wont detect the changes. File Conveyor uses inotify to be alerted to changes on the linux file system. On startup I'm pretty sure it uses polling, which is why it finds the files after restarting.

software2 commented 11 years ago

I'm not sure, not being a massive linux person.. as far as I am aware it is the local file system.. Would the mount results answer that question?

~$ mount /dev/xvda1 on / type ext3 (rw,noatime,barrier=0) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) none on /proc/xen type xenfs (rw)

How efficient is polling? Worst case is I am going to have to set a cron to restart fileconveyor!

wimleers commented 11 years ago

@software2 also first contacted me via e-mail, and for once I briefly replied there instead of directing him

I suspected the same as @edmooney:

It sounds like you're on a Linux system, which would mean that inotify is what powers instant file system change tracking. That doesn't work on networked file systems like gluster and NFS. Any chance you're using something like that?

I'm no Linux expert myself either, so I don't know how to interpret the mount output you provided, @software2. But it sure looks like you're on a file system where inotify does not work, or you must have somehow disabled inotify.

Could you please verify that inotify is working correctly by installing the inotify-tools package and then doing something like http://exyr.org/2011/inotify-run/? :) Thanks!