search

hertell / lsyncd

Automatically exported from code.google.com/p/lsyncd
GNU General Public License v2.0
0 stars 0 forks source link

Copy before write is finished #77

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Download a ISO or an other big file to an lsyncd controlled directory (wget)
2. in the lsyncd logfile, you will see that lsyncd start a copy to the mirrors 
if the download are complete or not.
3. The file is corrupt on the mirrors and will not updated any time.
   (this is an other issue that is listed)

What version of the product are you using? On what operating system?
2.04 from ubuntu

Original issue reported on code.google.com by fragst...@gmail.com on 16 Aug 2011 at 6:27

GoogleCodeExporter commented 9 years ago 
In my case it works as intended. During the download a rsync starts and 
transmitts a corrupt file - this is because Lsyncd using an asynchronous 
monitoring interface cannot know if someone is still accessing the file. After 
the file finished it receives a "CLOSE_WRITE" and starts another rsync and that 
will overwrite-repair the corrupt trunks. See if there is a rsync process 
running, and if wait a little longer?

Original comment by axk...@gmail.com on 16 Aug 2011 at 6:52

GoogleCodeExporter commented 9 years ago 
So i have looked deeper on my systems. i see only one rsync process that is 
running. after the wget is finished this file will not update this file. it's a 
special configuration needet fot this "overwrite-repair" ?

my configuration (importent configurations)

settings: delay=60; statusInterval=60; maxDeplay=60
sync: default.rsyncssh; excludeFrom; rsyncOps="-l6tus"

Original comment by fragst...@gmail.com on 16 Aug 2011 at 8:16

GoogleCodeExporter commented 9 years ago 
I just tried it again with rsyncssh, for me it works. Pleases wait until the 
download is finished, the 60 time delay after it is over and no rsync are 
running anymore. 

If the problem persists, please make a minimal test case, provide complete 
configuration and a complete logfile (-log all)

Original comment by axk...@gmail.com on 16 Aug 2011 at 8:44

GoogleCodeExporter commented 9 years ago 
Hi,

i will send you this information directly and not over the portal.
the logfile are without corrections of server or usernames.
please delete after watching.

the timecode if the logfile will be interesting are 23:49:09

the original file has a size of: 493.879.296 and the transfered file of: 
135.642.256

the tar file included the logfile and my configuration.

thx for help
Thomas

Original comment by fragst...@gmail.com on 16 Aug 2011 at 10:13

GoogleCodeExporter commented 9 years ago 
As discussed by email, there seems to be an issue with timestamps and --update. 
Whats the exact timestamps?

Please run `stat FILENAME' on source and target. If you manage also possibly 
between first and second rsync call.

Or run "while true; do stat FILENAME; sleep 1; done;" on the target to get a 
log how it changes.

Original comment by axk...@gmail.com on 18 Aug 2011 at 9:43

GoogleCodeExporter commented 9 years ago 
i think it's an error of the rsync process

source 
stat debian-6.0.2.1-amd64-netinst.iso
  File: „debian-6.0.2.1-amd64-netinst.iso“
  Size: 176160768   Blocks: 344408     IO Block: 4096   reguläre Datei
Device: 90e3h/37091d    Inode: 51692368    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    lala)   Gid: (    0/    lala)
Access: 2011-08-18 16:21:24.000000000 +0200
Modify: 2011-06-28 14:59:06.000000000 +0200
Change: 2011-08-18 16:21:24.000000000 +0200

remote
stat debian-6.0.2.1-amd64-netinst.iso 
  File: „debian-6.0.2.1-amd64-netinst.iso“
  Size: 53728356    Blocks: 104944     IO Block: 4096   reguläre Datei
Device: 811h/2065d  Inode: 58733658    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/lala)   Gid: ( 1000/lala)
Access: 2011-08-18 16:19:25.309667468 +0200
Modify: 2011-08-18 16:18:54.000000000 +0200
Change: 2011-08-18 16:19:25.309667468 +0200

the modification times are older on the remote site as this on the source.

Original comment by fragst...@gmail.com on 18 Aug 2011 at 2:35

GoogleCodeExporter commented 9 years ago 
Not the modify timestamp! The timestamps on the source are quite strange..

Original comment by axk...@gmail.com on 18 Aug 2011 at 2:43

GoogleCodeExporter commented 9 years ago 
Sure! The timestamp come from the remote server where i have downloaded this 
file

Original comment by fragst...@gmail.com on 18 Aug 2011 at 3:09

GoogleCodeExporter commented 9 years ago 
So rsync didn't preserve the modification timestamp on the first sync, like -t 
should tell it to?

Original comment by axk...@gmail.com on 18 Aug 2011 at 3:13

GoogleCodeExporter commented 9 years ago 
Any news about the timestamp issue?

Original comment by axk...@gmail.com on 26 Aug 2011 at 7:54

GoogleCodeExporter commented 9 years ago 
sorry. i have not spend time for this project but i think that the problems can 
be occurd by using an ubuntu alpha version on the second site.

if i have any news i will comment this here

Original comment by fragst...@gmail.com on 29 Aug 2011 at 5:51

GoogleCodeExporter commented 9 years ago 
so i have checkt this issue on two other systems.
the result is the same.

he start a copy and on finish the files will not resynced

the tests was made with version 2.0.5 and an centos 6 system as destination 
target and an debian/ubuntu system as source.

i have used other systems as before to exclude an issue from an virtual 
maschine.

Original comment by fragst...@gmail.com on 14 Sep 2011 at 3:00

GoogleCodeExporter commented 9 years ago 
You can easily workaround the issue by removing -u "--update". Otherwise you 
would need to find out why rsync creates the file with wrong timestamps in your 
case.

Original comment by axk...@gmail.com on 16 Sep 2011 at 10:14