Sub-directories and files are not getting copied to the destination

[harshbahua]

I am trying to run fpsync -n ${threads} -v src/directory dest/directory on a ~5GB files. The command is executed in bash script. I see the output as: 1684539262 ===> Analyzing filesystem... 1684539265 <=== Fpart crawling finished 1684539266 <=== Parts done: 77/77 (100%), remaining: 0 1684539266 <=== Time elapsed: 4s, remaining: ~0s (~0s/job) 1684539266 <=== Fpsync completed without error in 4s.

While inspecting the target or destination directory it appears that only ~50MB are getting copied.

• Not all sub-directories inside src/directory are getting copied eg: src/directory/sub-directory
• Not all files inside src/directory are getting copied eg: src/directory/files
• Not all files inside sub-directories are getting copied eg: src/directory/sub-directory/files

Am I missing something over here?

[frukto]

Have a look at the files in /tmp/fpsync/work/*/* .They contain the individual job commands. Check if they look in order. Maybe it is an issue with file system borders or symlinks.

[martymac]

Hi Harsh,

frukto's advice is good : try to see if your files/dirs have been included in partitions generated by fpart. If yes, then there was a problem during file replication (maybe wrong access rights ?). Those problems should appear in log files (see /tmp/fpsync/log/xxx/*.stderr).

If not, then the problem is before replicating the files, when generating partitions. Try to reproduce the problem by calling fpart manually (check options -l and -x). The following kind of command will produce partitions to stdout :

$ fpart -L -f 10000 src/directory dest/directory

You can grep that and also redirect stdout to /dev/null to see if fpart encounters read errors that would explain your missing files.

Hope this helps,

Ganael.

[harshbahua]

Hi,

Thanks for the response. I have tried few things and here are the observations:

• The commands at /tmp/fpsync/work/*/* look as expected.

  • Eg: /bin/sh -c ' /usr/bin/rsync -lptgoD -v --numeric-ids -d --files-from="/tmp/fpsync/parts/1684777057-4342/part.1" --from0 "/environment-content/extracted/server-root//" "/mnt/efs/tempManifest2/"' 1>/tmp/fpsync/log/1684777057-4342/1.stdout 2>/tmp/fpsync/log/1684777057-4342/1.stderr

• Checked for the stderr files as well and they are all empty.

• Checked the stdout files and I can see the file paths being present there.

• On running fpart -L -f 10000 src/directory dest/directory I see that the file paths are being generated and no errors thrown.

• I am currently running v1.5.1

• Another interesting observation: I am running the fpsync command in a script and it completes quickly. But when I induce sleep after fpsync command in the script, I see that the files are getting copied to destination directory even after fpsync command is completed.

Thanks, Harsh

[martymac]

Hello Harsh,

Are your running fpsync on Debian ?

If yes, then your problem may be related to Dash not allowing job control without a tty in non-interactive mode :

https://lore.kernel.org/dash/7091680.J8PY2HnTC3@home.martymac.org/T/#u

(there is a patch under review here: https://patchwork.kernel.org/project/dash/patch/dedaa3fa370ea9c4aeb1771b5568a7bef4065b04.1675113321.git.steffen@sdaoden.eu/)

Try running the same fpsync command manually (without embedding it in a bash script). If that works, then change fpsync's shebang and use bash instead, that should fix your problem.

Best regards,

Ganael.

[martymac]

Hello Harsh,

Assuming everything went well, I'll close this bug report for now. Feel free to re-open it if necessary.

Best regards,

Ganael.

[tghartland]

Hi,

I've just come across an issue that has pretty much the same symptoms to this (incomplete transfer, adding a sleep command after fpsync exits "fixes" it).

In my case it was because I was running in a containerised environment, rockylinux:8 base image with fpart package installed from EPEL. This container image doesn't include the ps command by default, so when fpsync calls ps to look for running rsync processes, it doesn't find them and assumes that the transfer has successfully completed. https://github.com/martymac/fpart/blob/a2fe718d6c8481f8c255916f0413dcb46fc68b1d/tools/fpsync#L780 After fpsync immediately exits, the container is stopped and all the running rsync processes are killed.

With the procps-ng package installed, fpsync correctly waits until the entire transfer is complete.

I would suggest that either the package providing ps is set as a dependency of the fpart package, or at least that the existence of ps is checked in the script as some other binaries are.

Thomas

[martymac]

Hi Thomas,

Thanks for your feedback. Indeed, 'ps' is necessary for fpsync to work. I've modified fpsync to make it check for that command, as well as a few others. You should probably contact the EPEL package maintainer to add an appropriate dependency too...

Best regards,

Ganael.

[tghartland]

Thanks for the quick reply! I'll see what I can do about the EPEL package.