Closed tkd4444 closed 4 years ago
Hi David,
Yes, it should be :
$ mkdir /tmp/{src,dst}
$ touch /tmp/src/{foo,bar,baz}
$ fpsync -o '-lptgoD -v --numeric-ids --exclude=bar' /tmp/src /tmp/dst
$ ls /tmp/dst
baz foo
Yeah, just tried that but it still syncs that folder. I think it is worth to mention that I'm trying to exclude a folder and not a file.
sudo "PATH=$PATH" /usr/local/bin/fpsync -o '-lptgoD -v --numeric-ids --exclude=cache' \
`pwd`/source/ `pwd`/destination/
It still copies over the cache folder which I'm trying to exclude.
Right, this is because fpsync calls fpart which, by default, generates a list of files to synchronize. Thus, excluding a directory at rsync level cannot work because it will always create a parent directory even if it is excluded :
$ mkdir /tmp/{src,dst}
$ mkdir /tmp/src/{foo,bar,baz}
$ touch /tmp/src/{foo,bar,baz}/file
$ fpsync -o '-lptgoD -v --numeric-ids --exclude=bar' /tmp/src /tmp/dst
$ ls /tmp/dst/bar
file
Here, rsync created the 'bar' directory because it was asked to synchronize bar/file.
You have to exclude the directory earlier by using fpart options :
$ fpsync -O '-x bar' /tmp/src /tmp/dst
$ ls /tmp/dst
baz foo
That way, bar/file will not appear at all in the (internal) file list to synchronize.
@martymac thanks for your assistance on this. Will it accept path in the -x
arg ? As it's now it's excluding everything from /tmp/src
with a name bar
e.g.
fpsync -O '-x /bar' /tmp/src /tmp/dst
Thanks !
This is the intended behaviour. Unfortunately, there is currently no way to exclude a specific path from fpart as -x only acts on file or directory names (not paths, but that could be an improvement).
Coming back to my first answer, rsync accepts relative paths in its --exclude option, so you may (this is to be tested) be able to exclude all files from that directory with something like : --exclude=bar/*
Note that fpart will crawl that directory and add its files to the produced partitions anyway.
Best regards, Ganael.
Hello David,
I've implemented ability to specify paths with options '-y', '-Y', '-x' and '-X', that should solve your problem. An exclusion will be considered to be a path when it contains at leat a '/' character. You can mix file names and paths and inclusion/exclusion rules remain the same as before.
Hope this helps,
Best regards,
Ganael.
Hello Ganael,
Thanks for spending time on this ! Much appreciated.
All the best ! David
Hey @martymac Downloaded and installed the v1.1.0
I just tried: sudo "PATH=$PATH" /usr/local/bin/fpsync -n $_thread_count -v -o "-a --stats --numeric-ids" -O '-X /cache' /tmp/src/ /tmp/dest/
and nothing really changed. It's still copying over the entire cache folder.
Thanks !
Hi David,
The fix is not in 1.1.0, but in git only (master branch) ; it will be integrated in the next release. You can rebuild fpart from source to get it now if necessary.
Best regards,
Ganael.
I tried to use the master and compiled it but it still copies over the directory.
Used:
sudo "PATH=$PATH" /usr/local/bin/fpsync -n $_thread_count -v -o "-a --stats --numeric-ids" -O '-X /cache' /tmp/src/ /tmp/dest/
$ fpsync
fpsync v1.2.0 - Sync directories in parallel using fpart
Copyright (c) 2014-2019 Ganael LAPLANCHE <ganael.laplanche@martymac.org>
Hi David,
Excluded path must patch the root path of the crawled filesystem. In your case, you should exclude the following path : '/tmp/src/cache'. Can you try that ?
Hey Ganael,
Nope, tried it and it still copies it over.
sudo "PATH=$PATH" /usr/local/bin/fpsync -n $_thread_count -v -o "-a --stats --numeric-ids" -O '-X '`pwd`'/src/cache' `pwd`/src/ `pwd`/trg/
Hi,
are you sure fpsync uses the latest version of fpart you compiled and not a previous version of the binary you installed before ?
Hey,
Yes, looks like it.
$ sudo /usr/local/bin/fpsync -v
fpsync v1.2.0 - Sync directories in parallel using fpart
Copyright (c) 2014-2019 Ganael LAPLANCHE <ganael.laplanche@martymac.org>
WWW: http://contribs.martymac.org
Usage: /usr/local/bin/fpsync [OPTIONS...] src_dir/ dst_url/
Usage: /usr/local/bin/fpsync [-r jobname] [OPTIONS...]
Hello,
I mean, check that the fpsync script itself uses the latest version of fpart. That latest version should be in your $PATH; it is determined that way by the script :
FPART_BIN="$(which fpart)"
Best regards,
Ganael.
Yes it is.
fpart v1.2.0
Copyright (c) 2011-2019 Ganael LAPLANCHE <ganael.laplanche@martymac.org>
WWW: http://contribs.martymac.org
Build options: debug=no, fts=system
OK, I see.
fpart exclusions, when used through fpsync, are relative to the src path (because fpsync changed its current working directory to the src dir), so you should use the following :
sudo "PATH=$PATH" /usr/local/bin/fpsync -n $_thread_count -v -o "-a --stats --numeric-ids" -O '-X ./cache'
pwd/src/
pwd/trg/
Remember you can build fpart with the --enable-debug option to get feedback about included/excluded files and directories.
Hey Ganael,
This did the trick:
'-X ./cache'
Thanks so much for your assistance !
Hey @martymac
Is the --exclude option supported in file mode ?
Thanks so much !