search

hjmangalam / parsyncfp

follow-on to parsync (parallel rsync) with better startup perf
Other
161 stars 19 forks source link

Trouble with spaces in the startdir path and source folder #38

Open jasonkotecki opened 3 years ago

jasonkotecki commented 3 years ago

I am trying to copy a folder that has spaces in the name as well as the startdir path to the source folder. Maybe I just have the syntax wrong, but I have tried several different ways with no luck. Below is the detail of my paths and the fpart log.

[root@cifsgw Sarah confocal]# pwd /pri/projects/pi_lab/People and projects/Sarah/Sarah confocal

[root@cifsgw Sarah confocal]# parsyncfp --NP=4 -startdir='/pri/projects/pi_lab/People and projects/Sarah/Sarah confocal' 'AAV LPS IL4' /sec/projects/pi_lab/People\ and\ projects/Sarah/Sarah\ confocal/

[root@cifsgw Sarah confocal]# cat /root/.parsyncfp/fpcache/fpart.log.14.39.20_2020-10-27 Examining filesystem... /pri/projects/pi_lab/People: No such file or directory and: No such file or directory projects/Sarah/Sarah: No such file or directory confocal/AAV: No such file or directory LPS: No such file or directory IL4: No such file or directory 0 file(s) found.

hjmangalam commented 3 years ago

Since you're specifying it from the command line, you'll have to quote the startdir. Harry

[after AI & autocorrect, I take no responsibility for this content]

On Tue, Oct 27, 2020, 11:47 AM jasonkotecki notifications@github.com wrote:

I am trying to copy a folder that has spaces in the name as well as the startdir path to the source folder. Maybe I just have the syntax wrong, but I have tried several different ways with no luck. Below is the detail of my paths and the fpart log.

[root@cifsgw Sarah confocal]# pwd /pri/projects/pi_lab/People and projects/Sarah/Sarah confocal

[root@cifsgw Sarah confocal]# parsyncfp --NP=4 -startdir='/pri/projects/pi_lab/People and projects/Sarah/Sarah confocal' 'AAV LPS IL4' /sec/projects/pi_lab/People\ and\ projects/Sarah/Sarah\ confocal/

[root@cifsgw Sarah confocal]# cat /root/.parsyncfp/fpcache/fpart.log.14.39.20_2020-10-27 Examining filesystem... /pri/projects/pi_lab/People: No such file or directory and: No such file or directory projects/Sarah/Sarah: No such file or directory confocal/AAV: No such file or directory LPS: No such file or directory IL4: No such file or directory 0 file(s) found.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hjmangalam/parsyncfp/issues/38, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y54NRIJ66PQQJQUIBDSM4IVXANCNFSM4TBHWEKA .

jasonkotecki commented 3 years ago

I did use a single quote. I just tried again with double quotes with the same result.

parsyncfp --NP=4 --startdir="/pri/projects/pi_lab/People and projects/Sarah/Sarah confocal/" "AAV LPS IL4" "/sec/projects/pi_lab/People and projects/Sarah/Sarah confocal/"

[root@cifsgw ~]# cat /root/.parsyncfp/fpcache/fpart.log.15.33.01_2020-10-27 Examining filesystem... /pri/projects/pi_lab/People: No such file or directory and: No such file or directory projects/Sarah/Sarah: No such file or directory confocal//AAV: No such file or directory LPS: No such file or directory IL4: No such file or directory 0 file(s) found.

hjmangalam commented 3 years ago

Sorry, I'm an idiot - you gave the example correctly. Slightly distracted by the Irvine fire. I'll check the internals of this asap. Harry

On Tue, Oct 27, 2020 at 12:37 PM jasonkotecki notifications@github.com wrote:

I did use a single quote. I just tried again with double quotes with the same result.

parsyncfp --NP=4 --startdir="/pri/projects/pi_lab/People and projects/Sarah/Sarah confocal/" "AAV LPS IL4" "/sec/projects/pi_lab/People and projects/Sarah/Sarah confocal/"

[root@cifsgw ~]# cat /root/.parsyncfp/fpcache/fpart.log.15.33.01_2020-10-27 Examining filesystem... /pri/projects/pi_lab/People: No such file or directory and: No such file or directory projects/Sarah/Sarah: No such file or directory confocal//AAV: No such file or directory LPS: No such file or directory IL4: No such file or directory 0 file(s) found.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hjmangalam/parsyncfp/issues/38#issuecomment-717488082, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y22SATF633CJIXURCDSM4OOJANCNFSM4TBHWEKA .

--

Harry Mangalam

hjmangalam commented 3 years ago

OK - seemed to be a problem with feeding fpart the correctly quoted names. I think this is now fixed. It will now take a quoted or unquoted dirs/files. ie either:

pfp --NP=4 --rsyncopts='-azs' --chunk=100M 'ogg music files' or pfp --NP=4 --rsyncopts='-azs' --chunk=100M ogg\ music\ files/ (above was tab-completed from the commandline - ogg)

or mixed: pfp --NP=4 --rsyncopts='-azs' --chunk=100M \ --startdir=ogg\ music\ files/ 'KT Tunstall' \ 'The Derek Trucks Band'

Just pushed the patch to github. Thanks for the bug report. Much obliged. Harry

On Tue, Oct 27, 2020 at 12:45 PM Harry Mangalam hjmangalam@gmail.com wrote:

Sorry, I'm an idiot - you gave the example correctly. Slightly distracted by the Irvine fire. I'll check the internals of this asap. Harry

On Tue, Oct 27, 2020 at 12:37 PM jasonkotecki notifications@github.com wrote:

I did use a single quote. I just tried again with double quotes with the same result.

parsyncfp --NP=4 --startdir="/pri/projects/pi_lab/People and projects/Sarah/Sarah confocal/" "AAV LPS IL4" "/sec/projects/pi_lab/People and projects/Sarah/Sarah confocal/"

[root@cifsgw ~]# cat /root/.parsyncfp/fpcache/fpart.log.15.33.01_2020-10-27 Examining filesystem... /pri/projects/pi_lab/People: No such file or directory and: No such file or directory projects/Sarah/Sarah: No such file or directory confocal//AAV: No such file or directory LPS: No such file or directory IL4: No such file or directory 0 file(s) found.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hjmangalam/parsyncfp/issues/38#issuecomment-717488082, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y22SATF633CJIXURCDSM4OOJANCNFSM4TBHWEKA .

--

Harry Mangalam

--

Harry Mangalam

hjmangalam commented 3 years ago

Hi Jason, Thanks for your continued perseverance. I've changed the way pfp checks for external utilities and it'll continue with a WARN if you don't have the IB or WiFi utils. Pushed to github as version 1.71.

Harry

On Mon, Nov 2, 2020 at 1:37 AM jasonkotecki notifications@github.com wrote:

I just tried to use the newest release and I'm getting an error with iwconfig. My system does not have a wireless interface so iwconfig is not available. The previous version must have detected my interface correctly as I didn't get this error.

which: no iwconfig in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lpp/mmfs/bin/:/root/bin)

FATAL ERROR : [iwconfig] not found. you can get 'fpart' here: https://github.com/martymac/fpart 'scut' and 'stats' here (useful in themselves): https://github.com/hjmangalam/scut and the rest via yum, apt-get, or google. Please install [iwconfig] or correct your PATH variable to include it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hjmangalam/parsyncfp/issues/38#issuecomment-720355935, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y25B32T6AMHZDGULM3SNZ4WBANCNFSM4TBHWEKA .

--

Harry Mangalam

jasonkotecki commented 3 years ago

Harry, thank you for making that change. More importantly, thank you for making such a great script/tool for copying large datasets. Working for a life sciences research organization, I utilize it greatly.

-Jason

hjmangalam commented 3 years ago

Thanks for that note. The movement of bio data sets was one of the drivers for my starting the project. I was a biologist before I went over to the dark side. H

[after AI & autocorrect, I take no responsibility for this content]

On Mon, Nov 2, 2020, 10:56 AM jasonkotecki notifications@github.com wrote:

Harry, thank you for making that change. More importantly, thank you for making such a great script/tool for copying large datasets. Working for a life sciences research organization, I utilize it greatly.

-Jason

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hjmangalam/parsyncfp/issues/38#issuecomment-720661622, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y6MLOWWMMYCEAWK6PTSN36GPANCNFSM4TBHWEKA .