sahib / rmlint

Extremely fast tool to remove duplicates and other lint from your filesystem
http://rmlint.rtfd.org
GNU General Public License v3.0
1.91k stars 132 forks source link

Script rmlint.sh - non unix-like paths - processing possible e.g. for $RECYCLE.BIN ? (Dollar in path) #553

Closed james-cook closed 2 years ago

james-cook commented 2 years ago

The rmlint command itself seems to have no problem with such paths / files / folders. Running the rmlint.sh using set -x I'm getting:

+ original_cmd /mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN
+ print_progress_prefix
+ [ -n true ]
+ PROGRESS_PERC=0
+ [ 9272 -gt 0 ]
+ PROGRESS_PERC=0
+ printf %b[%3d%%]%b  \e[1;34m 0 \e[0m
+ [  0%] [ 0 -eq 1 ]
+ PROGRESS_CURR=28
+ printf \e[0;32mKeeping:  \e[0m%s\n /mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN
./rmlint.sh: 453: ./rmlint.sh: Syntax error: Unterminated quoted string
Keeping:  /mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN

at which point the script exits.

The line from the rmlint.sh file is:

 450
    451 original_cmd  '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN' # original directory
    452 cp_symlink    '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/XXXXX 11.07.2017/My Documents/desktop/$RECYCLE.BIN' '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN' # duplicate directory
    453

Even just an option to continue instead of exiting would be useful

james-cook commented 2 years ago

Just re-ran the above (rmlint.sh) without -p and it did not exit.

cebtenzzre commented 2 years ago

I see no reason why a dollar sign would cause a syntax error inside of single quotes. Seems more like an improperly escaped apostrophe (the .sh formatter does attempt to escape them). Are you sure those are the right lines and there isn't an issue above them? You need to provide an example that fails with sh -n example.sh; this example checks fine:

original_cmd  '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN' # original directory
cp_symlink    '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/XXXXX 11.07.2017/My Documents/desktop/$RECYCLE.BIN' '/mnt/seagate/BernieX [77GB]/from benji seagate drive jan 2017/Documents/desktop/$RECYCLE.BIN' # duplicate directory
james-cook commented 2 years ago

Yes, I did find it strange. I have never had problems with special chars before over several years of rmlint use.

So, thinking about this problem, I think it is important to note that this error arose in the same run as https://github.com/sahib/rmlint/issues/552 i.e. I called the script with "-px" and for directories the rmlint command was being called, overwriting the rmlint.sh file itself... It shouldn't really be possible but is there any chance that the running rmlint.sh script would be affected by being overwritten on the filesystem while it itself is running? Or is the environment affected in some other way...

I only use "-p" (paranoid mode) because having run rmlint in the past I cannot truly trust my existing rmlint xattrs gathered over many years. I generally run rmlint with --xattr for speed and then run the script with "-p" for safety (this is also the reason I made the request in https://github.com/sahib/rmlint/issues/542)

My workaround in this case was to re-run the rmlint command without --xattr (very slow) and then run the generated script without "-p". For good measure I now also run with "--duplicatedirs" - so that the rmlint cannot be called inside the script.

cebtenzzre commented 2 years ago

According to this StackExchange post, overwriting a shell script while it's running can definitely confuse bash. So the best explanation as to why this syntax error would occur is that the file was truncated or modified while bash was executing it, and bash could not find the end of the quoted string it was parsing. This should be closed as a duplicate of #552.

james-cook commented 2 years ago

That is very interesting to know - I had always assumed that the version of the file was protected from outside influence. Happy to close this.