search

Unidata / LDM

The Unidata Local Data Manager (LDM) system includes network client and server programs designed for event-driven data distribution, and is the fundamental component of the Unidata Internet Data Distribution (IDD) system.
http://www.unidata.ucar.edu/software/ldm
Other
43 stars 27 forks source link

make scour find+rm more robust to filenames #26

Closed akrherz closed 7 years ago

akrherz commented 9 years ago

Filenames with quotes or other characters may cause xargs to fail like so $ ldmadmin scour xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

Use POSIX -exec rm -f {} + which should be more forgiving.

semmerson commented 9 years ago

@akrherz The scour(1) utility used to use the find(1) arguments -exec rm -f {};. That was replaced by the use of xargs(1) to increase performance. Do you have a lot of files that get scoured and, if so, is performance of scour(1) an issue?

akrherz commented 9 years ago

@semmerson thanks for the response. I always assumed slow scour was a result of poor filesystem performance and not from a slow command line option. Execing rm once for a thousand files vs execing rm a thousand times will certainly be faster. The issue at hand is having a file with a filename that causes the current command to fail. This bit us in development with a corrupt BUFR file caused a filename to get written with an apostrophe in the filename.

semmerson commented 9 years ago

@akrherz I can sympathize.

I think a better solution would be rewrite scour(1) as a C program. I'll add an issue to that effect.

semmerson commented 7 years ago

Added escaping of all characters except newlines. That should do it.

akrherz commented 7 years ago

Thanks @semmerson , but where is the commit?

semmerson commented 7 years ago

The escaping of all pathname characters goes back to April, 2010. The latest version definitely has it.