search

azet / community_bash_style_guide

Community Bash Style Guide: writing useful and modern bash scripts, seriously.
Creative Commons Attribution 4.0 International
298 stars 37 forks source link

Parallelism: xargs #10

Closed ax3l closed 7 years ago

ax3l commented 9 years ago

Hi and thanks for the great writeup :sparkles:

In the section "Basic parallelism" I missed xargs a little. It's pretty straight forward and I use it quite a lot for basic parallelism. Is this intentionally or are PRs adding one or two sentences about it welcome? :)

example: repack all hdf5 files in a directory with 64 processes in parallel

find ${dirName} -name "*.h5" | xargs -n1 -P64 -I{} \
        sh -c 'echo "compress $1 ..." &&   \
               h5repack -i $1 -o $1.gz -f GZIP=1 && mv $1.gz $1' _ {}
azet commented 9 years ago

You're right. I'm happy taking PRs for that.

azet commented 7 years ago

@ax3l ping?