piping something uses the number of given threads in parallel
program1 -t #CPU | program2 -t #CPU
So e.g. our bwa jobs sometimes us 96 processes rather than the requested 32... And I don't think this is limited to mkfifo.
So we need to split the available CPUs over all elements in the pipe.
Gurado
commented
10 years ago
piping something uses the number of given threads in parallel
program1 -t #CPU | program2 -t #CPUSo e.g. our bwa jobs sometimes us 96 processes rather than the requested 32... And I don't think this is limited to mkfifo. So we need to split the available CPUs over all elements in the pipe.