search

jpuritz / dDocent

a bash pipeline for RAD sequencing
ddocent.com
MIT License
52 stars 41 forks source link

Change ulimit for large sample sizes #36

Closed rwhetten closed 6 years ago

rwhetten commented 6 years ago

Ubuntu 14.04 by default has a limit of 1024 open files, controlled by the value of the ulimit variable, and the CreateIntervals function will fail at the samtools merge step if there are more than 1024 bam files to be merged. Adding a few lines of code can avoid this problem by auto-adjusting the ulimit variable for the current terminal session. Line 133-135 of the dDocent script evaluate the number of individuals and create the NumInd variable; one way to deal with the limit on open files is to test for $NumInd > 1000 and reset the ulimit value if necessary:

Increase limit on number of open files if $NumInd > 1000 (default limit is 1024 on Ubuntu)

if [ "$NumInd" -gt 1000 ]; then NewLimit=$(($NumInd + 100)) ulimit -n $NewLimit fi

jpuritz commented 6 years ago

Newest version has a new check to update the file limit.