On cluster's with slow shared storage performance can be improved by making use of local storage on a node.
To use local storage only a single node must be used.
Changes here add a setup-config.sh script to setup config.sh and smprofile/config.yaml to support running in this manner.
The local storage will be used to hold snakemake, conda environments, input data, and intermediate data.
Only output files and the top level logs will be stored on the shared storage.
When running in this mode the data directory -d should point to the local storage directory.
A new -o flag must be used to direct the output files to a shared directory.
An additional optimization used by this mode is directly running sbatch and a script.
Reduces the threads for remove_adapters to maximize performance when running on a single node.
On cluster's with slow shared storage performance can be improved by making use of local storage on a node. To use local storage only a single node must be used.
Changes here add a setup-config.sh script to setup config.sh and smprofile/config.yaml to support running in this manner. The local storage will be used to hold snakemake, conda environments, input data, and intermediate data. Only output files and the top level logs will be stored on the shared storage. When running in this mode the data directory
-dshould point to the local storage directory. A new-oflag must be used to direct the output files to a shared directory.An additional optimization used by this mode is directly running sbatch and a script.
Reduces the threads for remove_adapters to maximize performance when running on a single node.