Memory limit - Githubissues

Hello,

I keep getting a memory limit error when running REFMAKER. I was wondering in your development of the program was there general memeory allocation needed? I have 77 low coverage whole genomes that range from .5X-25X. Files are 300MB-25GB. I have allocated 245GB of memory and it still runs out of memory. I was wondering if I should down sample the higher coverage samples before running REFMAKER?

I am also getting a "finished abnormally" error which seems to be associated with memory allocation.

== Error ==  system call for: "['/data/gpfs/home/jhallas/anaconda3/envs/refmaker-env/bin/spades-hammer', '/data/gpfs/assoc/denovo/jhallas/deer/refmaker/res_refmaker/assembly/spades/Mazama_gouazoupira_195094/c
orrected/configs/config.info']" finished abnormally, OS return value: -9
None

This is my parameter file

#REFMAKER (v.1.0) parameter file
#------ global parameters ------------------------------------------------------
TOOLS=/data/gpfs/assoc/denovo/jhallas/deer/programs/REFMAKER-main/tools.sh                                                ## [1] path to file with dependencies
RES=/data/gpfs/assoc/denovo/jhallas/deer/refmaker/res_refmaker                                                              ## [2] output directory
THREADS=16                                                                      ## [3] number of threads to use for multithreading steps
#------ demultiplexing ---------------------------------------------------------
DATA=~/path/to/libraries/                                                       ## [4] path to sequencing reads libraries including barcode(s) file(s): *.barcodes
ERROR_RATE=0.17                                                                 ## [5] maximum allowed error rate as value between 0 and 1
READ_QUALITY=10                                                                 ## [6] Trim low-quality bases. If one value is given, only the 3' end is trimmed. If two comma-separated cutoffs are given, the 5' end is trimmed with the first cutoff, the 3' end with the second
MIN_READ_LENGTH=50                                                              ## [7] Discard reads shorter than this length.
BARCODE_LENGTH=6                                                                ## [8] Remove LENGTH bases from second read in a pair
#------ assembly samples -------------------------------------------------------
MEMORY=245                                                                      ## [9] maximal memory used in assembly
KMER=55,75                                                                      ## [10] k-mer size used in contigs assembly of each library. Single (here 55) or range values (as 21,33,55) allowed. Note: less than 128
SAMPLES_FILE=/data/gpfs/assoc/denovo/jhallas/deer/refmaker/samples.txt                                                        ## [11] samples file. Specific format required:  (1) sample name with Genus_species_(subsp)_taxid_attributes; (2) path to forward reads; (3) path to reverse reads;

Here is my slurm submission file

#!/bin/bash

#SBATCH --account=cpu-s1-bionres-0
#SBATCH --partition=cpu-s1-bionres-0
#SBATCH --mail-type=FAIL
#SBATCH --mail-type=END
#SBATCH --mail-user=joshua.hallas@gmail.com
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=2450
#SBATCH --time=14-00:00:00
#SBATCH --error=/data/gpfs/assoc/denovo/jhallas/deer/scripts/out_refmaker/%x_%A.err
#SBATCH --output=/data/gpfs/assoc/denovo/jhallas/deer/scripts/out_refmaker/%x_%A.out

####### LOAD SOFTWARE #######
source activate refmaker-env

####### FILE PATHS #######
REFMAKER=/data/gpfs/assoc/denovo/jhallas/deer/programs/REFMAKER-main/refmaker
PARAMETER=/data/gpfs/assoc/denovo/jhallas/deer/refmaker/parameter.txt

####### MANAGMENT #######
echo $HOSTNAME
start_time=$(date +%s)

###### COMMANDS ######
echo "Begin assembly..."
$REFMAKER -m assembly -o sample -p $PARAMETER

Thank you for any advice.

-josh

cpouchon / REFMAKER

Memory limit #3