Falcon not using all cluster power

[dcopetti]

hello, I have been trying to run Falcon on our cluster (SGE, 4x12 cpus, 128 Gb of RAM) and the process is runnign, but is not using all the resources, as well as it is not writing anything in the 0- folder (not even the m_ fodlers). It is running since two days now, with phases when it was using 8-10 CPUs on one node, but none on the others. I would like to have it using all the cluster for the most computing-intensive part, at least. I am trying to assemble a ~400 Mb genome from >60x raw data, selfed plant. Below here cfg file and the screenshot of the usage in the last day. Thanks,

Dario

[General]
#manual at https://github.com/PacificBiosciences/FALCON/wiki/Manual
#job_type = local
job_type = SGE

# list of files of the initial subread fasta files
input_fofn = input.fofn

input_type = raw
#input_type = preads

# The length cutoff used for seed reads used for initial mapping
length_cutoff = 17000

# The length cutoff used for seed reads usef for pre-assembly
length_cutoff_pr = 15000

# Cluster queue setting
sge_option_da = -pe smp 12 -q all.q
sge_option_la = -pe smp 2 -q all.q
sge_option_pda = -pe smp 8 -q all.q
sge_option_pla = -pe smp 2 -q all.q
sge_option_fc = -pe smp 8 -q all.q
sge_option_cns = -pe smp 4 -q all.q

# concurrency setting
pa_concurrent_jobs = 48
cns_concurrent_jobs = 48
ovlp_concurrent_jobs = 48

# overlapping options for Daligner
pa_HPCdaligner_option =  -v -dal128 -t16 -e.70 -l1000 -s1000
ovlp_HPCdaligner_option = -v -dal128 -t32 -h60 -e.96 -l500 -s1000

pa_DBsplit_option = -x500 -s400
ovlp_DBsplit_option = -x500 -s400

# error correction consensus option
falcon_sense_option = --output_multi --min_idt 0.70 --min_cov 4 --local_match_count_threshold 2 --max_n_read 200 --n_core 6

# overlap filtering options
overlap_filtering_setting = --max_diff 60 --max_cov 90 --min_cov 5 --bestn 10
#cov values for a 30x e.c. reads ?

[dcopetti]

Hello Falcon developers, Do you have a solution for this issue? The process is nor running since 11 days and nothing has been written, not even the m_ folders. How do we fix this? Thanks,

Dario

[pb-cdunn]

You might have to talk to a local sysadmin. Every local cluster has its own idiosyncrasies.

But start with smaller genomes and work your way up. If you can find E.coli reads, that's a good one to start with. You can experiment with 'synth0', a 5kbp synthetic simulated example found in FALCON-examples. To run that, follow the directions in the FALCON-integrate wiki. You can run it locally first, and then set job_type=sge and try your grid.