Issues running fc_run on UGE system

[RenzoTale88]

Good morning, I'm trying to run FALCON (falcon-kit 1.8.1) on a set of CCS reads, followed by unzipping and phasing with HiC data. I'm trying to run it on a Univa GridEngine, for which I created the following configuration file:

[General]
input_type = preads
input_fofn = /PATH/TO/ASM/FALCON/fc_run_HiFi.fofn
pwatcher_type=blocking

# not relevant for HiFi but include anyway
# The length cutoff used for seed reads used for initial mapping
genome_size = 2700000000
seed_coverage = 20
length_cutoff = -1

# not relevant for HiFi but include anyway
# overlapping options for Daligner
pa_daligner_option = -e0.8 -l1000 -k18 -h70 -w8 -s100
pa_HPCdaligner_option = -v -B128 -M24
pa_HPCTANmask_option = -k18 -h480 -w8 -e.8 -s100
pa_HPCREPmask_option = -k18 -h480 -w8 -e.8 -s100
#pa_REPmask_code=1,20;10,15;50,10
pa_DBsplit_option = -x500 -s400
# error correction consensus option
falcon_sense_option = --output-multi --min-idt 0.70 --min-cov 3 --max-n-read 100 --n-core 4

# Parameters relevant to HiFi/CCS assembly.
length_cutoff_pr = 10000
ovlp_daligner_option = -k24 -h1024 -e.98 -l1000 -s100
ovlp_HPCdaligner_option = -v -B128 -M24
ovlp_DBsplit_option = -s200

# experimenent with "--min-idt" to collapse (98-99) or split haplotypes (up to 99.9) during contig assembly
# if you plan to unzip, collapse first using ~98, lower for very divergent haplotypes
# ignore indels looks at only substitutions in overlaps, allows higher overlap stringency to reduce repeat-induced errors
overlap_filtering_setting = --max-diff 400 --max-cov 400 --min-cov 2 --n-core 24 --min-idt 98 --ignore-indels

[job.defaults]
use_tmpdir=/PATH/TO/ASM/TMPDIR
job_type=sge
pwatcher_type=blocking
JOB_QUEUE=default
MB=64768
NPROC=2
njobs=4
submit = qsub -S /bin/bash -sync y -V \
  -N ${JOB_NAME}      \
  -o "${JOB_STDOUT}"  \
  -e "${JOB_STDERR}"  \
  -pe sharedmem ${NPROC}    \
  -l h_vmem=${MB}M    \
  "${JOB_SCRIPT}"

[job.step.da]
NPROC=4
MB=32000
njobs=100
[job.step.la]
NPROC=8
MB=16000
njobs=100
[job.step.cns]
NPROC=8
MB=16000
njobs=100
[job.step.pda]
NPROC=8
MB=16000
njobs=100
[job.step.pla]
NPROC=4
MB=16000
njobs=100
[job.step.asm]
NPROC=24
MB=16000
njobs=1

The software seems to generate the database successfully, submitting the generation phase to the queue and returning the results just fine. However, the subsequent Daligner stage fails due to lack of space:

falcon-kit 1.8.1 (pip thinks "falcon-kit 1.8.1")
pypeflow 2.3.0
INFO:root:For multiprocessing, parallel njobs=8 (cpu_count=40, nproc=8, nproc_per_uow=1)
Traceback (most recent call last):
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/falcon_kit/mains/generic_run_units_of_work.py", line 152, in <module>
    main()
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/falcon_kit/mains/generic_run_units_of_work.py", line 148, in main
    run(**vars(args))
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/falcon_kit/mains/generic_run_units_of_work.py", line 100, in run
    exe_pool = Pool(njobs, initializer=Start)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/falcon_kit/multiproc.py", line 31, in Pool
    return multiprocessing.Pool(processes, *args, **kwds)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/pool.py", line 191, in __init__
    self._setup_queues()
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/pool.py", line 343, in _setup_queues
    self._inqueue = self._ctx.SimpleQueue()
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/queues.py", line 336, in __init__
    self._rlock = ctx.Lock()
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 28] No space left on device
2021-09-10 07:36:55,065 - root - WARNING - Call '/bin/bash user_script.sh' returned 256.
2021-09-10 07:36:55,065 - root - INFO - CD: '/PATH/TO/ASM/TMPDIR/atalenti/pypetmp//PATH/TO/ASM/1-preads_ovl/d2021-09-10 07:36:55,065 - root - INFO - CD: '/PATH/TO/ASM/1-preads_ovl/daligner-runs/j_0876' -> '/PATH/TO/ASM2021-09-10 07:36:55,065 - root - CRITICAL - Error in /PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py with args="{'json_fn': '/exports/cmvm/Traceback (most recent call last):
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py", line 280, in <module>
    main()
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py", line 272, in main
    run(**vars(parsed_args))
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py", line 266, in run
    run_cfg_in_tmpdir(cfg, tmpdir, '.')
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py", line 241, in run_cfg_in_tmpdir
    run_bash(bash_template, myinputs, myoutputs, parameters)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/do_task.py", line 200, in run_bash
    util.system(cmd)
  File "/PATH/TO/Andrea/myanaconda/pbasm/lib/python3.8/site-packages/pypeflow/io.py", line 27, in syscall
    raise Exception(msg)
Exception: Call '/bin/bash user_script.sh' returned 256.

This is odd, since I have over 2Tb of storage available for the workflow to access, and setting the temporary directory to a local one doesn't seem to fix the issue. I'm unsure whether this count as a bug or just a configuration error. Am I missing something?

Thanks in advance for the help, Andrea

[gconcepcion]

Hello,

Falcon is no longer in development, it has been replaced by IPA- Improved Phased Assembly which I recommend you try instead as you should receive better results: https://github.com/PacificBiosciences/pbipa

Alternatively the other pipeline that I tend to recommend is hifiasm which is also fast and accurate and very simple to run - significantly easier than our old FALCON workflow: https://github.com/chhylp123/hifiasm

Other good HiFi assembly pipelines also include FLYE: https://github.com/fenderglass/Flye and Peregrine: https://github.com/cschin/Peregrine

Hope this helps

EDIT: I should note that hifiasm can also use your Hi-C reads to completely phase your genome. I would definitely give it a try.