search

PacificBiosciences / FALCON_unzip

Making diploid assembly becomes common practice for genomic study
BSD 3-Clause Clear License
30 stars 18 forks source link

falcon_unzip failed at 3-unzip/2-htigs/chunk_000***F #130

Closed zjzace closed 6 years ago

zjzace commented 6 years ago

Hello everyone!

I assembled a genome with 890Mb genome size using falcon and falcon_unzip. However, falcon_unzip always fails at 7 chunk_000***F in 3-unzip/2-htigs. The log file suggest it failed at "generate_haplotigs_for_ctg".

Here is my falcon_unzip configuration:

[job.defaults]
njobs = 60
NPROC = 1
MB = 46080

[General]
job_type = local

pwatcher_type = blocking
job_type = string
job_queue = bash -C ${CMD} > ${STDOUT_FILE} 2> ${STDERR_FILE}

[Unzip]

input_fofn= input.fofn
input_bam_fofn= input_bam.fofn
unzip_concurrent_jobs = 30
quiver_concurrent_jobs = 30
unzip_concurrent_jobs = 30
quiver_concurrent_jobs = 30

And following is the stderr:

**Failure in generate_haplotigs_for_ctg**((u'001762F', u'/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/0-phasing/001762F/uow-00/proto', './uow-001762F', u'../../..'))
Traceback (most recent call last):
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 76, in run_generate_haplotigs_for_ctg
    return generate_haplotigs_for_ctg(ctg_id, out_dir, unzip_dir, proto_dir, logger)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 221, in generate_haplotigs_for_ctg
    fragmented_snp_haplotigs = fragment_haplotigs(filtered_snp_haplotigs, aln_dict, clippoints, bubble_tree, fp_proto_log)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 483, in fragment_haplotigs
    new_haplotigs = fragment_single_haplotig(haplotig, aln, clippoints, bubble_tree, fp_proto_log)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 535, in fragment_single_haplotig
    tp = tiling_path.convert_split_lines_to_tiling_path(haplotig.path, len(haplotig.seq))
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/proto/tiling_path.py", line 165, in convert_split_lines_to_tiling_path
    return TilingPath(edges, contig_sequence_len)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/proto/tiling_path.py", line 38, in __init__
    assert(contig_sequence_len >= tiling_len)
AssertionError
Traceback (most recent call last):
  File "/users/s1155101875/software/Python-2.7.9/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/users/s1155101875/software/Python-2.7.9/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 1454, in <module>
    main()
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 1450, in main
    args.func(args)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 1217, in cmd_apply
    result = run_generate_haplotigs_for_ctg(exe)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 76, in run_generate_haplotigs_for_ctg
    return generate_haplotigs_for_ctg(ctg_id, out_dir, unzip_dir, proto_dir, logger)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 221, in generate_haplotigs_for_ctg
    fragmented_snp_haplotigs = fragment_haplotigs(filtered_snp_haplotigs, aln_dict, clippoints, bubble_tree, fp_proto_log)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 483, in fragment_haplotigs
    new_haplotigs = fragment_single_haplotig(haplotig, aln, clippoints, bubble_tree, fp_proto_log)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 535, in fragment_single_haplotig
    tp = tiling_path.convert_split_lines_to_tiling_path(haplotig.path, len(haplotig.seq))
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/proto/tiling_path.py", line 165, in convert_split_lines_to_tiling_path
    return TilingPath(edges, contig_sequence_len)
  File "/lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/falcon_unzip/proto/tiling_path.py", line 38, in __init__
    assert(contig_sequence_len >= tiling_len)
AssertionError
2018-07-14 15:42:18,380 - root - WARNING - Call '/bin/bash user_script.sh' returned 256.
2018-07-14 15:42:18,381 - root - WARNING - CD: '/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/2-htigs/chunk_001762F' -> '/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/2-htigs/chunk_001762F'
2018-07-14 15:42:18,381 - root - WARNING - CD: '/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/2-htigs/chunk_001762F' -> '/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/2-htigs/chunk_001762F'
2018-07-14 15:42:18,381 - root - CRITICAL - Error in /lustre/users/s1155101875/software/Falcon/fc_env_180712/lib/python2.7/site-packages/pypeflow/do_task.py with args="{'json_fn': '/lustre/scratch/s1155101875/Falcon_Assembly/3-unzip/2-htigs/chunk_001762F/task.json',\n 'timeout': 30,\n 'tmpdir': None}"
Traceback (most recent call last):

Can anyone help me out? Mnay thanks.

pgonzale60 commented 6 years ago

Hi,

I think at least @whc2 and me have the same issue. I hope that my config and error files can ease to identify the faulting parameter or code.

Briefly, my error was "Failure in generate_haplotigs_for_ctg((u'000..."

[INFO 2018-07-13 15:14:13] Loading the alignments.
[INFO 2018-07-13 15:14:13] Reorienting haplotigs.
[INFO 2018-07-13 15:14:13]   - qname = 000559F-HAP8361-000559F.1000001.0
[INFO 2018-07-13 15:14:13] 
[INFO 2018-07-13 15:14:13]  start = (1, 426), end = (4195, 4618)
[INFO 2018-07-13 15:14:13]  start = (4195, 4618), end = (24737, 25049)
[INFO 2018-07-13 15:14:13] pos_of_interest for q_name: 000559F-HAP8361-000559F.1000001.0
[INFO 2018-07-13 15:14:13] ((1, 426), (4195, 4618), '000559F-HAP8361-000559F.1000001.0', 26729, '000559F', 44892, ('000559F', 1000001, 0))
[INFO 2018-07-13 15:14:13] ((4195, 4618), (24737, 25049), '000559F-HAP8361-000559F.1000001.0', 26729, '000559F', 44892, ('000559F', 1000001, 0))
[INFO 2018-07-13 15:14:13] 
[ERROR 2018-07-13 15:14:13] Failure in generate_haplotigs_for_ctg((u'000559F', u'/dfs1/bio/pmgonza1/mnark/output/falcon/run2/3-unzip/0-phasing/000559F/uow-00/proto', './uow-000559F', u'../../..'))
Traceback (most recent call last):
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 76, in run_generate_haplotigs_for_ctg
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 221, in generate_haplotigs_for_ctg
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 483, in fragment_haplotigs
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/mains/graphs_to_h_tigs_2.py", line 535, in fragment_single_haplotig
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/proto/tiling_path.py", line 165, in convert_split_lines_to_tiling_path
  File "/tmp/pip-install-aGjt1F/falcon-unzip/falcon_unzip/proto/tiling_path.py", line 38, in __init__
AssertionError

Like @whc2, I have tried to solve this as suggested in the troubleshooting guide but without success. Jobs of other chunks have finished succesfully, although this was more narrowly defined by @whc2.

Best regards.

all.log.txt chunk_000559F.pwatcher.dir.stderr.txt chunk_000559F.pwatcher.dir.stdout.txt sge_fc_unzip.cfg.txt

zjzace commented 6 years ago

Many thanks for your response @pgonzale60. I have tried to use the previous version to redo the assembly. Luckily, the assembly was finished with no error reported.

pb-cdunn commented 6 years ago

This is fixed in the latest binary release.