search

bcgsc / tigmint

⛓ Correct misassemblies using linked AND long reads
https://bcgsc.github.io/tigmint/
GNU General Public License v3.0
55 stars 13 forks source link

tigmint molecule Error 1 #30

Closed darencard closed 4 years ago

darencard commented 5 years ago

I'm trying to run Tigmint with Arks/Links using the arks-make pipeline. My run gets through the mapping but hits a snag in the tigmint molecule step. The error message doesn't give much to go on so I'm hoping someone can provide some help. Here is the error from STDERR:

make[1]: *** [assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed] Error 1
make[1]: *** Deleting file `assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed'
make: *** [assembly.tigmint.fa] Error 2

The command it errors on is:

command time -v -o assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed.time /n/home13/dcard/.linuxbrew/Cellar/tigmint/1.1.2_2/libexec/bin/tigmint-molecule -a0.65 -n5 -q0 -d50000 -s2000 assembly.chromium_reads.sortbx.bam | sort -k1,1 -k2,2n -k3,3n >assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed

Any information and guidance is greatly appreciated. Also, is there a way to restart the run without having to map everything again? Or is that a feature of make (I'm new to using it like this)?

Thank you, Daren Card

lcoombe commented 5 years ago

Hi @darencard,

Could you provide your full log (full stdout, stderr), at least from after the tigmint-molecule step? As you say there was an error there, but I can't tell what it was just based on the make errors. I'd assume there was either a python error or perhaps a sort error.

Yes, if you re-launch the pipeline once you've solved the error, it will re-start after the alignment stage. You should see a (probably large) BAM file in your working directory, and it will be able to recognize the presence of this file and start at tigmint-molecule. If you want to double check that is working properly, you can always add -n to your tigmint-make command -- this is a 'dry-run', so it will run all the commands without executing them.

Lauren

darencard commented 5 years ago

Hi Lauren,

Thanks for the response. Here is the stdout:

command time -v -o assembly.tigmint.fa.time tigmint tigmint draft=assembly reads=chromium_reads minsize=2000 as=0.65 nm=5 dist=50000 mapq=0 trim=0 span=20 window=1000 t=48
make[1]: Entering directory `/n/holylfs/LABS/edwards_lab/dcard/Lerista_10xChromium/Supernova_assemblies/Ler83_Ledwardsae_Supernova_default_2019-04-26/all_reads/tigmint-arks-links'
bwa index assembly.fa
[bwt_gen] Finished constructing BWT in 416 iterations.
command time -v -o assembly.chromium_reads.sortbx.bam.time bwa mem -t48 -pC assembly.fa chromium_reads.fq.gz | samtools view -u -F4 | samtools sort -@48 -tBX -T$(mktemp -u -t assembly.chromium_reads.sortbx.bam.XXXXXX) -o assembly.chromium_reads.sortbx.bam
command time -v -o assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed.time /n/home13/dcard/.linuxbrew/Cellar/tigmint/1.1.2_2/libexec/bin/tigmint-molecule -a0.65 -n5 -q0 -d50000 -s2000 assembly.chromium_reads.sortbx.bam | sort -k1,1 -k2,2n -k3,3n >assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed
make[1]: Leaving directory `/n/holylfs/LABS/edwards_lab/dcard/Lerista_10xChromium/Supernova_assemblies/Ler83_Ledwardsae_Supernova_default_2019-04-26/all_reads/tigmint-arks-links'

And here is the stderr. I collapsed down the BWA lines, which are all pretty-much the same.

/n/home13/dcard/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 197: 51342 Illegal instruction     (core dumped) "$HOMEBREW_PREFIX/opt/curl/bin/curl" --version > /dev/null
/n/home13/dcard/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 209: 51346 Illegal instruction     (core dumped) "$HOMEBREW_PREFIX/opt/git/bin/git" --version > /dev/null
sh: line 1: 51438 Illegal instruction     (core dumped) tput cols 2> /dev/null
sh: line 1: 51472 Illegal instruction     (core dumped) tput cols 2> /dev/null
[bwa_index] Pack FASTA... 35.52 sec
[bwa_index] Construct BWT for the packed sequence...
[BWTIncCreate] textLength=3481766454, availableWord=256989896
[BWTIncConstructFromPacked] 10 iterations done. 99999990 characters processed.
[BWTIncConstructFromPacked] 20 iterations done. 199999990 characters processed.
[BWTIncConstructFromPacked] 30 iterations done. 299999990 characters processed.
[BWTIncConstructFromPacked] 40 iterations done. 399999990 characters processed.
[BWTIncConstructFromPacked] 50 iterations done. 499999990 characters processed.
[BWTIncConstructFromPacked] 60 iterations done. 599999990 characters processed.
[BWTIncConstructFromPacked] 70 iterations done. 699999990 characters processed.
[BWTIncConstructFromPacked] 80 iterations done. 799999990 characters processed.
[BWTIncConstructFromPacked] 90 iterations done. 899999990 characters processed.
[BWTIncConstructFromPacked] 100 iterations done. 999999990 characters processed.
[BWTIncConstructFromPacked] 110 iterations done. 1099999990 characters processed.
[BWTIncConstructFromPacked] 120 iterations done. 1199999990 characters processed.
[BWTIncConstructFromPacked] 130 iterations done. 1299999990 characters processed.
[BWTIncConstructFromPacked] 140 iterations done. 1399999990 characters processed.
[BWTIncConstructFromPacked] 150 iterations done. 1499999990 characters processed.
[BWTIncConstructFromPacked] 160 iterations done. 1599999990 characters processed.
[BWTIncConstructFromPacked] 170 iterations done. 1699999990 characters processed.
[BWTIncConstructFromPacked] 180 iterations done. 1799999990 characters processed.
[BWTIncConstructFromPacked] 190 iterations done. 1899999990 characters processed.
[BWTIncConstructFromPacked] 200 iterations done. 1999999990 characters processed.
[BWTIncConstructFromPacked] 210 iterations done. 2099999990 characters processed.
[BWTIncConstructFromPacked] 220 iterations done. 2199999990 characters processed.
[BWTIncConstructFromPacked] 230 iterations done. 2299999990 characters processed.
[BWTIncConstructFromPacked] 240 iterations done. 2399999990 characters processed.
[BWTIncConstructFromPacked] 250 iterations done. 2499999990 characters processed.
[BWTIncConstructFromPacked] 260 iterations done. 2599999990 characters processed.
[BWTIncConstructFromPacked] 270 iterations done. 2699999990 characters processed.
[BWTIncConstructFromPacked] 280 iterations done. 2799999990 characters processed.
[BWTIncConstructFromPacked] 290 iterations done. 2894948294 characters processed.
[BWTIncConstructFromPacked] 300 iterations done. 2979341094 characters processed.
[BWTIncConstructFromPacked] 310 iterations done. 3054345622 characters processed.
[BWTIncConstructFromPacked] 320 iterations done. 3121005862 characters processed.
[BWTIncConstructFromPacked] 330 iterations done. 3180249622 characters processed.
[BWTIncConstructFromPacked] 340 iterations done. 3232901622 characters processed.
[BWTIncConstructFromPacked] 350 iterations done. 3279694822 characters processed.
[BWTIncConstructFromPacked] 360 iterations done. 3321280742 characters processed.
[BWTIncConstructFromPacked] 370 iterations done. 3358238374 characters processed.
[BWTIncConstructFromPacked] 380 iterations done. 3391082358 characters processed.
[BWTIncConstructFromPacked] 390 iterations done. 3420270166 characters processed.
[BWTIncConstructFromPacked] 400 iterations done. 3446208310 characters processed.
[BWTIncConstructFromPacked] 410 iterations done. 3469258182 characters processed.
[bwa_index] 2061.61 seconds elapse.
[bwa_index] Update BWT... 23.86 sec
[bwa_index] Pack forward-only FASTA... 23.31 sec
[bwa_index] Construct SA from BWT and Occ... 512.27 sec
[main] Version: 0.7.15-r1140
[main] CMD: bwa index assembly.fa
[main] Real time: 2670.040 sec; CPU: 2656.579 sec
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 3465704 sequences (480000004 bp)...
[M::process] 0 single-end sequences; 3465704 paired-end sequences
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (31896, 1111800, 18503, 23301)
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (166, 252, 358)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 742)
[M::mem_pestat] mean and std.dev: (265.94, 140.31)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 934)
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (322, 401, 514)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 898)
[M::mem_pestat] mean and std.dev: (420.40, 155.82)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 1090)
[M::mem_pestat] analyzing insert size distribution for orientation RF...
[M::mem_pestat] (25, 50, 75) percentile: (46, 109, 224)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 580)
[M::mem_pestat] mean and std.dev: (136.26, 121.65)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 758)
[M::mem_pestat] analyzing insert size distribution for orientation RR...
[M::mem_pestat] (25, 50, 75) percentile: (175, 257, 367)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 751)
[M::mem_pestat] mean and std.dev: (272.66, 141.49)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 943)
[M::mem_pestat] skip orientation FF
[M::mem_pestat] skip orientation RF
[M::mem_pestat] skip orientation RR
[M::mem_process_seqs] Processed 3465704 reads in 5209.140 CPU sec, 1616.809 real sec
...
[M::process] 0 single-end sequences; 173256 paired-end sequences
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (1626, 54624, 871, 1161)
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (160, 254, 347)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 721)
[M::mem_pestat] mean and std.dev: (258.90, 138.10)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 908)
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (322, 402, 517)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 907)
[M::mem_pestat] mean and std.dev: (421.90, 157.45)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 1102)
[M::mem_pestat] analyzing insert size distribution for orientation RF...
[M::mem_pestat] (25, 50, 75) percentile: (43, 97, 212)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 550)
[M::mem_pestat] mean and std.dev: (132.81, 122.65)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 719)
[M::mem_pestat] analyzing insert size distribution for orientation RR...
[M::mem_pestat] (25, 50, 75) percentile: (180, 256, 372)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 756)
[M::mem_pestat] mean and std.dev: (273.70, 142.81)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 948)
[M::mem_pestat] skip orientation FF
[M::mem_pestat] skip orientation RF
[M::mem_pestat] skip orientation RR
[M::mem_process_seqs] Processed 173256 reads in 213.352 CPU sec, 54.777 real sec
[main] Version: 0.7.15-r1140
[main] CMD: bwa mem -t48 -pC assembly.fa chromium_reads.fq.gz
[main] Real time: 330045.985 sec; CPU: 1273612.571 sec
[bam_sort_core] merging from 816 files...
make[1]: *** [assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed] Error 1
make[1]: *** Deleting file `assembly.chromium_reads.as0.65.nm5.molecule.size2000.bed'
make: *** [assembly.tigmint.fa] Error 2

Good to know the restarting is easy. I have been meaning to learn more about make and that provides even more justification!

Cheers, Daren

lcoombe commented 5 years ago

Hi Daren,

Thank you for the full logs! The error is unfortunately still a bit cryptic to me -- I don't see any error messages from the tigmint-molecule step other than the make errors.

To troubleshoot, I would try running the tigmint-molecule command separately, and see what happens. Just in case that provides more info about the error, perhaps in the output bed file that the makefile is automatically deleting. Hopefully that will give us a bit more information to go on.

Yes, I'm a big make fan! I've started making all of my pipelines with it -- really nice for parameter sweeping as well! (For example if you wanted to sweep on span, you could re-run the makefile and it will start at the tigmint-cut stage without re-doing the alignments, etc).

Lauren

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your interest in Tigmint!