Open apcamargo opened 4 years ago
Version: commit 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7
commit 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7
Command:
./conterminator/build/bin/conterminator protein GTDB97.faa taxonomy/mapping_new conterminator_phylum_level conterminator_tmp --threads 64 --ncbi-tax-dump taxonomy --kingdoms 3,9,22,28,71,85,104,113,188,435,558,600,661,798,948,2112,2179,2185,2197,2222,2231,2290,2346,2367,2457,2526,2599,2662,2804,2844,2936,2947,3026,3093,3196,3236,3259,3295,3332,3478,3486,3519,3588,3721,3764,3806,3852,3869,3971,4049,4123,4138,4295,4380,4393,4426,4438,4480,4574,4637,4747,4807,4962,5058,5211,5295,5301,5341,5428,5455,5610,5785,5850,5876,6053,6077,6352,6994,7152,7497,7678,7997,8069,8163,8265,8415,8508,9033,9459,9518,9602,9620,9659,9816,9982,10092,10112,10373,10409,10624,10701,10941,10980,11478,12122,12426,12439,12474,12788,12916,13990,14290,14512,14627,14692,14712,14955,15712,16418,16488,17029,17211,17233,17428,17878,17918,18303,18551,19564,19757,19952,20114,20624,21463,22205,22539,22684,23843,24111,24497,26693,27254,29002,29170,30513,31392,32011,33650,34643,34865,35143
Output:
Tmp conterminator_tmp folder does not exist or is not a directory. Create dir conterminator_tmp protein GTDB97.faa taxonomy/mapping_new conterminator_phylum_level conterminator_tmp --threads 64 --ncbi-tax-dump taxonomy --kingdoms 3,9,22,28,71,85,104,113,188,435,558,600,661,798,948,2112,2179,2185,2197,2222,2231,2290,2346,2367,2457,2526,2599,2662,2804,2844,2936,2947,3026,3093,3196,3236,3259,3295,3332,3478,3486,3519,3588,3721,3764,3806,3852,3869,3971,4049,4123,4138,4295,4380,4393,4426,4438,4480,4574,4637,4747,4807,4962,5058,5211,5295,5301,5341,5428,5455,5610,5785,5850,5876,6053,6077,6352,6994,7152,7497,7678,7997,8069,8163,8265,8415,8508,9033,9459,9518,9602,9620,9659,9816,9982,10092,10112,10373,10409,10624,10701,10941,10980,11478,12122,12426,12439,12474,12788,12916,13990,14290,14512,14627,14692,14712,14955,15712,16418,16488,17029,17211,17233,17428,17878,17918,18303,18551,19564,19757,19952,20114,20624,21463,22205,22539,22684,23843,24111,24497,26693,27254,29002,29170,30513,31392,32011,33650,34643,34865,35143 MMseqs Version: 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7 Substitution matrix nucl:nucleotide.out,aa:blosum62.out Add backtrace false Alignment mode 3 Allow wrapped scoring false E-value threshold 0.001 Seq. id. threshold 0.95 Min. alignment length 0 Seq. id. mode 0 Alternative alignments 0 Coverage threshold 0.95 Coverage mode 0 Max sequence length 65535 Compositional bias 1 Realign hits false Max reject 2147483647 Max accept 2147483647 Include identical seq. id. false Preload mode 0 Pseudo count a 1 Pseudo count b 1.5 Score bias 0 Gap open cost 11 Gap extension cost 1 Threads 64 Compressed 0 Verbosity 3 Seed substitution matrix nucl:nucleotide.out,aa:VTML80.out Sensitivity 4 K-mer size 0 K-score 2147483647 Alphabet size 21 Split database 0 Split mode 2 Split memory limit 0 Diagonal scoring true Exact k-mer matching 0 Mask residues 0 Mask lower case residues 0 Minimum diagonal score 15 Spaced k-mers 1 Spaced k-mer pattern Local temporary path Rescore mode 0 Remove hits by seq. id. and coverage false Sort results 0 Mask profile 1 Profile e-value threshold 0.001 Use global sequence weighting false Allow deletions false Filter MSA 1 Maximum seq. id. threshold 0.9 Minimum seq. id. 0 Minimum score per column -20 Minimum coverage 0 Select N most diverse seqs 1000 Omit consensus false Min codons in orf 30 Max codons in length 32734 Max orf gaps 2147483647 Contig start mode 2 Contig end mode 2 Orf start mode 1 Forward frames 1,2,3 Reverse frames 1,2,3 Translation table 1 Translate orf 0 Use all table starts false Offset of numeric ids 0 Create lookup 0 Add orf stop false createdb GTDB97.faa conterminator_tmp/6937418719709452993/sequencedb Converting sequences [=================================================================================================== 1 Mio. sequences processed =================================================================================================== 2 Mio. sequences processed =================================================================================================== 3 Mio. sequences processed =================================================================================================== 4 Mio. sequences processed =================================================================================================== 5 Mio. sequences processed =================================================================================================== 6 Mio. sequences processed =================================================================================================== 7 Mio. sequences processed =================================================================================================== 8 Mio. sequences processed =================================================================================================== 9 Mio. sequences processed =================================================================================================== 10 Mio. sequences processed =================================================================================================== 11 Mio. sequences processed =================================================================================================== 12 Mio. sequences processed =================================================================================================== 13 Mio. sequences processed =================================================================================================== 14 Mio. sequences processed =================================================================================================== 15 Mio. sequences processed =================================================================================================== 16 Mio. sequences processed =================================================================================================== 17 Mio. sequences processed =================================================================================================== 18 Mio. sequences processed =================================================================================================== 19 Mio. sequences processed =================================================================================================== 20 Mio. sequences processed =================================================================================================== 21 Mio. sequences processed =================================================================================================== 22 Mio. sequences processed =================================================================================================== 23 Mio. sequences processed =================================================================================================== 24 Mio. sequences processed =================================================================================================== 25 Mio. sequences processed =================================================================================================== 26 Mio. sequences processed =================================================================================================== 27 Mio. sequences processed =================================================================================================== 28 Mio. sequences processed =================================================================================================== 29 Mio. sequences processed =================================================================================================== 30 Mio. sequences processed =================================================================================================== 31 Mio. sequences processed =================================================================================================== 32 Mio. sequences processed =================================================================================================== 33 Mio. sequences processed =================================================================================================== 34 Mio. sequences processed =================================================================================================== 35 Mio. sequences processed =================================================================================================== 36 Mio. sequences processed =================================================================================================== 37 Mio. sequences processed =================================================================================================== 38 Mio. sequences processed =================================================================================================== 39 Mio. sequences processed =================================================================================================== 40 Mio. sequences processed =================================================================================================== 41 Mio. sequences processed =================================================================================================== 42 Mio. sequences processed =================================================================================================== 43 Mio. sequences processed =================================================================================================== 44 Mio. sequences processed =================================================================================================== 45 Mio. sequences processed =================================================================================================== 46 Mio. sequences processed =================================================================================================== 47 Mio. sequences processed =================================================================================================== 48 Mio. sequences processed =================================================================================================== 49 Mio. sequences processed =================================================================================================== 50 Mio. sequences processed =================================================================================================== 51 Mio. sequences processed =================================================================================================== 52 Mio. sequences processed =================================================================================================== 53 Mio. sequences processed =================================================================================================== 54 Mio. sequences processed =================================================================================================== 55 Mio. sequences processed =================================================================================================== 56 Mio. sequences processed =================================================================================================== 57 Mio. sequences processed =================================================================================================== 58 Mio. sequences processed =================================================================================================== 59 Mio. sequences processed =================================================================================================== 60 Mio. sequences processed =================================================================================================== 61 Mio. sequences processed =================================================================================================== 62 Mio. sequences processed =================================================================================================== 63 Mio. sequences processed =================================================================================================== 64 Mio. sequences processed =================================================================================================== 65 Mio. sequences processed =================================================================================================== 66 Mio. sequences processed =================================================================================================== 67 Mio. sequences processed =================================================================================================== 68 Mio. sequences processed =================================================================================================== 69 Mio. sequences processed =================================================================================================== 70 Mio. sequences processed =================================================================================================== 71 Mio. sequences processed =================================================================================================== 72 Mio. sequences processed =================================================================================================== 73 Mio. sequences processed =========================================================================== Time for merging to sequencedb_h: 0h 0m 39s 357ms Time for merging to sequencedb: 0h 1m 3s 797ms Database type: Aminoacid Time for merging to sequencedb.lookup: 0h 0m 0s 3ms Time for processing: 0h 4m 39s 300ms Tmp conterminator_tmp/6937418719709452993/createtaxdb folder does not exist or is not a directory. Download taxdump.tar.gz 2020-05-05 13:45:37 URL:https://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz [51998442/51998442] -> "-" [1] Database created Remove temporary files conterminator_tmp/6937418719709452993/createtaxdb/createindex.sh: line 58: [: : integer expression expected Tmp conterminator_tmp/6937418719709452993/linclust folder does not exist or is not a directory. kmermatcher conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref --sub-mat nucl:nucleotide.out,aa:blosum62.out --alph-size 21 --min-seq-id 0.95 --kmer-per-seq 100 --spaced-kmer-mode 1 --kmer-per-seq-scale 0 --adjust-kmer-len 0 --mask 0 --mask-lower-case 0 --cov-mode 0 -k 0 -c 0.95 --max-seq-len 65535 --hash-shift 67 --split-memory-limit 0 --include-only-extendable 0 --ignore-multi-kmer 0 --threads 64 --compressed 0 -v 3 kmermatcher conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref --sub-mat nucl:nucleotide.out,aa:blosum62.out --alph-size 21 --min-seq-id 0.95 --kmer-per-seq 100 --spaced-kmer-mode 1 --kmer-per-seq-scale 0 --adjust-kmer-len 0 --mask 0 --mask-lower-case 0 --cov-mode 0 -k 0 -c 0.95 --max-seq-len 65535 --hash-shift 67 --split-memory-limit 0 --include-only-extendable 0 --ignore-multi-kmer 0 --threads 64 --compressed 0 -v 3 Database size: 73751620 type: Aminoacid Reduced amino acid alphabet: (A S T) (C) (D B N) (E Q Z) (F Y) (G) (H) (I V) (K R) (L J M) (P) (W) (X) Not enough memory to process at once need to split [=================================================================] 73.75M 43s 227ms Process file into 2 parts Generate k-mers list for 1 split [=================================================================] 73.75M 57s 821ms Sort kmer 0h 2m 44s 971ms Sort by rep. sequence 0h 0m 36s 775ms Generate k-mers list for 2 split [=================================================================] 73.75M 59s 874ms Sort kmer 0h 0m 21s 36ms Sort by rep. sequence 0h 0m 3s 483ms Merge splits ... Time for fill: 0h 1m 32s 459ms Time for merging to pref: 0h 0m 22s 166ms Time for processing: 0h 11m 40s 553ms rescorediagonal conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore1 --sub-mat nucl:nucleotide.out,aa:blosum62.out --rescore-mode 0 --wrapped-scoring 0 --filter-hits 0 -e 0.001 -c 0.95 -a 0 --cov-mode 0 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --add-self-matches 0 --sort-results 0 --db-load-mode 0 --threads 64 --compressed 0 -v 3 [=================================================================] 73.75M 30s 492ms Time for merging to pref_rescore1: 0h 0m 30s 761ms Time for processing: 0h 1m 32s 935ms clust conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore1 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pre_clust --cluster-mode 0 --max-iterations 1000 --similarity-type 2 --threads 64 --compressed 0 -v 3 Clustering mode: Set Cover [=================================================================] 1.00M 0s 65ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 201ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 198ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 194ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 194ms [=================================================================] 1.00M 0s 192ms [=================================================================] 1.00M 0s 191ms [=================================================================] 1.00M 0s 188ms [=================================================================] 1.00M 0s 186ms [=================================================================] 1.00M 0s 183ms [=================================================================] 1.00M 0s 178ms [=================================================================] 1.00M 0s 174ms [=================================================================] 1.00M 0s 169ms [=================================================================] 1.00M 0s 155ms [=================================================================] 1.00M 0s 131ms [=================================================================] 751.62K 0s 94ms Sort entries Find missing connections Found 6021788 new connections. Reconstruct initial order [=================================================================] 1.00M 0s 194ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 197ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 196ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 195ms [=================================================================] 1.00M 0s 194ms [=================================================================] 1.00M 0s 194ms [=================================================================] 1.00M 0s 193ms [=================================================================] 1.00M 0s 192ms [=================================================================] 1.00M 0s 190ms [=================================================================] 1.00M 0s 188ms [=================================================================] 1.00M 0s 185ms [=================================================================] 1.00M 0s 183ms [=================================================================] 1.00M 0s 177ms [=================================================================] 1.00M 0s 173ms [=================================================================] 1.00M 0s 169ms [=================================================================] 1.00M 0s 155ms [=================================================================] 1.00M 0s 131ms [=================================================================] 751.62K 0s 95ms Add missing connections [=================================================================] 73.75M 3s 71ms Time for read in: 0h 0m 55s 134ms Total time: 0h 1m 10s 664ms Size of the sequence database: 73751620 Size of the alignment database: 73751620 Number of clusters: 69196986 Writing results 0h 0m 25s 185ms Time for merging to pre_clust: 0h 0m 20s 696ms Time for processing: 0h 2m 19s 837ms createsubdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy -v 3 --subdb-mode 1 Time for merging to input_step_redundancy: 0h 0m 9s 486ms Time for processing: 0h 0m 25s 257ms createsubdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter1 -v 3 --subdb-mode 1 Time for merging to pref_filter1: 0h 0m 9s 362ms Time for processing: 0h 0m 25s 757ms filterdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter1 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter2 --filter-file conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy Filtering using file(s) [=================================================================] 69.20M 18s 92ms Time for merging to pref_filter2: 0h 0m 30s 266ms Time for processing: 0h 1m 5s 686ms rescorediagonal conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter2 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore2 --sub-mat nucl:nucleotide.out,aa:blosum62.out --rescore-mode 1 --wrapped-scoring 0 --filter-hits 1 -e 0.001 -c 0.95 -a 0 --cov-mode 0 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --add-self-matches 0 --sort-results 0 --db-load-mode 0 --threads 64 --compressed 0 -v 3 [=================================================================] 69.20M 29s 670ms Time for merging to pref_rescore2: 0h 0m 29s 593ms Time for processing: 0h 1m 19s 75ms align conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore2 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/aln --sub-mat nucl:nucleotide.out,aa:blosum62.out -a 0 --alignment-mode 3 --wrapped-scoring 0 -e 0.001 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --alt-ali 0 -c 0.95 --cov-mode 0 --max-seq-len 65535 --comp-bias-corr 1 --realign 0 --max-rejected 2147483647 --max-accept 2147483647 --add-self-matches 0 --db-load-mode 0 --pca 1 --pcb 1.5 --score-bias 0 --gap-open 11 --gap-extend 1 --threads 64 --compressed 0 -v 3 Compute score, coverage and sequence identity Query database size: 69196986 type: Aminoacid Target database size: 69196986 type: Aminoacid Calculation of alignments *** Error in `/global/cscratch1/sd/acamargo/GTDB/./conterminator/build/bin/conterminator': munmap_chunk(): invalid pointer: 0x00002ab2d008fe80 *** conterminator_tmp/6937418719709452993/linclust/7399153080635485316/linclust.sh: line 75: 27861 Aborted $RUNNER "$MMSEQS" "${ALIGN_MODULE}" "$INPUT" "$INPUT" "$RESULTDB" "${TMP_PATH}/aln" ${ALIGNMENT_PAR} [=Error: Alignment step died Error: kmermatcher step died srun: error: nid00369: task 0: Exited with exit code 1 srun: Terminating job step 30385822.0
Updating lib/mmseqs to MMSeq2's latest commit fixed the issue
lib/mmseqs
Version:
commit 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7
Command:
Output: