search

bioinfologics / satsuma2

FFT cross-correlation based synteny aligner, (re)designed to make full use of parallel computing
41 stars 13 forks source link

terminate called after throwing an instance of 'std::length_error' #10

Closed mictadlo closed 5 years ago

mictadlo commented 6 years ago

Hi, I compiled satsuma2 with gcc 7.2 and run it on 150GB memory computer and I got:

terminate called after throwing an instance of 'std::length_error'
  what():  vector::_M_default_append
/work/waterhouse_team/miniconda2/envs/satsuma/bin//satsuma_run.sh: line 11: 20993 Aborted                 (core dumped) /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 11 results-benthVStaboco/9/kmatch_results.k11 11 10 1

Here is the full log:

SATSUMA: Welcome to SatsumaSynteny! Current date and time: 2018/04/12 08:29:21
Path for Satsuma2: '/work/waterhouse_team/miniconda2/envs/satsuma/bin/'
Executing SatsumaSynteny2
Setting up grid.
Preparing...
select=0    chunks=956975
chunks: 956975
select=0    chunks=25915
chunks: 25915
size x=39874 y=1080 size=43063920
Initializing multimatches...
Done.
SATSUMA: Acquiring seeds, date and time: 2018/04/12 08:30:04
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 11 results-benthVStaboco/9/kmatch_results.k11 11 10 1; touch results-benthVStaboco/9/kmatch_results.k11.finished
TIME_LOG: TIME_LOG: 15234858048051523485804805 -  -  Loading array  Loading array 

Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71564018 elements created
TIME_LOG: 1523485806284 -  Sorting array 
Kmer array sorted
Kmer array filtered to 46937 elements
Kmer array with 2672922505 elements created
TIME_LOG: 1523485877647 -  Sorting array 
Kmer array sorted
terminate called after throwing an instance of 'std::length_error'
  what():  vector::_M_default_append
/work/waterhouse_team/miniconda2/envs/satsuma/bin//satsuma_run.sh: line 11: 20993 Aborted                 (core dumped) /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 11 results-benthVStaboco/9/kmatch_results.k11 11 10 1
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 13 results-benthVStaboco/9/kmatch_results.k13 13 12 1; touch results-benthVStaboco/9/kmatch_results.k13.finished
TIME_LOG: TIME_LOG: 1523486782425 -  Loading array 1523486782425 -  Loading array 

Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71549823 elements created
TIME_LOG: 1523486783897 -  Sorting array 
Kmer array sorted
Kmer array filtered to 7257300 elements
Kmer array with 2672576657 elements created
TIME_LOG: 1523486855305 -  Sorting array 
Kmer array sorted
Kmer array filtered to 343170 elements
Starting to create matching positions
28420 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 12kmers to results-benthVStaboco/9/kmatch_results.k13
27967 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 15 results-benthVStaboco/9/kmatch_results.k15 15 14 1; touch results-benthVStaboco/9/kmatch_results.k15.finished
TIME_LOG: 1523487155753 -  Loading array 
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
TIME_LOG: 1523487155753 -  Loading array 
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Kmer array with 71535643 elements created
TIME_LOG: 1523487157223 -  Sorting array 
Kmer array sorted
Kmer array filtered to 30508471 elements
Kmer array with 2672230813 elements created
TIME_LOG: 1523487228643 -  Sorting array 
Kmer array sorted
Kmer array filtered to 105677914 elements
Starting to create matching positions
3331474 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 14kmers to results-benthVStaboco/9/kmatch_results.k15
2876906 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 17 results-benthVStaboco/9/kmatch_results.k17 17 16 1; touch results-benthVStaboco/9/kmatch_results.k17.finished
TIME_LOG: 1523487558322 -  Loading array 
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
TIME_LOG: 1523487558322 -  Loading array 
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71521470 elements created
TIME_LOG: 1523487559795 -  Sorting array 
Kmer array sorted
Kmer array filtered to 42362691 elements
Kmer array with 2671884969 elements created
TIME_LOG: 1523487631152 -  Sorting array 
Kmer array sorted
Kmer array filtered to 668495369 elements
Starting to create matching positions
7984017 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 16kmers to results-benthVStaboco/9/kmatch_results.k17
5189080 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 19 results-benthVStaboco/9/kmatch_results.k19 19 18 1; touch results-benthVStaboco/9/kmatch_results.k19.finished
TIME_LOG: TIME_LOG: 15234879802081523487980208 -  Loading array  -  Loading array 

Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Kmer array with 71507303 elements created
TIME_LOG: 1523487981679 -  Sorting array 
Kmer array sorted
Kmer array filtered to 47397363 elements
Kmer array with 2671539128 elements created
TIME_LOG: 1523488053037 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1117820443 elements
Starting to create matching positions
5993620 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 18kmers to results-benthVStaboco/9/kmatch_results.k19
2516931 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 21 results-benthVStaboco/9/kmatch_results.k21 21 20 1; touch results-benthVStaboco/9/kmatch_results.k21.finished
TIME_LOG: TIME_LOG: 15234884021481523488402148 -  -  Loading array  Loading array 

Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Kmer array with 71493140 elements created
TIME_LOG: 1523488403620 -  Sorting array 
Kmer array sorted
Kmer array filtered to 50880810 elements
Kmer array with 2671193290 elements created
TIME_LOG: 1523488475028 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1329595381 elements
Starting to create matching positions
4716262 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 20kmers to results-benthVStaboco/9/kmatch_results.k21
1418044 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 23 results-benthVStaboco/9/kmatch_results.k23 23 22 1; touch results-benthVStaboco/9/kmatch_results.k23.finished
TIME_LOG: TIME_LOG: 15234888259551523488825955 -  Loading array  -  Loading array 

Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71478985 elements created
TIME_LOG: 1523488827426 -  Sorting array 
Kmer array sorted
Kmer array filtered to 53769215 elements
Kmer array with 2670847457 elements created
TIME_LOG: 1523488898825 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1469351970 elements
Starting to create matching positions
4118266 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 22kmers to results-benthVStaboco/9/kmatch_results.k23
1082842 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 25 results-benthVStaboco/9/kmatch_results.k25 25 24 1; touch results-benthVStaboco/9/kmatch_results.k25.finished
TIME_LOG: TIME_LOG: 15234892532881523489253288 -  Loading array  -  Loading array 

Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71464835 elements created
TIME_LOG: 1523489254760 -  Sorting array 
Kmer array sorted
Kmer array filtered to 56247595 elements
Kmer array with 2670501628 elements created
TIME_LOG: 1523489326207 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1585012704 elements
Starting to create matching positions
3656508 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 24kmers to results-benthVStaboco/9/kmatch_results.k25
885345 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 27 results-benthVStaboco/9/kmatch_results.k27 27 26 1; touch results-benthVStaboco/9/kmatch_results.k27.finished
TIME_LOG: 1523489680650 -  Loading array 
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
TIME_LOG: 1523489680650 -  Loading array 
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71450691 elements created
TIME_LOG: 1523489682125 -  Sorting array 
Kmer array sorted
Kmer array filtered to 58380074 elements
Kmer array with 2670155806 elements created
TIME_LOG: 1523489753675 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1685783214 elements
Starting to create matching positions
3229382 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 26kmers to results-benthVStaboco/9/kmatch_results.k27
727689 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 29 results-benthVStaboco/9/kmatch_results.k29 29 28 1; touch results-benthVStaboco/9/kmatch_results.k29.finished
TIME_LOG: TIME_LOG: 1523490109908 -  Loading array 1523490109908 -  Loading array 

Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Kmer array with 71436559 elements created
TIME_LOG: 1523490111383 -  Sorting array 
Kmer array sorted
Kmer array filtered to 60208324 elements
Kmer array with 2669809985 elements created
TIME_LOG: 1523490182954 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1774130396 elements
Starting to create matching positions
2835057 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 28kmers to results-benthVStaboco/9/kmatch_results.k29
605970 matches dumped
Running seed pre-filter: 
  /work/waterhouse_team/miniconda2/envs/satsuma/bin//KMatch Nitab-v4.5_genome_Chr_Edwards2017/9.fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta 31 results-benthVStaboco/9/kmatch_results.k31 31 30 1; touch results-benthVStaboco/9/kmatch_results.k31.finished
TIME_LOG: TIME_LOG: 1523490537984 -  Loading array 1523490537984 -  Loading array 

Loading fasta /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta into kmer array
Loading fasta Nitab-v4.5_genome_Chr_Edwards2017/9.fasta into kmer array
Kmer array with 71422438 elements created
TIME_LOG: 1523490539459 -  Sorting array 
Kmer array sorted
Kmer array filtered to 61776037 elements
Kmer array with 2669464168 elements created
TIME_LOG: 1523490611052 -  Sorting array 
Kmer array sorted
Kmer array filtered to 1851670095 elements
Starting to create matching positions
2473167 matching positions
Sorting the matches
Dumping matches of 1 kmers with jumps of up to 30kmers to results-benthVStaboco/9/kmatch_results.k31
495300 matches dumped
Waiting for seed pre-filters...
loading results for k=11
/var/spool/PBS/mom_priv/jobs/3036313.pbs.SC: line 16: 20466 Segmentation fault      (core dumped) SatsumaSynteny2 -threads 5 -q Nitab-v4.5_genome_Chr_Edwards2017/9.fasta -t /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta -slaves 100 -sl_mem 100 -o results-benthVStaboco/9
terminate called after throwing an instance of 'CMException'
/var/spool/PBS/mom_priv/jobs/3036313.pbs.SC: line 17: 33215 Aborted                 (core dumped) BlockDisplaySatsuma -i results-benthVStaboco/9/satsuma_summary.chained.out -q Nitab-v4.5_genome_Chr_Edwards2017/9.fasta -t /work/waterhouse_team/Solanaceae/Nbenth_SolGV251/SolGNb251_Genome-1-19.fasta > results-benthVStaboco/9/mizbee.txt

-----
PBS Job 3036313.pbs
CPU time  : 01:18:08
Wall time : 01:29:54
Mem usage : 81388512kb

What did I miss?

Thank you in advance,

Michal

jonwright99 commented 6 years ago

Hi Michal, It looks like you're comparing Nicotiana chromosomes. Are these from publically available assemblies? If so, I'll download them and try to recreate the error.

mictadlo commented 6 years ago

Hi Jon, Yes, it is. Have you had any luck?

Michal

jonwright99 commented 6 years ago

Can you point me towards the public data? In April the Nbenth_SolGV251 assembly was not publically available and your supervisor wouldn't allow you to share it.

jonwright99 commented 5 years ago

I think this has been fixed in issue #12 so I'm closing it