search

isovic / graphmap

GraphMap - A highly sensitive and accurate mapper for long, error-prone reads http://www.nature.com/ncomms/2016/160415/ncomms11307/full/ncomms11307.html Note: This was the original repository which will no longer be officially maintained. Please use the new official repository here:
https://github.com/lbcb-sci/graphmap2
MIT License
178 stars 44 forks source link

Segmentation fault #70

Closed mictadlo closed 6 years ago

mictadlo commented 7 years ago

Hi, I got Segmentation fault during alignment step:

[14:01:44 BuildIndexes] Loading reference sequences.
[14:01:48 SetupIndex_] Building the index for shape: '11110111101111'.
[14:01:49 Create] Allocated memory for a list of 202099661 seeds (128 bits each) (0.00002 sec, diff: 0.84232 sec).
[14:01:49 Create] Memory consumption: [currentRSS = 1184 MB, peakRSS = 1184 MB]
[14:01:49 Create] Collecting seeds.
[14:01:49 Create] Minimizer seeds will be used. Minimizer window is 5.

[14:01:49 Create] [currentRSS = 1184 MB, peakRSS = 1184 MB] Sequence: 1/46, len: 29070452, name: 'chr01'
[14:01:50 Create] [currentRSS = 1336 MB, peakRSS = 1336 MB] Sequence: 2/46, len: 29511734, name: 'chr02'
[14:01:50 Create] [currentRSS = 1482 MB, peakRSS = 1482 MB] Sequence: 3/46, len: 35020413, name: 'chr03'
[14:01:51 Create] [currentRSS = 1666 MB, peakRSS = 1666 MB] Sequence: 4/46, len: 37105743, name: 'chr04'
[14:01:53 Create] [currentRSS = 1862 MB, peakRSS = 1862 MB] Sequence: 5/46, len: 41853232, name: 'chr05'
[14:01:54 Create] [currentRSS = 2074 MB, peakRSS = 2074 MB] Sequence: 6/46, len: 37593364, name: 'chr06'
[14:01:55 Create] [currentRSS = 2272 MB, peakRSS = 2272 MB] Sequence: 7/46, len: 35028021, name: 'chr07'
[14:01:56 Create] [currentRSS = 2442 MB, peakRSS = 2442 MB] Sequence: 8/46, len: 44889171, name: 'chr08'
[14:01:57 Create] [currentRSS = 2672 MB, peakRSS = 2672 MB] Sequence: 9/46, len: 41306725, name: 'chr09'
[14:01:59 Create] [currentRSS = 2886 MB, peakRSS = 2886 MB] Sequence: 10/46, len: 37674811, name: 'chr10'
[14:02:00 Create] [currentRSS = 3078 MB, peakRSS = 3078 MB] Sequence: 11/46, len: 27954350, name: 'chr11'
[14:02:01 Create] [currentRSS = 3224 MB, peakRSS = 3224 MB] Sequence: 12/46, len: 616199, name: 'mito1'
[14:02:01 Create] [currentRSS = 3226 MB, peakRSS = 3226 MB] Sequence: 13/46, len: 610266, name: 'mito2'
[14:02:01 Create] [currentRSS = 3230 MB, peakRSS = 3230 MB] Sequence: 14/46, len: 1980050, name: 'mito3'
[14:02:01 Create] [currentRSS = 3240 MB, peakRSS = 3240 MB] Sequence: 15/46, len: 924339, name: 'mito4'
[14:02:01 Create] [currentRSS = 3246 MB, peakRSS = 3246 MB] Sequence: 16/46, len: 474705, name: 'mito5'
[14:02:01 Create] [currentRSS = 3248 MB, peakRSS = 3248 MB] Sequence: 17/46, len: 491779, name: 'mito6'
[14:02:01 Create] [currentRSS = 3250 MB, peakRSS = 3250 MB] Sequence: 18/46, len: 263417, name: 'mito7'
[14:02:01 Create] [currentRSS = 3252 MB, peakRSS = 3252 MB] Sequence: 19/46, len: 456355, name: 'mito8'
[14:02:01 Create] [currentRSS = 3254 MB, peakRSS = 3254 MB] Sequence: 20/46, len: 332376, name: 'mito9'
[14:02:01 Create] [currentRSS = 3256 MB, peakRSS = 3256 MB] Sequence: 21/46, len: 699348, name: 'mito10'
[14:02:01 Create] [currentRSS = 3260 MB, peakRSS = 3260 MB] Sequence: 22/46, len: 172499, name: 'mito11'
[14:02:01 Create] [currentRSS = 3260 MB, peakRSS = 3260 MB] Sequence: 23/46, len: 169972, name: 'ENA|HF677508|HF677508.1'
[14:02:01 Create] [currentRSS = 3262 MB, peakRSS = 3262 MB] Sequence: 24/46, len: 29070452, name: 'chr01'
[14:02:02 Create] [currentRSS = 3412 MB, peakRSS = 3412 MB] Sequence: 25/46, len: 29511734, name: 'chr02'
[14:02:03 Create] [currentRSS = 3558 MB, peakRSS = 3558 MB] Sequence: 26/46, len: 35020413, name: 'chr03'
[14:02:04 Create] [currentRSS = 3742 MB, peakRSS = 3742 MB] Sequence: 27/46, len: 37105743, name: 'chr04'
[14:02:05 Create] [currentRSS = 3938 MB, peakRSS = 3938 MB] Sequence: 28/46, len: 41853232, name: 'chr05'
[14:02:06 Create] [currentRSS = 4150 MB, peakRSS = 4150 MB] Sequence: 29/46, len: 37593364, name: 'chr06'
[14:02:09 Create] [currentRSS = 4348 MB, peakRSS = 7354 MB] Sequence: 30/46, len: 35028021, name: 'chr07'
[14:02:10 Create] [currentRSS = 4518 MB, peakRSS = 7354 MB] Sequence: 31/46, len: 44889171, name: 'chr08'
[14:02:11 Create] [currentRSS = 4748 MB, peakRSS = 7354 MB] Sequence: 32/46, len: 41306725, name: 'chr09'
[14:02:12 Create] [currentRSS = 4960 MB, peakRSS = 7354 MB] Sequence: 33/46, len: 37674811, name: 'chr10'
[14:02:13 Create] [currentRSS = 5154 MB, peakRSS = 7354 MB] Sequence: 34/46, len: 27954350, name: 'chr11'
[14:02:14 Create] [currentRSS = 5298 MB, peakRSS = 7354 MB] Sequence: 35/46, len: 616199, name: 'mito1'
[14:02:14 Create] [currentRSS = 5302 MB, peakRSS = 7354 MB] Sequence: 36/46, len: 610266, name: 'mito2'
[14:02:14 Create] [currentRSS = 5306 MB, peakRSS = 7354 MB] Sequence: 37/46, len: 1980050, name: 'mito3'
[14:02:14 Create] [currentRSS = 5316 MB, peakRSS = 7354 MB] Sequence: 38/46, len: 924339, name: 'mito4'
[14:02:14 Create] [currentRSS = 5320 MB, peakRSS = 7354 MB] Sequence: 39/46, len: 474705, name: 'mito5'
[14:02:14 Create] [currentRSS = 5324 MB, peakRSS = 7354 MB] Sequence: 40/46, len: 491779, name: 'mito6'
[14:02:15 Create] [currentRSS = 5326 MB, peakRSS = 7354 MB] Sequence: 41/46, len: 263417, name: 'mito7'
[14:02:15 Create] [currentRSS = 5328 MB, peakRSS = 7354 MB] Sequence: 42/46, len: 456355, name: 'mito8'
[14:02:15 Create] [currentRSS = 5330 MB, peakRSS = 7354 MB] Sequence: 43/46, len: 332376, name: 'mito9'
[14:02:15 Create] [currentRSS = 5332 MB, peakRSS = 7354 MB] Sequence: 44/46, len: 699348, name: 'mito10'
[14:02:15 Create] [currentRSS = 5336 MB, peakRSS = 7354 MB] Sequence: 45/46, len: 172499, name: 'mito11'
[14:02:15 Create] [currentRSS = 5336 MB, peakRSS = 7354 MB] Sequence: 46/46, len: 169972, name: 'ENA|HF677508|HF677508.1'
[14:02:15 Create] Final memory allocation after collecting seeds: [currentRSS = 5338 MB, peakRSS = 7354 MB]
[14:02:15 Create] Sorting the seeds using 12 threads.
[14:02:20 Create] Generating the hash table.
[14:02:22 Create] Calculating the distribution statistics for key counts.
[14:02:22 Create] Index statistics: average key count = 21.790629, max key count = 512124.000000, std dev = 168.266576, percentil (99.00%) (count cutoff) = 233.000000
[14:02:23 Create] Memory consumption: [currentRSS = 6127 MB, peakRSS = 7354 MB]
[14:02:23 SetupIndex_] Finished building index.
[14:02:23 SetupIndex_] Storing the index to file: ‘/files/ref.fasta.gmidx'.
[14:02:28 Index] Memory consumption: [currentRSS = 5742 MB, peakRSS = 7354 MB]

[14:02:28 Run] Hits will be thresholded at the percentil value (percentil: 99.000000%, frequency: 233).
[14:02:28 Run] Minimizers will be used. Minimizer window length: 5
[14:02:28 Run] Automatically setting the maximum allowed number of regions: max. 2907, attempt to reduce after 0
[14:02:28 Run] Reference genome is assumed to be linear.
[14:02:28 Run] Only one alignment will be reported per mapped read.
[14:02:28 ProcessReads] Reads will be loaded in batches of up to 1024 MB in size.
[14:02:33 ProcessReads] Batch of 59999 reads (1024 MiB) loaded in 5.71 sec. (8381032 bases)
[14:02:33 ProcessReads] Memory consumption: [currentRSS = 6739 MB, peakRSS = 7354 MB]
[14:02:33 ProcessReads] Using 12 threads.

[14:02:33 ProcessReads] [CPU time: 5.71 sec, RSS: 6740 MB] Read: 11/59999 (0.02%) [m: 0, u: 0], length = 4348, qname: m160705_083016_42229_c10102517255000000182...
[14:02:34 ProcessReads] [CPU time: 12.85 sec, RSS: 6878 MB] Read: 31/59999 (0.05%) [m: 14, u: 6], length = 7326, qname: m160705_083016_42229_c101025172550000001...
[14:02:34 ProcessReads] [CPU time: 16.03 sec, RSS: 6883 MB] Read: 41/59999 (0.07%) [m: 22, u: 8], length = 6606, qname: m160705_083016_42229_c101025172550000001...
[14:02:35 ProcessReads] [CPU time: 18.95 sec, RSS: 6883 MB] Read: 51/59999 (0.09%) [m: 31, u: 9], length = 6953, qname: m160705_083016_42229_c101025172550000001...
[14:02:35 ProcessReads] [CPU time: 21.65 sec, RSS: 6888 MB] Read: 60/59999 (0.10%) [m: 36, u: 13], length = 11687, qname: m160705_083016_42229_c1010251725500000...
[14:02:35 ProcessReads] [CPU time: 26.36 sec, RSS: 6891 MB] Read: 72/59999 (0.12%) [m: 47, u: 14], length = 12436, qname: m160705_083016_42229_c1010251725500000...
[14:02:36 ProcessReads] [CPU time: 31.28 sec, RSS: 6904 MB] Read: 91/59999 (0.15%) [m: 62, u: 18], length = 31236, qname: m160705_083016_42229_c1010251725500000...
[14:02:38 ProcessReads] [CPU time: 59.76 sec, RSS: 7022 MB] Read: 198/59999 (0.33%) [m: 134, u: 53], length = 13695, qname: m160705_083016_42229_c10102517255000...
[14:02:38 ProcessReads] [CPU time: 64.59 sec, RSS: 7022 MB] Read: 212/59999 (0.35%) [m: 146, u: 55], length = 22859, qname: m160705_083016_42229_c10102517255000...
[14:02:40 ProcessReads] [CPU time: 81.76 sec, RSS: 7028 MB] Read: 261/59999 (0.44%) [m: 184, u: 66], length = 306, qname: m160705_083016_42229_c1010251725500000...
[14:02:40 ProcessReads] [CPU time: 82.98 sec, RSS: 7031 MB] Read: 267/59999 (0.45%) [m: 190, u: 66], length = 12382, qname: m160705_083016_42229_c10102517255000...
[14:02:40 ProcessReads] [CPU time: 88.05 sec, RSS: 7109 MB] Read: 289/59999 (0.48%) [m: 209, u: 69], length = 7574, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 91.03 sec, RSS: 7045 MB] Read: 299/59999 (0.50%) [m: 216, u: 72], length = 3814, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 92.47 sec, RSS: 7048 MB] Read: 305/59999 (0.51%) [m: 219, u: 75], length = 6443, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 94.80 sec, RSS: 7048 MB] Read: 314/59999 (0.52%) [m: 228, u: 75], length = 7370, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 98.11 sec, RSS: 7048 MB] Read: 321/59999 (0.54%) [m: 233, u: 77], length = 1872, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 99.13 sec, RSS: 7056 MB] Read: 325/59999 (0.54%) [m: 236, u: 78], length = 5394, qname: m160705_083016_42229_c101025172550000...
[14:02:41 ProcessReads] [CPU time: 101.32 sec, RSS: 7057 MB] Read: 332/59999 (0.55%) [m: 239, u: 82], length = 5872, qname: m160705_083016_42229_c10102517255000...
[14:02:42 ProcessReads] [CPU time: 103.98 sec, RSS: 7044 MB] Read: 344/59999 (0.57%) [m: 249, u: 84], length = 15687, qname: m160705_083016_42229_c1010251725500...
[14:02:42 ProcessReads] [CPU time: 109.03 sec, RSS: 7055 MB] Read: 365/59999 (0.61%) [m: 267, u: 87], length = 2683, qname: m160705_083016_42229_c10102517255000...
[14:02:42 ProcessReads] [CPU time: 110.15 sec, RSS: 7055 MB] Read: 370/59999 (0.62%) [m: 271, u: 88], length = 15387, qname: m160705_083016_42229_c1010251725500...
[14:02:43 ProcessReads] [CPU time: 115.02 sec, RSS: 7056 MB] Read: 382/59999 (0.64%) [m: 281, u: 90], length = 6321, qname: m160705_083016_42229_c10102517255000...
[14:02:43 ProcessReads] [CPU time: 124.74 sec, RSS: 7056 MB] Read: 405/59999 (0.68%) [m: 297, u: 97], length = 9637, qname: m160705_083016_42229_c10102517255000...
[14:02:44 ProcessReads] [CPU time: 136.24 sec, RSS: 7125 MB] Read: 445/59999 (0.74%) [m: 327, u: 107], length = 7368, qname: m160705_083016_42229_c1010251725500...
[14:02:45 ProcessReads] [CPU time: 139.02 sec, RSS: 7125 MB] Read: 454/59999 (0.76%) [m: 333, u: 110], length = 3924, qname: m160705_083016_42229_c1010251725500...
[14:02:45 ProcessReads] [CPU time: 140.77 sec, RSS: 7125 MB] Read: 460/59999 (0.77%) [m: 337, u: 112], length = 11873, qname: m160705_083016_42229_c101025172550...
[14:02:45 ProcessReads] [CPU time: 144.70 sec, RSS: 7127 MB] Read: 477/59999 (0.80%) [m: 354, u: 112], length = 10618, qname: m160705_083016_42229_c101025172550...
[14:02:45 ProcessReads] [CPU time: 148.28 sec, RSS: 7146 MB] Read: 493/59999 (0.82%) [m: 368, u: 114], length = 3475, qname: m160705_083016_42229_c1010251725500...
[14:02:45 ProcessReads] [CPU time: 149.73 sec, RSS: 7146 MB] Read: 499/59999 (0.83%) [m: 373, u: 115], length = 11617, qname: m160705_083016_42229_c101025172550...
[14:02:46 ProcessReads] [CPU time: 160.34 sec, RSS: 7112 MB] Read: 537/59999 (0.90%) [m: 401, u: 125], length = 6591, qname: m160705_083016_42229_c1010251725500...
[14:02:47 ProcessReads] [CPU time: 162.58 sec, RSS: 7112 MB] Read: 546/59999 (0.91%) [m: 407, u: 128], length = 13421, qname: m160705_083016_42229_c101025172550...
[14:02:47 ProcessReads] [CPU time: 167.00 sec, RSS: 7091 MB] Read: 567/59999 (0.95%) [m: 423, u: 133], length = 14492, qname: m160705_083016_42229_c101025172550...
[14:02:47 ProcessReads] [CPU time: 172.32 sec, RSS: 7091 MB] Read: 587/59999 (0.98%) [m: 441, u: 135], length = 11667, qname: m160705_083016_42229_c101025172550...
[14:02:48 ProcessReads] [CPU time: 176.89 sec, RSS: 7091 MB] Read: 597/59999 (1.00%) [m: 447, u: 139], length = 3217, qname: m160705_083016_42229_c1010251725500...
[14:02:48 ProcessReads] [CPU time: 178.49 sec, RSS: 7091 MB] Read: 601/59999 (1.00%) [m: 450, u: 140], length = 13338, qname: m160705_083016_42229_c101025172550...
[14:02:48 ProcessReads] [CPU time: 184.81 sec, RSS: 7128 MB] Read: 619/59999 (1.03%) [m: 465, u: 143], length = 2394, qname: m160705_083016_42229_c1010251725500...
[14:02:48 ProcessReads] [CPU time: 186.00 sec, RSS: 7070 MB] Read: 623/59999 (1.04%) [m: 468, u: 144], length = 12095, qname: m160705_083016_42229_c101025172550...
[14:02:49 ProcessReads] [CPU time: 190.70 sec, RSS: 7057 MB] Read: 639/59999 (1.07%) [m: 480, u: 148], length = 14300, qname: m160705_083016_42229_c101025172550...
[14:02:49 ProcessReads] [CPU time: 196.10 sec, RSS: 7058 MB] Read: 655/59999 (1.09%) [m: 494, u: 150], length = 16998, qname: m160705_083016_42229_c101025172550...
[14:02:50 ProcessReads] [CPU time: 201.50 sec, RSS: 7094 MB] Read: 681/59999 (1.14%) [m: 514, u: 156], length = 6779, qname: m160705_083016_42229_c1010251725500...
[14:02:50 ProcessReads] [CPU time: 204.67 sec, RSS: 7058 MB] Read: 690/59999 (1.15%) [m: 523, u: 156], length = 11684, qname: m160705_083016_42229_c101025172550...
[14:02:50 ProcessReads] [CPU time: 208.58 sec, RSS: 7060 MB] Read: 706/59999 (1.18%) [m: 536, u: 159], length = 2811, qname: m160705_083016_42229_c1010251725500...
[14:02:50 ProcessReads] [CPU time: 209.81 sec, RSS: 7072 MB] Read: 712/59999 (1.19%) [m: 542, u: 159], length = 19633, qname: m160705_083016_42229_c101025172550...
[14:02:51 ProcessReads] [CPU time: 216.35 sec, RSS: 7075 MB] Read: 736/59999 (1.23%) [m: 563, u: 162], length = 8631, qname: m160705_083016_42229_c1010251725500...
[14:02:51 ProcessReads] [CPU time: 219.30 sec, RSS: 7078 MB] Read: 746/59999 (1.24%) [m: 571, u: 164], length = 10666, qname: m160705_083016_42229_c101025172550...
[14:02:52 ProcessReads] [CPU time: 222.93 sec, RSS: 7080 MB] Read: 758/59999 (1.26%) [m: 581, u: 166], length = 4342, qname: m160705_083016_42229_c1010251725500...
[14:02:52 ProcessReads] [CPU time: 224.81 sec, RSS: 7080 MB] Read: 768/59999 (1.28%) [m: 587, u: 170], length = 11077, qname: m160705_083016_42229_c101025172550...
[14:02:53 ProcessReads] [CPU time: 240.68 sec, RSS: 7111 MB] Read: 816/59999 (1.36%) [m: 620, u: 185], length = 11473, qname: m160705_083016_42229_c101025172550.../var/spool/PBS/mom_priv/jobs/1956588.pbs.SC: line 13: 16993 Segmentation fault      ./bin/Linux-x64/graphmap align -t 12 -d /files/RSnSeQ_110.fastq -r /files/ref.fasta -o /files/alignement.sam

-----
PBS Job 1956588.pbs
CPU time  : 00:05:37
Wall time : 00:01:11
Mem usage : 6611812kb

What did I miss?

Thank you in advance.

Michal

npavlovikj commented 7 years ago

Hi,

I am encountering the same problem as described above. I have been running "graphmap" with the following command "graphmap align -t 8 -r ref.fasta -d reads.fasta -o gm_results.sam". However, for some reads from "ref.fasta" I get "Segmentation fault": [20:59:03 ProcessReads] [CPU time: 1028.20 sec, RSS: 1425 MB] Read: 72876/81352 (89.58%) [m: 60681, u: 12188], length = 1586, qname: SSscaffold-832 1586 /var/spool/slurmd/job7479643/slurm_script: line 15: 137866 Segmentation fault graphmap align -t 8 -r ref.fasta -d reads.fasta -o gm_results.sam

I am using 150gb of RAM, and the consumption of this job is ~2GB, so I don't think it is a memory issue. One of the reads for which this happens is:

SSscaffold-832 1586 CCGGCGCCCTGGTCGGCGGCCGTCCCGGCCACGGCCCGGACGGCGAGCTGACCGTGGCCGCCGCCCCGGGCGTGTACGAG When I remove these reads, and run "graphmap" again, the alignment finishes ok.

I would appreciate if you have any suggestions why this occurs, and how can I can avoid it.

Thank you, Natasha

npavlovikj commented 7 years ago

@isovic , thank you so much for fixing this issue. I tried the new release, and the "Segmentation fault" error is now resolved.

isovic commented 6 years ago

Hi Natasha, Thank you for reporting that the issue is resolved!

Michal, did the update solve the segfault you were experiencing?

Best regards, Ivan.

mictadlo commented 6 years ago

I did not have time to try it again but @nnnagara said it works for him with the new release.