DALIGNER seg fault - Githubissues

StefanoLonardi commented 8 years ago

I was getting segmentation faults in the DALIGNER step, so I took the most recent source code from Gene Myers' github repository, and the segmentation faults went away. You might want to update your third-party links to most recent version of the DALIGNER. Other users might encounter the same problem.

fxia22 commented 8 years ago

Thank you, we will update it.

ilanshom commented 8 years ago

Hi, Stefano. It turns out that some changes in the new DAligner are causing the consensus part of HINGE to break. So we are temporarily reverting to the older version of DAligner until we can fix things to work with the new version.

StefanoLonardi commented 8 years ago

Thanks for letting me know. I was able to run all the steps using the most recent DALIGNER until pruning_and_clipping.py. This last steps produced the following messages

stelo@H4:~/HINGE/data_dark$ pruning_and_clipping.py cowpea.edges.hinges cowpea.hinge.list demo
couldn't finish sparsification1970
couldn't finish sparsification1970

I spent many days running the newest DALIGNER on my dataset (because the old version was segfaulting). I will wait for you to fix HINGE with the new version. Please post here a message when you think HINGE is fixed. I tried to run the next step, and here is the message

stelo@H4:~/HINGE/data_dark$ get_draft_path.py $PWD cowpea cowpeademo.G2.graphml
Traceback (most recent call last):
  File "/home/stelo/HINGE/scripts/get_draft_path.py", line 125, in <module>
    segment = get_string(path_var)
  File "/home/stelo/HINGE/scripts/get_draft_path.py", line 44, in get_string
    assert itm[1][0] >= itm[1][1]
AssertionError

govinda-kamath commented 8 years ago

Thanks Sefano!

The first step is essentially a message with respect to the sparsification we do to visualise the assembly. While the consensus does not run, you can visualise the quality of the assembly by viewing cowpeademo.G2c.graphml on Gephi and running the forceatlas2 protocol (alternatively you can visualise it networkx and forceatlas2 on python as done in cell 17-18 here (without the colouring though)).

govinda-kamath commented 8 years ago

Hi Stefano,

We've fixed the new version of daligner with HINGE.

Thanks, Govinda.

StefanoLonardi commented 8 years ago

thanks. I will update HINGE and try again.

StefanoLonardi commented 8 years ago

I updated HINGE, but I am still getting the same error

stelo@H4:~/HINGE/data_dark$ get_draft_path.py $PWD cowpea cowpeademo.G2.graphml
Traceback (most recent call last):
  File "/home/stelo/HINGE/scripts/get_draft_path.py", line 125, in <module>
    segment = get_string(path_var)
  File "/home/stelo/HINGE/scripts/get_draft_path.py", line 44, in get_string
    assert itm[1][0] >= itm[1][1]
AssertionError

Below please find the files I have (I have tried pruning_and_clipping.py again before running get_draft_path.py)

-rw-rw-r--  1 stelo stelo  1506184035 Aug  9 08:10 .cowpea.bps
-rw-rw-r--  1 stelo stelo     2416756 Aug 12 13:16 cowpea.contained.txt
-rw-rw-r--  1 stelo stelo   957355832 Aug 12 01:16 cowpea.coverage.txt
-rw-rw-r--  1 stelo stelo         468 Aug  9 18:22 cowpea.db
-rw-rw-r--  1 stelo stelo     1093924 Aug 12 13:53 cowpea.deadends.txt
-rw-rw-r--  1 stelo stelo     2334923 Aug 12 13:55 cowpea.debug
-rw-rw-r--  1 stelo stelo    23932878 Aug 19 16:20 cowpeademo.G00.graphml
-rw-rw-r--  1 stelo stelo    21227197 Aug 20 14:30 cowpeademo.G0.graphml
-rw-rw-r--  1 stelo stelo     4032589 Aug 20 14:30 cowpeademo.G1.graphml
-rw-rw-r--  1 stelo stelo      786102 Aug 20 14:30 cowpeademo.G2c.graphml
-rw-rw-r--  1 stelo stelo     4032589 Aug 20 14:30 cowpeademo.G2.graphml
-rw-rw-r--  1 stelo stelo      690132 Aug 20 14:30 cowpeademo.G2s.graphml
-rw-rw-r--  1 stelo stelo      785247 Aug 20 14:30 cowpeademo.Gc.graphml
-rw-rw-r--  1 stelo stelo      689269 Aug 20 14:30 cowpeademo.Gs.graphml
-rw-rw-r--  1 stelo stelo     1949816 Aug 12 13:55 cowpea.edges.1
-rw-rw-r--  1 stelo stelo     1980410 Aug 12 13:55 cowpea.edges.2
-rw-rw-r--  1 stelo stelo     2813482 Aug 12 13:55 cowpea.edges.greedy
-rw-rw-r--  1 stelo stelo     2895764 Aug 12 13:55 cowpea.edges.hinges
-rw-rw-r--  1 stelo stelo     2303732 Aug 12 13:55 cowpea.edges.hinges2
-rw-rw-r--  1 stelo stelo    77029959 Aug 12 13:55 cowpea.edges.skipped
-rw-rw-r--  1 stelo stelo           0 Aug 11 23:34 cowpea.filtered.fasta
-rw-rw-r--  1 stelo stelo     3394678 Aug 12 10:47 cowpea.garbage.txt
-rw-rw-r--  1 stelo stelo   873925812 Aug 12 13:55 cowpea.hgraph
-rw-rw-r--  1 stelo stelo     1092523 Aug 12 13:53 cowpea.hinge.list
-rw-rw-r--  1 stelo stelo    16544497 Aug 12 08:16 cowpea.hinges.txt
-rw-rw-r--  1 stelo stelo           0 Aug 11 23:34 cowpea.homologous.txt
-rw-rw-r--  1 stelo stelo    30272352 Aug  9 18:22 .cowpea.idx
-rw-rw-r--  1 stelo stelo     8620027 Aug 12 13:16 cowpea.killed.hinges
-rw-rw-r--  1 stelo stelo 93056878860 Aug 11 21:44 cowpea.las
-rw-rw-r--  1 stelo stelo     9677331 Aug 12 01:16 cowpea.mas
-rw-rw-r--  1 stelo stelo      214853 Aug 12 13:16 cowpea.max
-rw-rw-r--  1 stelo stelo     5625208 Aug 11 21:55 .cowpea.qual.anno
-rw-rw-r--  1 stelo stelo    60451270 Aug 11 21:55 .cowpea.qual.data
-rw-rw-r--  1 stelo stelo    19650442 Aug 12 08:16 cowpea.repeat.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 01:16 debug.txt
-rw-rw-r--  1 stelo stelo    90159028 Aug 12 13:16 edges.bkw.backup.txt
-rw-rw-r--  1 stelo stelo   101888561 Aug 12 13:16 edges.fwd.backup.txt
-rw-rw-r--  1 stelo stelo     2511184 Aug 12 13:55 edges.g_out.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 13:53 hinge_debug.txt
drwxrwxr-x  2 stelo stelo        4096 Aug 12 10:37 log/
-rw-rw-r--  1 stelo stelo    76574278 Aug  9 08:07 map.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 13:17 overlap_debug.txt
-rw-rw-r--  1 stelo stelo  1506184035 Aug  9 08:10 .cowpea.bps
-rw-rw-r--  1 stelo stelo     2416756 Aug 12 13:16 cowpea.contained.txt
-rw-rw-r--  1 stelo stelo   957355832 Aug 12 01:16 cowpea.coverage.txt
-rw-rw-r--  1 stelo stelo         468 Aug  9 18:22 cowpea.db
-rw-rw-r--  1 stelo stelo     1093924 Aug 12 13:53 cowpea.deadends.txt
-rw-rw-r--  1 stelo stelo     2334923 Aug 12 13:55 cowpea.debug
-rw-rw-r--  1 stelo stelo    23932878 Aug 19 16:20 cowpeademo.G00.graphml
-rw-rw-r--  1 stelo stelo    21227197 Aug 20 14:30 cowpeademo.G0.graphml
-rw-rw-r--  1 stelo stelo     4032589 Aug 20 14:30 cowpeademo.G1.graphml
-rw-rw-r--  1 stelo stelo      786102 Aug 20 14:30 cowpeademo.G2c.graphml
-rw-rw-r--  1 stelo stelo     4032589 Aug 20 14:30 cowpeademo.G2.graphml
-rw-rw-r--  1 stelo stelo      690132 Aug 20 14:30 cowpeademo.G2s.graphml
-rw-rw-r--  1 stelo stelo      785247 Aug 20 14:30 cowpeademo.Gc.graphml
-rw-rw-r--  1 stelo stelo      689269 Aug 20 14:30 cowpeademo.Gs.graphml
-rw-rw-r--  1 stelo stelo     1949816 Aug 12 13:55 cowpea.edges.1
-rw-rw-r--  1 stelo stelo     1980410 Aug 12 13:55 cowpea.edges.2
-rw-rw-r--  1 stelo stelo     2813482 Aug 12 13:55 cowpea.edges.greedy
-rw-rw-r--  1 stelo stelo     2895764 Aug 12 13:55 cowpea.edges.hinges
-rw-rw-r--  1 stelo stelo     2303732 Aug 12 13:55 cowpea.edges.hinges2
-rw-rw-r--  1 stelo stelo    77029959 Aug 12 13:55 cowpea.edges.skipped
-rw-rw-r--  1 stelo stelo           0 Aug 11 23:34 cowpea.filtered.fasta
-rw-rw-r--  1 stelo stelo     3394678 Aug 12 10:47 cowpea.garbage.txt
-rw-rw-r--  1 stelo stelo   873925812 Aug 12 13:55 cowpea.hgraph
-rw-rw-r--  1 stelo stelo     1092523 Aug 12 13:53 cowpea.hinge.list
-rw-rw-r--  1 stelo stelo    16544497 Aug 12 08:16 cowpea.hinges.txt
-rw-rw-r--  1 stelo stelo           0 Aug 11 23:34 cowpea.homologous.txt
-rw-rw-r--  1 stelo stelo    30272352 Aug  9 18:22 .cowpea.idx
-rw-rw-r--  1 stelo stelo     8620027 Aug 12 13:16 cowpea.killed.hinges
-rw-rw-r--  1 stelo stelo 93056878860 Aug 11 21:44 cowpea.las
-rw-rw-r--  1 stelo stelo     9677331 Aug 12 01:16 cowpea.mas
-rw-rw-r--  1 stelo stelo      214853 Aug 12 13:16 cowpea.max
-rw-rw-r--  1 stelo stelo     5625208 Aug 11 21:55 .cowpea.qual.anno
-rw-rw-r--  1 stelo stelo    60451270 Aug 11 21:55 .cowpea.qual.data
-rw-rw-r--  1 stelo stelo    19650442 Aug 12 08:16 cowpea.repeat.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 01:16 debug.txt
-rw-rw-r--  1 stelo stelo    90159028 Aug 12 13:16 edges.bkw.backup.txt
-rw-rw-r--  1 stelo stelo   101888561 Aug 12 13:16 edges.fwd.backup.txt
-rw-rw-r--  1 stelo stelo     2511184 Aug 12 13:55 edges.g_out.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 13:53 hinge_debug.txt
drwxrwxr-x  2 stelo stelo        4096 Aug 12 10:37 log/
-rw-rw-r--  1 stelo stelo    76574278 Aug  9 08:07 map.txt
-rw-rw-r--  1 stelo stelo           0 Aug 12 13:17 overlap_debug.txt

StefanoLonardi commented 8 years ago

I forgot

stelo@H4:~/HINGE$ git show
commit 513880ada65de86424fb92b89a8c452d3ab2a32a
Merge: 33d4940 62d5fec
Author: ilanshom <ilanshom@gmail.com>
Date:   Wed Aug 17 00:25:01 2016 -0700

    Merge pull request #56 from fxia22/dev

    Dev

ilanshom commented 8 years ago

Hi Stefano,

We haven't been able to reproduce that error here. Would it be possible for you to send us the file cowpeademo.G2.graphml for us to debug that? The graphml file doesn't contain the sequence information, so you wouldn't be sharing any data (in case that's a concern for you).

Thanks.

StefanoLonardi commented 8 years ago

Ok, let me see whether I can attach it here. cowpeademo.G2.graphml.gz

agroppi commented 8 years ago

Hi I had the same error. I think I have solved my problem by updating daligner and HINGE. At least the step using HPC.aligner is now working without problem.

Best

ilanshom commented 8 years ago

Stefano, we've updated the get_draft_path script, and I believe the new one should work on your cowpea data. Could you please try it and let us know what happens? Thanks.

StefanoLonardi commented 8 years ago

It worked. At the end it printed a bunch of numbers, but it did not fail. I am now running draft_assembly, hopefully I can get to the end 👍

HingeAssembler / HINGE

DALIGNER seg fault #54