LACHESIS segmentation fault in Ubuntu

[JingaJenga]

When we compile LACHESIS on an Ubuntu system and run it on the provided test case, it crashes with a segmentation fault. The segfault occurs at the opening curly bracket of the function ChromLinkMatrix::ReadFile. Here's the stack trace provided by gdb:

Program received signal SIGSEGV, Segmentation fault.
ChromLinkMatrix::ReadFile (this=0x7ffffff83f60, CLM_file=...)
    at ChromLinkMatrix.cc:280
280              {
 
===============
 
(gdb) i stack
#0  ChromLinkMatrix::ReadFile (this=0x7ffffff83f60, CLM_file=...) at ChromLinkMatrix.cc:280
#1  0x000000000040d980 in ChromLinkMatrix (CLM_file=..., this=0x7ffffff83f60) at ChromLinkMatrix.h:65
#2  LachesisOrdering (run_params=...) at Lachesis.cc:177
#3  0x0000000000407532 in main (argc=2, argv=0x7fffffffe698) at Lachesis.cc:281

If anyone with access to an Ubuntu build wants to help investigate this, please download and install LACHESIS (see README.txt for instructions) and then run it on the test case: Lachesis INIs/test_case.ini.

-- Josh

[congmaowang]

Hi Congmao,

I think we've seen a similar problem with Lachesis. It might be the same as this error: https://github.com/shendurelab/LACHESIS/issues/1

Can you let me know what operating system you're using? Also, instead of talking over e-mail, can we continue this conversation on the LACHESIS GitHub page so that future users will be able to learn from our discussion? Thanks!

-- Josh

On Mon, Feb 17, 2014 at 2:16 PM, Congmao congmao.wang@tuebingen.mpg.de wrote: Dear Josh,

When I test the Lachesis on the test case data, an error of "Segmentation fault" exist, seems a memory problem. This problem both exist in the newest master version as well as the released version from http://shendurelab.github.io/LACHESIS/

The error seems when execute this line in the Lachesis.cc file ChromLinkMatrix clm( run_params._out_dir + "/cached_data/group" + i_str + ".CLM" );

Many thanks for your help! Congmao

[congmaowang]

Hi Josh, Yes, the OS I used is also Ubuntu (12.04), and samtools-0.1.19, boost-1.53.0. The error seems crash with "Segmentation fault" when execute ChromLinkMatrix clm( run_params.outdir + "/cached_data/group" + i_str + ".CLM" ); under Lachesis.cc By the way, which OS you used for Lachesis?

best, Congmao

[JingaJenga]

I developed LACHESIS in GNU/Linux. I've heard reports of it working fine in just about every other kind of Linux except Ubuntu. I wish I could say what's causing the problem, but I've already done some research and came up with nothing.

It's unlikely to be the culprit, but I used samtools v0.1.18 and boost v-1.47.0.

-- Josh

[congmaowang]

Ok, but does it also work for the other debian based linux, such as debian or mint etc? Anyway, I think it should not caused by the samtools or boost, but I will have a try:)

Congmao

[dodecaploid]

Did anyone sort this out? I am also having the same or a similar problem on a Red Hat system. Do I need to build an GNU-linux machine(updated my comment... i wrote Ubuntu, but meant GNU-linux) with the same samtools and boost versions to get this going? Perhaps I'll try. I'd really like to get this going.

Any ideas about whether these warnings might be relevant?

g++ -c ChromLinkMatrix.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 In file included from /usr/lib/gcc/x86_64-redhat-linux/4.4.6/../../../../include/c++/4.4.6/backward/hash_set:60, from /usr/include/boost/graph/adjacency_list.hpp:25, from ChromLinkMatrix.cc:24: /usr/lib/gcc/x86_64-redhat-linux/4.4.6/../../../../include/c++/4.4.6/backward/backward_warning.h:28:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. g++ -c GenomeLinkMatrix.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -c TrueMapping.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -c LinkSizeDistribution.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -c ContigOrdering.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -c ClusterVec.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -c RunParams.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 RunParams.cc: In member function ‘void RunParams::VerifySAMFileHeaders() const’: RunParams.cc:78: warning: ‘N_targets’ may be used uninitialized in this function g++ -c TextFileParsers.cc -Wall -g -O3 -ansi -pedantic -Iinclude -I/programs/samtools-0.1.19 g++ -Wall -g -O3 -ansi -pedantic Lachesis.o Reporter.o ChromLinkMatrix.o GenomeLinkMatrix.o TrueMapping.o LinkSizeDistribution.o ContigOrdering.o ClusterVec.o RunParams.o TextFileParsers.o -o Lachesis -Linclude -lJtime -lJgtools -lJmarkov -L/programs/boost_1_46_1/libs/stage/lib -lboost_system -lboost_filesystem -lboost_regex -L/programs/samtools-0.1.19 -lbam -lz -lpthread

[congmaowang]

Still have no ideas, whether it can work with samtools v0.1.18 and boost v-1.47.0 based on a Red Hat system?. As Josh said now only Ubuntu can't work, it either can't work on a Red Hat system, it is quite important to fix this bug since these two are the most important linux OS.

[fbemm]

I tested lachesis now on three different system (Ubuntu 13.04, CentOS7, openSUSE13.2). I always end up with the same segfault during the ordering step. Would really like to use the package.

[JingaJenga]

Hi,

The current version of Lachesis is well advanced from the publicly available version, with a lower error rate. If you need to run Lachesis, please contact me directly (jnburton@uw.edu) and we can discuss how to scaffold your assembly.

-- Josh

On Tue, Dec 9, 2014 at 10:46 AM, fbemm notifications@github.com wrote:

I tested lachesis now on three different system (Ubuntu 13.04, CentOS7, openSUSE13.2). I always end up with the same segfault during the ordering step. Would really like to use the package.

— Reply to this email directly or view it on GitHub https://github.com/shendurelab/LACHESIS/issues/1#issuecomment-66334605.

[makirc]

Everyone running into this problem should try increasing the stack size limit using ulimit. LACHESIS was developed on systems with 10MB of stack (RHEL6). Fedora, Ubuntu and other systems by default set a limit at 8MB. Increasing this limit made it work on a Fedora for me.

ulimit -s 10240

[nvictus]

I built Lachesis against Boost 1.54.0 on Ubuntu 14.04.1. I managed to evade the segfault by setting ulimit -s unlimited. Now it chokes on computing a MST (running the test case):

Tue Dec  8 20:55:32 2015: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group20.CLM
Tue Dec  8 20:55:32 2015: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group21.CLM
Tue Dec  8 20:55:32 2015: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group22.CLM
Tue Dec  8 20:55:32 2015: Ordering on cluster #0
Tue Dec  8 20:55:32 2015: ChromLinkMatrix::ReadFile   <-  out/test_case/cached_data/group0.CLM      N contigs = 829
Tue Dec  8 20:55:32 2015: Loading contig RE lengths for use in normalization <- out/test_case/cached_data/group0.CLM.RE_sites
Tue Dec  8 20:55:32 2015: MakeTrunkOrder!
min_len_reduced = 15, N_long_contigs = 675
Tue Dec  8 20:55:32 2015: prim_minimum_spanning_tree (iteration #0, previous trunk_size = 0)
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::negative_edge> >'
  what():  The graph may not contain an edge with negative weight.
Aborted

[AmaliT]

Hi

I am having the same issue as @nvictus . We are using boost 1.54, samtools 0.1.18 and have set ulimit to 10240 as suggested. We are running centos6.

Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group15.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group16.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group17.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group18.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group19.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group20.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group21.CLM
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::WriteFile(CLM) -> out/test_case/cached_data/group22.CLM
Thu Jan 21 17:39:09 2016: Ordering on cluster #0
Thu Jan 21 17:39:09 2016: ChromLinkMatrix::ReadFile   <-  out/test_case/cached_data/group0.CLM          N contigs = 829
Thu Jan 21 17:39:09 2016: Loading contig RE lengths for use in normalization <- out/test_case/cached_data/group0.CLM.RE_sites
Thu Jan 21 17:39:09 2016: MakeTrunkOrder!
min_len_reduced = 15, N_long_contigs = 675
Thu Jan 21 17:39:09 2016: prim_minimum_spanning_tree (iteration #0, previous trunk_size = 0)
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::negative_edge> >'
  what():  The graph may not contain an edge with negative weight.
Aborted (core dumped)
[hraaxt@aklppf21:LachesisTest]$ ulimit -s
10240
[hraaxt@aklppf21:LachesisTest]$ env|grep -i boost
LD_LIBRARY_PATH=/software/x86_64/boost_1_54_0/lib:/software/OSutils/git-2.7.0/lib64:/software/x86_64/biobambam/lib/:/software/x86_64/bamtools/lib/:/software/x86_64/lib:/software/x86_64/samtools/src/htslib:/usr/lib64/openmpi/lib/

Any help would be much appreciated.

[eburgueno]

@nvictus I believe we got this fixed by adding -std=c++0x to the CFLAGS on every Makefile, and recompiling LACHESIS again. Can you try on your end to see if this helps?

[rwhetten]

@eburgueno - I encountered the same error as @nvictus, but adding -std=c++0x to the CFLAGS in each Makefile (in the root LACHESIS directory, plus the include/, include/gtools, and include/markov directories) did not solve the problem. My system is Ubuntu 14.04.4, with ulimit set to unlimited. Without the -std=c++0x in the Makefiles, the code compiles with warnings but without errors, but aborts at the MST step as described by @nvictus. Adding -std=c++0x to the Makefiles results in a failure to compile with an error in the include/markov directory related to the HMM.cc file.

 HMM.cc: In member function \u2018bool HMM::AdjustProbsToBaumWelch(const WDAG&)\u2019:
    HMM.cc:406:38: error: call of overloaded     \u2018isnan(__gnu_cxx::__alloc_traits<std::allocator<double> >::value_type&)\u2019 is ambiguous
      if ( isnan( new_trans_probs[S1][S2] ) )
                                          ^
    HMM.cc:406:38: note: candidates are:
    In file included from /usr/include/features.h:374:0,
                 from /usr/include/x86_64-linux-gnu/c++/4.8/bits/os_defines.h:39,
                 from /usr/include/x86_64-linux-gnu/c++/4.8/bits/c++config.h:426,
                 from /usr/include/c++/4.8/bits/stl_algobase.h:59,
                 from /usr/include/c++/4.8/vector:60,
                 from MarkovModel.h:30,
                 from HMM.h:41,
                 from HMM.cc:2:
    /usr/include/x86_64-linux-gnu/bits/mathcalls.h:234:1: note: int isnan(double)
     __MATHDECL_1 (int,isnan,, (_Mdouble_ __value)) __attribute__ ((__const__));
     ^
    In file included from WDAG.h:12:0,
                 from HMM.h:42,
                 from HMM.cc:2:
    /usr/include/c++/4.8/cmath:626:3: note: constexpr bool std::isnan(long double)
       isnan(long double __x)
       ^
    /usr/include/c++/4.8/cmath:622:3: note: constexpr bool std::isnan(double)
       isnan(double __x)
       ^
    /usr/include/c++/4.8/cmath:618:3: note: constexpr bool std::isnan(float)
       isnan(float __x)
       ^
    HMM.cc: In member function \u2018void HMM::DrawPNGAtState(const string&, size_t, size_t) const\u2019:
    HMM.cc:644:24: warning: ignoring return value of \u2018int system(const char*)\u2019, declared with attribute warn_unused_result [-Wunused-result]
   system( cmd.c_str() );
                        ^
    HMM.cc:646:24: warning: ignoring return value of \u2018int system(const char*)\u2019, declared with attribute warn_unused_result [-Wunused-result]
   system( cmd.c_str() );
                        ^

[4learning]

@eburgueno, @rwhetten , I modified the CFLAGS in all Makefile. It is true that HMM.c did not compiled successfully. Then I compiled this one without the modified flag, the rest with the modified flag. Together with the setting of "ulimit -s unlimited", the segmentation fault did not show up any more.

[rwhetten]

@4learning - thanks for the update. When you say "I compiled this one without the modified flag", do you mean everything in the /include/markov directory? As I understand the Makefile in that directory, HMM.cc is compiled together with MarkovChain.cc, MarkovModel.cc, SymbolSet.cc, TestMarkovModel.cc, and WDAG.cc.

[4learning]

@rwhetten sorry for my late reply. Only HMM.cc was compiled without the flag. I moved the compiled results without the flag to a temp folder. Then re-compile everything with the flag. Since only HMM.cc was not compiled successfully, I just copied the files compiled without the flag over. To my surprise, this solved the segmentation fault issue with my datasets.

[congmaowang]

If it works, could it be possible to update the code, which can be easily compiled:)

At 2016-08-09 21:43:24, "4learning" notifications@github.com wrote:

@rwhetten sorry for my late reply. Only HMM.cc was compiled without the flag. I moved the compiled results without the flag to a temp folder. Then re-compile everything with the flag. Since only HMM.cc was not compiled successfully, I just copied the files compiled without the flag over. To my surprise, this solved the segmentation fault issue with my datasets.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[ghuryejay]

Did recompiling the code with modified flags work for anyone? I recompiled everything except markov folder with -std=c++0x. I set ulimit to 10240. I then tried running the Lachesis, and still got this error:

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::negative_edge> >'
  what():  The graph may not contain an edge with negative weight.
Aborted

[robinweide]

@4learning @eburgueno

Could you please elaborate on your methods to get it to work? I still get segfaults when:

- $LACHESIS_BOOST_DIR = /home/robin/bin/boost_1_61_0
- $LACHESIS_SAMTOOLS_DIR = /home/robin/bin/samtools-0.1.18/
- stack size = 10240
- gcc version 6.1.1 20160621 (Red Hat 6.1.1-3) (GCC) 
- added CFLAGS += -std=c++0x to all four Makefiles

Linux version 4.5.5-300.fc24.x86_64 (mockbuild@bkernel01.phx2.fedoraproject.org) (gcc version 6.1.1 20160510 (Red Hat 6.1.1-2) (GCC) ) #1 SMP Thu May 19 13:05:32 UTC 2016

Boost build with: ./bootstrap.sh --prefix=/usr/local ; sudo ./b2 install --with=all

How does this look like for you?

Also, did you do any modifications to the scripts?

Thanks a lot!

[ghuryejay]

I used this fork https://github.com/abelew/LACHESIS and built with boost 1.61. It ran perfectly without seg faulting. I ran it on different data than theirs and it still worked fine.