search

vgteam / vg

tools for working with genome variation graphs
https://biostars.org/tag/vg/
Other
1.1k stars 195 forks source link

vg find command to get subgraphs of genomic regions failed #2585

Open JYLeeBioinfo opened 4 years ago

JYLeeBioinfo commented 4 years ago

1. What were you trying to do? I was trying to do extract a subgraph of a certain genome region.

The original graph I want to extract the subgrpahs from is hg38 with alternative paths from SV call VCF files.

vg find -x N_sorted_chr1-1mb.xg -E chr1:998639-998887 > individual_svs/N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg 2> individual_svs/N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg.err

2. What did you want to happen? I wanted to get a vg file

3. What actually happened? The following error occurred and empty vg file was generated

terminate called after throwing an instance of 'std::runtime_error'
  what():  Attempted to get handle for node 30 not present in graph
ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug.
Stack trace path: /tmp/vg_crash_JnUw8A/stacktrace.txt

One thing strange is that not all regions I tested failed. I tested 14 regions and 13 regions failed and 1 region succeeded.

4. If you got a line like Stack trace path: /somewhere/on/your/computer/stacktrace.txt, please copy-paste the contents of that file here:

all 13 regions that failed gave me the following error message

#13   Object "", at 0xffffffffffffffff, in
#12   Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x785150, in _start
#11   Object "/lib64/libc-2.17.so", at 0x363e821b44, in __libc_start_main
#10   Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x6ca530, in main
      Source "src/main.cpp", line 74, in main [0x6ca530]
#9    Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0xc75817, in vg::subcommand::Subcommand::operator()(int, char**) const
    | Source "src/subcommand/subcommand.cpp", line 72, in operator()
      Source "/appl/compiler/gcc/7.4.0/include/c++/7.4.0/bits/std_function.h", line 706, in operator() [0xc75817]
        703:     {
        704:       if (_M_empty())
        705:    __throw_bad_function_call();
      > 706:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
        707:     }
        708:
        709: #if __cpp_rtti
#8    Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0xbc7922, in main_find(int, char**)
      Source "src/subcommand/find_main.cpp", line 588, in main_find [0xbc7922]
#7    Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x106c785, in vg::algorithms::extract_id_range(handlegraph::HandleGraph const&, long const&, long const&, handlegraph::MutableHandleGraph&)
      Source "src/algorithms/subgraph.cpp", line 267, in extract_id_range [0x106c785]
#6    Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x157b82c, in xg::XG::get_handle(long const&, bool) const
      Source "deps/xg/src/xg.cpp", line 1580, in get_handle [0x157b82c]
#5    Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc53802, in __cxa_throw
      Source "../../../../libstdc++-v3/libsupc++/eh_throw.cc", line 93, in __cxa_throw [0x7feeddc53802]
#4    Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc535c0, in std::terminate()
      Source "../../../../libstdc++-v3/libsupc++/eh_terminate.cc", line 57, in terminate [0x7feeddc535c0]
#3    Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc53575, in __cxxabiv1::__terminate(void (*)())
      Source "../../../../libstdc++-v3/libsupc++/eh_terminate.cc", line 47, in __cxa_begin_catch [0x7feeddc53575]
#2    Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc55794, in __gnu_cxx::__verbose_terminate_handler()
      Source "../../../../libstdc++-v3/libsupc++/vterminate.cc", line 95, in __verbose_terminate_handler [0x7feeddc55794]
#1    Object "/lib64/libc-2.17.so", at 0x363e8370e7, in abort
#0    Object "/lib64/libc-2.17.so", at 0x363e8359d9, in raise

5. What data and command can the vg dev team use to make the problem happen? data : https://drive.google.com/open?id=1x1VbO2zcPhRjOVPIEL2W-UIXbOOIKfyw This is vg and xg I generated using Momig-tools vcf2xg.sh

command

vg find -x N_sorted_chr1-1mb.xg -E chr1:998639-998887 > N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg

6. What does running vg version say?

vg version v1.19.0 "Tramutola"
Compiled with g++ (GCC) 7.4.0 on Linux
Linked against libstd++ 20181206
Built by root@node10
ekg commented 4 years ago

You need to specify -p [REGION] -E

-E is just an optimal flag.

-p must be given the region specification.

On Thu, Jan 2, 2020, 08:19 hd00ljy notifications@github.com wrote:

1. What were you trying to do? I was trying to do extract a subgraph of a certain genome region.

The original graph I want to extract the subgrpahs from is hg38 with alternative paths from SV call VCF files.

vg find -x N_sorted_chr1-1mb.xg -E chr1:998639-998887 > individual_svs/N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg 2> individual_svs/N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg.err

2. What did you want to happen? I wanted to get a vg file

3. What actually happened? The following error occurred and empty vg file was generated

terminate called after throwing an instance of 'std::runtime_error' what(): Attempted to get handle for node 30 not present in graph ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug. Stack trace path: /tmp/vg_crash_JnUw8A/stacktrace.txt

One thing strange is that not all regions I tested failed. I tested 14 regions and 13 regions failed and 1 region succeeded.

4. If you got a line like Stack trace path: /somewhere/on/your/computer/stacktrace.txt, please copy-paste the contents of that file here:

all 13 regions that failed gave me the following error message

13 Object "", at 0xffffffffffffffff, in

12 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x785150, in _start

11 Object "/lib64/libc-2.17.so", at 0x363e821b44, in __libc_start_main

10 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x6ca530, in main

  Source "src/main.cpp", line 74, in main [0x6ca530]

9 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0xc75817, in vg::subcommand::Subcommand::operator()(int, char**) const

| Source "src/subcommand/subcommand.cpp", line 72, in operator()
  Source "/appl/compiler/gcc/7.4.0/include/c++/7.4.0/bits/std_function.h", line 706, in operator() [0xc75817]
    703:     {
    704:       if (_M_empty())
    705:    __throw_bad_function_call();
  > 706:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
    707:     }
    708:
    709: #if __cpp_rtti

8 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0xbc7922, in main_find(int, char**)

  Source "src/subcommand/find_main.cpp", line 588, in main_find [0xbc7922]

7 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x106c785, in vg::algorithms::extract_id_range(handlegraph::HandleGraph const&, long const&, long const&, handlegraph::MutableHandleGraph&)

  Source "src/algorithms/subgraph.cpp", line 267, in extract_id_range [0x106c785]

6 Object "/appl/compiler/gcc/7.4.0/applib1/vg/1.19.0/bin/vg", at 0x157b82c, in xg::XG::get_handle(long const&, bool) const

  Source "deps/xg/src/xg.cpp", line 1580, in get_handle [0x157b82c]

5 Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc53802, in __cxa_throw

  Source "../../../../libstdc++-v3/libsupc++/eh_throw.cc", line 93, in __cxa_throw [0x7feeddc53802]

4 Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc535c0, in std::terminate()

  Source "../../../../libstdc++-v3/libsupc++/eh_terminate.cc", line 57, in terminate [0x7feeddc535c0]

3 Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc53575, in cxxabiv1::terminate(void (*)())

  Source "../../../../libstdc++-v3/libsupc++/eh_terminate.cc", line 47, in __cxa_begin_catch [0x7feeddc53575]

2 Object "/appl/compiler/gcc/7.4.0/lib64/libstdc++.so.6.0.24", at 0x7feeddc55794, in __gnu_cxx::__verbose_terminate_handler()

  Source "../../../../libstdc++-v3/libsupc++/vterminate.cc", line 95, in __verbose_terminate_handler [0x7feeddc55794]

1 Object "/lib64/libc-2.17.so", at 0x363e8370e7, in abort

0 Object "/lib64/libc-2.17.so", at 0x363e8359d9, in raise

5. What data and command can the vg dev team use to make the problem happen? data : https://drive.google.com/open?id=1x1VbO2zcPhRjOVPIEL2W-UIXbOOIKfyw

command

vg find -x N_sorted_chr1-1mb.xg -E chr1:998639-998887 > N_sorted_chr1-1mb_chr1-998639-998887__12_1.vg

6. What does running vg version say?

vg version v1.19.0 "Tramutola" Compiled with g++ (GCC) 7.4.0 on Linux Linked against libstd++ 20181206 Built by root@node10

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vgteam/vg/issues/2585?email_source=notifications&email_token=AABDQEMTSX2TRGTH5GGDUX3Q3WIRBA5CNFSM4KB7JZ4KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IDT7LBQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEIBJS7XZV2LNHQULZLQ3WIRBANCNFSM4KB7JZ4A .

JYLeeBioinfo commented 4 years ago

Hi, it seems the options have changed when vg moved from version 1.19 to 1.20

I am currently using vg1.19.0 and this is the help message I get

    -p, --path TARGET      find the node(s) in the specified path range(s) TARGET=path[:pos1[-pos2]]
    -E, --path-dag TARGET  like -p, but gets any node in the partial order from pos1 to pos2, assumes id sorted DAG

I have tried for hours to install the vg1.20.0 but no success

It's quite hard for me to update VG since the server OS is CentOS 6.10

ekg commented 4 years ago

Does the v1.20 binary not work for you?

On Thu, Jan 2, 2020, 15:48 hd00ljy notifications@github.com wrote:

Hi, it seems the options have changed when vg moved from version 1.19 to 1.20

I am currently using vg1.19.0 and this is the help message I get

-p, --path TARGET      find the node(s) in the specified path range(s) TARGET=path[:pos1[-pos2]]
-E, --path-dag TARGET  like -p, but gets any node in the partial order from pos1 to pos2, assumes id sorted DAG

I have tried for hours to install the vg1.20.0 but no success

It's quite hard for me to update VG since the server OS is CentOS 6.10

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vgteam/vg/issues/2585?email_source=notifications&email_token=AABDQEPKQPLXFIMSHOTVAWTQ3X5CZA5CNFSM4KB7JZ4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH6QDBI#issuecomment-570229125, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEIKPCDIG5EXGROALQDQ3X5CZANCNFSM4KB7JZ4A .

JYLeeBioinfo commented 4 years ago

Sadly, it does not work...

Kernel too old error for both 1.19 and 1.20 binaries.

So I'm trying to build 1.20 from source

ekg commented 4 years ago

I assume you don't have access to docker on this system.

On Thu, Jan 2, 2020, 17:54 hd00ljy notifications@github.com wrote:

Sadly, it does not work...

Kernel too old error for both 1.19 and 1.20 binaries.

So I'm trying to build 1.20 from source

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vgteam/vg/issues/2585?email_source=notifications&email_token=AABDQEP2465QKSMBT3JIGMDQ3YL25A5CNFSM4KB7JZ4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH6ZT5A#issuecomment-570268148, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEJZE6WODSDOHEYO3ULQ3YL25ANCNFSM4KB7JZ4A .

JYLeeBioinfo commented 4 years ago

I tried docker but "kernel too old" message also arose within the containers made from CentOS 6, 7 and ubuntu18.04 images.

JYLeeBioinfo commented 4 years ago

I succeeded in installing vg1.20 but still got the same error.

It seems the xg and vg I built using Momig-tools are causing the error.

So, I tried building vg file using vg construct with vg1.20, but got the another error :dizzy_face:

command

vg construct -r /data/REFERENCE/hg38/Homo_sapiens_assembly38.fasta -v N_sorted_chr1-1mb.vcf  -a  -t 28 --handle-sv --flat-alts -p \
 > N_sorted_chr1-1mb.vcf.vg-construct.vg 2> N_sorted_chr1-1mb.vcf.vg-construct.vg.err

error

vg: src/Variant.cpp:349: bool vcflib::Variant::canonicalize(FastaReference&, std::vector<FastaReference*>, bool, int): Assertion `canonicalizable()' failed.
ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug.
Stack trace path: /tmp/vg_crash_VkQ0Ok/stacktrace.txt
Please include the stack trace file in your bug report!

trace

Crash report for vg v1.20.0-138-gf275f10 "Ginestra"
Stack trace (most recent call last):
#11   Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x794c80, in _start
#10   Object "/lib64/libc-2.12.so", at 0x7f61a0356d1f, in __libc_start_main
#9    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x6d9c60, in main
      Source "src/main.cpp", line 75, in main [0x6d9c60]
#8    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xc900c7, in vg::subcommand::Subcommand::operator()(
int, char**) const
    | Source "src/subcommand/subcommand.cpp", line 72, in operator()
      Source "/usr/local/include/c++/7.4.0/bits/std_function.h", line 706, in operator() [0xc900c7]
        703:     {
        704:       if (_M_empty())
        705:    __throw_bad_function_call();
      > 706:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
        707:     }
        708:
        709: #if __cpp_rtti
#7    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xbc2ce3, in main_construct(int, char**)
      Source "src/subcommand/construct_main.cpp", line 347, in main_construct [0xbc2ce3]
#6    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xd556d3, in vg::Constructor::construct_graph(std::v
ector<FastaReference*, std::allocator<FastaReference*> > const&, std::vector<vcflib::VariantCallFile*, std::allocator<vcflib::VariantCallFile*> > const&, std::vec
tor<FastaReference*, std::allocator<FastaReference*> > const&, std::function<void (vg::Graph&)>)
      Source "src/constructor.cpp", line 2099, in construct_graph [0xd556d3]
#5    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xd53cce, in vg::Constructor::construct_graph(std::_
_cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, FastaReference&, vg::VcfBuffer&, std::vector<FastaReference*, std::allocator<FastaRefer
ence*> > const&, std::function<void (vg::Graph&)>)
      Source "src/constructor.cpp", line 1794, in construct_graph [0xd53cce]
#4    Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x1119167, in vcflib::Variant::canonicalize(FastaRef
erence&, std::vector<FastaReference*, std::allocator<FastaReference*> >, bool, int)
#3    Object "/lib64/libc-2.12.so", at 0x7f61a036372f, in __assert_fail
#2    Object "/lib64/libc-2.12.so", at 0x7f61a036366d, in __assert_fail_base
#1    Object "/lib64/libc-2.12.so", at 0x7f61a036bcd4, in abort
#0    Object "/lib64/libc-2.12.so", at 0x7f61a036a4f5, in raise
ekg commented 4 years ago

With docker you should be able to build from source inside the docker image.

This error suggests that you have something strange in your VCF file.

What kinds of variants are in there?

On Fri, Jan 3, 2020, 05:22 hd00ljy notifications@github.com wrote:

I succeeded in installing vg1.20 but still got the same error.

It seems the xg and vg I built using Momig-tools are causing the error.

So, I tried building vg file using vg construct with vg1.20, but got the another error 😵

command

vg construct -r /data/REFERENCE/hg38/Homo_sapiens_assembly38.fasta -v N_sorted_chr1-1mb.vcf -a -t 28 --handle-sv --flat-alts -p \

N_sorted_chr1-1mb.vcf.vg-construct.vg 2> N_sorted_chr1-1mb.vcf.vg-construct.vg.err

error

vg: src/Variant.cpp:349: bool vcflib::Variant::canonicalize(FastaReference&, std::vector<FastaReference*>, bool, int): Assertion `canonicalizable()' failed.

ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug.

Stack trace path: /tmp/vg_crash_VkQ0Ok/stacktrace.txt

Please include the stack trace file in your bug report!

trace

Crash report for vg v1.20.0-138-gf275f10 "Ginestra"

Stack trace (most recent call last):

11 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x794c80, in _start

10 Object "/lib64/libc-2.12.so", at 0x7f61a0356d1f, in __libc_start_main

9 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x6d9c60, in main

  Source "src/main.cpp", line 75, in main [0x6d9c60]

8 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xc900c7, in vg::subcommand::Subcommand::operator()(

int, char**) const

| Source "src/subcommand/subcommand.cpp", line 72, in operator()

  Source "/usr/local/include/c++/7.4.0/bits/std_function.h", line 706, in operator() [0xc900c7]

    703:     {

    704:       if (_M_empty())

    705:    __throw_bad_function_call();

  > 706:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);

    707:     }

    708:

    709: #if __cpp_rtti

7 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xbc2ce3, in main_construct(int, char**)

  Source "src/subcommand/construct_main.cpp", line 347, in main_construct [0xbc2ce3]

6 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xd556d3, in vg::Constructor::construct_graph(std::v

ector<FastaReference, std::allocator<FastaReference> > const&, std::vector<vcflib::VariantCallFile, std::allocator<vcflib::VariantCallFile> > const&, std::vec

tor<FastaReference, std::allocator<FastaReference> > const&, std::function<void (vg::Graph&)>)

  Source "src/constructor.cpp", line 2099, in construct_graph [0xd556d3]

5 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0xd53cce, in vg::Constructor::constructgraph(std::

_cxx11::basic_string<char, std::char_traits, std::allocator >, FastaReference&, vg::VcfBuffer&, std::vector<FastaReference*, std::allocator<FastaRefer

ence*> > const&, std::function<void (vg::Graph&)>)

  Source "src/constructor.cpp", line 1794, in construct_graph [0xd53cce]

4 Object "/data2/hd00ljy/data_processing/PMI_Nanopore_WGS/tools_new/11_graph_genome_docker/vg/bin/vg", at 0x1119167, in vcflib::Variant::canonicalize(FastaRef

erence&, std::vector<FastaReference, std::allocator<FastaReference> >, bool, int)

3 Object "/lib64/libc-2.12.so", at 0x7f61a036372f, in __assert_fail

2 Object "/lib64/libc-2.12.so", at 0x7f61a036366d, in __assert_fail_base

1 Object "/lib64/libc-2.12.so", at 0x7f61a036bcd4, in abort

0 Object "/lib64/libc-2.12.so", at 0x7f61a036a4f5, in raise

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vgteam/vg/issues/2585?email_source=notifications&email_token=AABDQELDYBZND25AQ64QWKLQ324RTA5CNFSM4KB7JZ4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIAHZ7A#issuecomment-570457340, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEN3KGOAJCUDYI6RJSLQ324RTANCNFSM4KB7JZ4A .

JYLeeBioinfo commented 4 years ago

I have long-read based SV calls from sniffles SV caller

INS,DEL,TRA,DUP,INV,INVDUP and etc..

For some variants, alt allele is represented like ,