search

vgteam / vg

tools for working with genome variation graphs
https://biostars.org/tag/vg/
Other
1.12k stars 194 forks source link

Giraffe construct the graph and indexes failure #3919

Open Yufeng98 opened 1 year ago

Yufeng98 commented 1 year ago

1. What were you trying to do? Build the graph and indexes for Giraffe mapping.

2. What did you want to happen? Generate .dist, .giraffe.gdz, *.min files.

3. What actually happened?

[vg autoindex] Executing command: /data3/yufenggu/vg/bin/vg autoindex --workflow giraffe -r chr22.fa -v chr22.vcf.gz -p chr22                                                                                 
[IndexRegistry]: Checking for phasing in VCF(s).                                                                                                                                                              
[IndexRegistry]: Chunking inputs for parallelism.                                                                                                                                                             
[IndexRegistry]: Constructing VG graph from FASTA and VCF input.                                                                                                                                              
warning:[vg::Constructor] Unsupported IUPAC ambiguity codes found in 22; coercing to N.                                                                                                                       
[IndexRegistry]: Constructing GBWT from VG graph and phased VCF input.                                                                                                                                        
[IndexRegistry]: Stripping allele paths from VG.                                                                                                                                                              
[IndexRegistry]: Constructing XG graph from VG graph.                                                                                                                                                         
[IndexRegistry]: Downsampling full GBWT.                                                                                                                                                                      
[IndexRegistry] forked child 630417                                                                                                                                                                           
[IndexRegistry]: Constructing GBZ.                                                                                                                                                                            
[IndexRegistry]: Constructing distance index for Giraffe.                                                                                                                                                     
terminate called after throwing an instance of 'std::invalid_argument'                                                                                                                                        
  what():  Need 27 bits to represent value 69942624 but only have 26                                                                                                                                          
━━━━━━━━━━━━━━━━━━━━                                                                                                                                                                                          
Crash report for vg v1.47.0-34-g6b5b805ce "Ostuni"                                                                                                                                                            
Stack trace (most recent call last):                                                                                                                                                                          
#21   Object "", at 0xffffffffffffffff, in                                                                                                                                                                    
#20   Object "/data3/yufenggu/vg/bin/vg", at 0x55f446a2a554, in _start                                                                                                                                        
#19   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f28f1ed9e3f, in __libc_start_main@@GLIBC_2.34                                                                                                       
      Source "../csu/libc-start.c", line 392, in __libc_start_main_impl [0x7f28f1ed9e3f]                                                                                                                      
#18   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f28f1ed9d8f, in __libc_start_call_main
      Source "../sysdeps/nptl/libc_start_call_main.h", line 58, in __libc_start_call_main [0x7f28f1ed9d8f]                                                                                                    
#17   Object "/data3/yufenggu/vg/bin/vg", at 0x55f4469fd83e, in main                                                                                                                                          
      Source "src/main.cpp", line 88, in main [0x55f4469fd83e]                                                                                                                                                
#16   Object "/data3/yufenggu/vg/bin/vg", at 0x55f44718798b, in vg::subcommand::Subcommand::operator()(int, char**) const                                                                                         | Source "src/subcommand/subcommand.cpp", line 75, in operator()                                                                                                                                          
      Source "/usr/include/c++/9/bits/std_function.h", line 688, in operator() [0x55f44718798b]                                                                                                               
        685:     {                                                                                                                                                                                            
        686:       if (_M_empty())                                                                                                                                                                            
        687:    __throw_bad_function_call();                                                                                                                                                                  
      > 688:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);                                                                                                                         
        689:     }                                                                                                                                                                                            
        690:                                                                                                                                                                                                  
        691: #if __cpp_rtti                                                                                                                                                                                   
#15   Object "/data3/yufenggu/vg/bin/vg", at 0x55f44703a06c, in main_autoindex(int, char**)                                                                                                                   
      Source "src/subcommand/autoindex_main.cpp", line 355, in main_autoindex [0x55f44703a06c]                                                                                                                
#14   Object "/data3/yufenggu/vg/bin/vg", at 0x55f4474153ce, in vg::IndexRegistry::make_indexes(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<st
d::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)                                                                                                                     
      Source "src/index_registry.cpp", line 4012, in make_indexes [0x55f4474153ce]                                                                                                                            
#13   Object "/data3/yufenggu/vg/bin/vg", at 0x55f4473ffe89, in vg::IndexRegistry::execute_recipe(std::pair<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::les
s<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, unsigned long> const&
, vg::IndexingPlan const*, vg::AliasGraph&)                                                                                                                                                                   
    | Source "src/index_registry.cpp", line 4963, in execute                                                                                                                                                  
    | Source "src/index_registry.cpp", line 5128, in operator()                                                                                                                                               
      Source "/usr/include/c++/9/bits/std_function.h", line 688, in execute_recipe [0x55f4473ffe89]                                                                                                           
        685:     {                                                                                                                                                                                            
        686:       if (_M_empty())                                                                                                                                                                            
        687:    __throw_bad_function_call();                                                                                                                                                                  
      > 688:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
        689:     }                                                                                                                                                                                            
        690:                                                                                                                                                                                                  
        691: #if __cpp_rtti                                                                                                                                                                                   
#12   Object "/data3/yufenggu/vg/bin/vg", at 0x55f44741a88d, in std::_Function_handler<std::vector<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::allocator<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocat
or<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > (std::vector<vg::IndexFile const*, std::allocator<vg::IndexFile const*> > const&, vg::IndexingPlan const*, vg::Alia
sGraph&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator
<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&), vg::VGIndexes::get_vg_index_registry()::{lambda(std::vector<vg::IndexFile const*, std::allocator<vg::IndexFile c
onst*> > const&, vg::IndexingPlan const*, vg::AliasGraph&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_trai
ts<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)#53}>::_M_invoke(std::_Any_data const&, std::vector<vg::IndexFil
e const*, std::allocator<vg::IndexFile const*> > const&, vg::IndexingPlan const*&&, vg::AliasGraph&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::
__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)                      
      Source "/usr/include/c++/9/bits/std_function.h", line 286, in _M_invoke [0x55f44741a88d]                                                                                                                
        283:       _M_invoke(const _Any_data& __functor, _ArgTypes&&... __args)                                                                                                                               
        284:       {                                                                                                                                                                                          
        285:    return (*_Base::_M_get_pointer(__functor))(                                                                                                                                                   
      > 286:        std::forward<_ArgTypes>(__args)...);                                                                                                                                                      
        287:       }                                                                                                                                                                                          
        288:     };                                                                                                                                                                                           
#11   Object "/data3/yufenggu/vg/bin/vg", at 0x55f44741a754, in vg::VGIndexes::get_vg_index_registry()::{lambda(std::vector<vg::IndexFile const*, std::allocator<vg::IndexFile const*> > const&, vg::IndexingP
lan const*, vg::AliasGraph&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
 > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)#53}::operator()(std::vector<vg::IndexFile const*, std::allocator<vg::IndexFile const*> > cons
t&, vg::IndexingPlan const*, vg::AliasGraph&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std
::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const [clone .isra.0]                                                       
      Source "src/index_registry.cpp", line 3577, in operator() [0x55f44741a754]                                                                                                                              
#10   Object "/data3/yufenggu/vg/bin/vg", at 0x55f44741a579, in vg::VGIndexes::get_vg_index_registry()::{lambda(handlegraph::HandleGraph const&, vg::IndexingPlan const*, std::set<std::__cxx11::basic_string<
char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_t
raits<char>, std::allocator<char> > > > const&)#52}::operator()(handlegraph::HandleGraph const&, vg::IndexingPlan const*, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) 
const [clone .isra.0]                                                                                                                                                                                         
      Source "src/index_registry.cpp", line 3552, in operator() [0x55f44741a579]                                                                                                                              
#9    Object "/data3/yufenggu/vg/bin/vg", at 0x55f44763e84a, in vg::fill_in_distance_index(bdsg::SnarlDistanceIndex*, handlegraph::HandleGraph const*, vg::HandleGraphSnarlFinder const*, unsigned long)            Source "src/snarl_distance_index.cpp", line 37, in fill_in_distance_index [0x55f44763e84a]                                                                                                              
#8    Object "/data3/yufenggu/vg/bin/vg", at 0x55f447b6324c, in bdsg::SnarlDistanceIndex::get_snarl_tree_records(std::vector<bdsg::SnarlDistanceIndex::TemporaryDistanceIndex const*, std::allocator<bdsg::Sna
rlDistanceIndex::TemporaryDistanceIndex const*> > const&, handlegraph::HandleGraph const*)                                                                                                                    
    | Source "bdsg/src/snarl_distance_index.cpp", line 5992, in set_last_child_offset                                                                                                                         
    | Source "bdsg/src/snarl_distance_index.cpp", line 5050, in operator=                                                                                                                                     
      Source "bdsg/include/bdsg/internal/mapped_structs.hpp", line 1846, in get_snarl_tree_records [0x55f447b6324c]                                                                                           
#7    Object "/data3/yufenggu/vg/bin/vg", at 0x55f447afc086, in bdsg::CompatIntVector<bdsg::yomo::Allocator<unsigned long> >::pack(unsigned long, unsigned long, unsigned long)                               
      Source "bdsg/include/bdsg/internal/mapped_structs.hpp", line 1813, in pack [0x55f447afc086]                                                                                                             
#6    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7f28f21f0517, in __cxa_throw                                                                                                               
#5    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7f28f21f02b6, in std::terminate()                                                                                                          
#4    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7f28f21f024b, in                                                                                                                           
#3    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7f28f21e4bbd, in                                                                                                                           
#2    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f28f1ed87f2, in abort                                                                                                                               
      Source "./stdlib/abort.c", line 79, in abort [0x7f28f1ed87f2]                                                                                                                                           
#1    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f28f1ef2475, in raise                                                                                                                               
      Source "../sysdeps/posix/raise.c", line 26, in raise [0x7f28f1ef2475]                                                                                                                                   
#0    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f28f1f46a7c, in pthread_kill@@GLIBC_2.34                                                                                                            
    | Source "./nptl/pthread_kill.c", line 89, in __pthread_kill_internal                                                                                                                                     
    | Source "./nptl/pthread_kill.c", line 78, in __pthread_kill_implementation                                                                                                                               
      Source "./nptl/pthread_kill.c", line 44, in __pthread_kill [0x7f28f1f46a7c]                                                                                                                             
ERROR: Signal 6 occurred. VG has crashed. Visit https://github.com/vgteam/vg/issues/new/choose to report a bug.                                                                                               
Please include this entire error log in your bug report!

4. If you got a line like Stack trace path: /somewhere/on/your/computer/stacktrace.txt, please copy-paste the contents of that file here:

Place stacktrace here.

5. What data and command can the vg dev team use to make the problem happen? Reference: chr22.fa extracted from the GRCh38 (extract the reference of chr22 and change the name to 22 to make consistent with the VCF file) downloaded from https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa

VCF: chr22.vcf.gz downloaded from http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/release/20190312_biallelic_SNV_and_INDEL/ALL.chr$i.shapeit2_integrated_snvindels_v2a_27022019.GRCh38.phased.vcf.gz

# command
/data3/yufenggu/vg/bin/vg autoindex --workflow giraffe -r chr22.fa -v chr22.vcf.gz -p chr22

6. What does running vg version say?

vg version v1.47.0-34-g6b5b805ce "Ostuni"
Yufeng98 commented 1 year ago

The test for small/x and 1mb1kgp/z work well.

jeizenga commented 1 year ago

I think this might have been fixed already in this PR: https://github.com/vgteam/vg/issues/3884. Do you get the same error if you build vg from the master branch?

Yufeng98 commented 1 year ago

I got the same error even if building vg from the master branch.