vg autoindex not work in rpvg module

[ld9866]

Dear developer: When using the module of vg autoindex, we provided the required files as required, but we still kept being reminded that we were missing the required files. May I ask how to solve this problem? Code: vg autoindex --threads 10 --workflow mpmap --workflow rpvg --prefix vg_rpvg --ref-fasta Sscrofall.fa --vcf primates-pg.vcf.gz --gfa primates.gfa --tx-gff ref.gtf Result: [IndexRegistry]: Checking for phasing in VCF(s). [IndexRegistry]: Checking for haplotype lines in GFA. error:[vg autoindex] Input is not sufficient to create indexes Inputs GTF/GFF Reference FASTA Reference GFA w/ Haplotypes VCF are insufficient to create target index Haplotype-Transcript GBWT

[jeizenga]

You've run across a missing feature in vg autoindex that I've been meaning to implement for a while now. We have external pipelines that can create spliced pangenome graphs and pantranscriptomes from GFA input, but I haven't imported that logic into vg autoindex yet. For the time being, vg autoindex only supports FASTA + VCF input, and the VCF inputs need to contain phased haplotypes. The FASTA and VCF that you provided would be sufficient, except that it seems that the VCF isn't phased.

[ld9866]

Thank you for your prompt reply! I still have some questions I'd like to ask you. Firstly, we used the minigraph-cactus generated vcf file, I don't really understand how you say "phase" should work, I found pggb working well, but minigraph-cactus is not very good. Secondly, We took code and compared a standard 5 G transcriptome data to the human pan-genome, which was built using pggb, but we found that it was very slow, it took 3 days to complete a transcriptome, and we used an AMD7763 processor, which is very fast in other ways, I wonder if there is any way to speed up this process. Thank you! Best Yours

Map simulated RNA-seq reads using vg mpmap

vg mpmap -t 20 -x vg_rna.spliced.xg -g vg_rna.spliced.gcsa -d vg_rna.spliced.dist -f example_data/x_rna_1.fq -f example_data/x_rna_2.fq > mpmap.gamp

[jeizenga]

I talked to the main mingraph-cactus developer, and apparently the issue is that the VCF produced by that pipeline is uniformly haploid if the phased assemblies haven't been paired into diplotypes. A haploid VCF is trivially phased, so vg autoindex should actually treat these VCFs as phased. I have a PR fixing the bug now.

[jeizenga]

Regarding the speed issues, I expect is due to the complexity of the graph. PGGB tends to create graphs with a much more complicated topology than we have designed most of the tools in VG for. We haven't yet come up with robust ways to index and map to PGGB graphs, although it is something that we are working on. For the time being, my recommendation is to use the minigraph-cactus graphs. We may need to troubleshoot the indexing a bit more, but once it works I suspect you will get better performance on them.

[ld9866]

Hello developers! We have tried to install the updated vg software for many times, but I find it very difficult. The error is as follows. How can I solve this problem? Could you send me a source of the original code for me? My email: is tanxingdemogu9866@gmail.com

. ./source_me.sh && /usr/bin/g++ -I/vg/include -isystem /vg/include -I. -I/vg/src -I/h7/vg/src/unittest -I/vg/src/unittest/support -I/vg/src/subcommand -I/vg/include/dynami/cairo -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/pixman-1 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /u=return-type -std=c++14 -ggdb -g -I /vg/include -I/vg/include/dynamic -I /vg-v1.46.1.d/Software/vg_2023_03_17/vg-v1.46.1.dep.change/include/dynamic -fopenmp -msse4.2 -MMD -MP -c -o obj/subcommand/augment_main.o src/subcommand/augment_main.cpp In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:42:14: error: redefinition of ‘xg::pos_t xg::make_pos_t(const nid_t&, bool, const size_t&)’ 42 | inline pos_t make_pos_t(const nid_t& id, bool is_rev, const size_t& off) { | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:42:14: note: ‘xg::pos_t xg::make_pos_t(const nid_t&, bool, const size_t&)’ previously defined here 42 | inline pos_t make_pos_t(const nid_t& id, bool is_rev, const size_t& off) { | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:47:14: error: redefinition of ‘xg::nid_t xg::id(const pos_t&)’ 47 | inline nid_t id(const pos_t& pos) { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:47:14: note: ‘xg::nid_t xg::id(const pos_t&)’ previously defined here 47 | inline nid_t id(const pos_t& pos) { | ^~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:52:13: error: redefinition of ‘bool xg::is_rev(const pos_t&)’ 52 | inline bool is_rev(const pos_t& pos) { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:52:13: note: ‘bool xg::is_rev(const pos_t&)’ previously defined here 52 | inline bool is_rev(const pos_t& pos) { | ^~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:57:15: error: redefinition of ‘std::size_t xg::offset(const pos_t&)’ 57 | inline size_t offset(const pos_t& pos) { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:57:15: note: ‘size_t xg::offset(const pos_t&)’ previously defined here 57 | inline size_t offset(const pos_t& pos) { | ^~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:62:15: error: redefinition of ‘xg::nid_t& xg::get_id(xg::pos_t&)’ 62 | inline nid_t& get_id(pos_t& pos) { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:62:15: note: ‘xg::nid_t& xg::get_id(xg::pos_t&)’ previously defined here 62 | inline nid_t& get_id(pos_t& pos) { | ^~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:67:14: error: redefinition of ‘bool& xg::get_is_rev(xg::pos_t&)’ 67 | inline bool& get_is_rev(pos_t& pos) { | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:67:14: note: ‘bool& xg::get_is_rev(xg::pos_t&)’ previously defined here 67 | inline bool& get_is_rev(pos_t& pos) { | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:72:16: error: redefinition of ‘std::size_t& xg::get_offset(xg::pos_t&)’ 72 | inline size_t& get_offset(pos_t& pos) { | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:72:16: note: ‘size_t& xg::get_offset(xg::pos_t&)’ previously defined here 72 | inline size_t& get_offset(pos_t& pos) { | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:77:13: error: redefinition of ‘bool xg::is_empty(const pos_t&)’ 77 | inline bool is_empty(const pos_t& pos) { | ^~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:77:13: note: ‘bool xg::is_empty(const pos_t&)’ previously defined here 77 | inline bool is_empty(const pos_t& pos) { | ^~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:83:14: error: redefinition of ‘xg::pos_t xg::reverse(const pos_t&, std::size_t)’ 83 | inline pos_t reverse(const pos_t& pos, size_t node_length) { | ^~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:83:14: note: ‘xg::pos_t xg::reverse(const pos_t&, size_t)’ previously defined here 83 | inline pos_t reverse(const pos_t& pos, size_t node_length) { | ^~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:93:14: error: redefinition of ‘xg::pos_t xg::reverse_base_pos(const pos_t&, std::size_t)’ 93 | inline pos_t reverse_base_pos(const pos_t& pos, size_t node_length) { | ^~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:93:14: note: ‘xg::pos_t xg::reverse_base_pos(const pos_t&, size_t)’ previously defined here 93 | inline pos_t reverse_base_pos(const pos_t& pos, size_t node_length) { | ^~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:103:22: error: redefinition of ‘std::ostream& xg::operator<<(std::ostream&, const pos_t&)’ 103 | inline std::ostream& operator<<(std::ostream& out, const pos_t& pos) { | ^~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:103:22: note: ‘std::ostream& xg::operator<<(std::ostream&, const pos_t&)’ previously defined here 103 | inline std::ostream& operator<<(std::ostream& out, const pos_t& pos) { | ^~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:107:19: error: redefinition of ‘const char xg::complement [256]’ 107 | static const char complement[256] = {'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', // 8 | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:107:19: note: ‘const char xg::complement [256]’ previously defined here 107 | static const char complement[256] = {'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', // 8 | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:140:13: error: redefinition of ‘char xg::reverse_complement(const char&)’ 140 | inline char reverse_complement(const char& c) { | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:140:13: note: ‘char xg::reverse_complement(const char&)’ previously defined here 140 | inline char reverse_complement(const char& c) { | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:144:20: error: redefinition of ‘std::string xg::reverse_complement(const string&)’ 144 | inline std::string reverse_complement(const std::string& seq) { | ^~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:144:20: note: ‘std::string xg::reverse_complement(const string&)’ previously defined here 144 | inline std::string reverse_complement(const std::string& seq) { | ^~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:153:13: error: redefinition of ‘void xg::reverse_complement_in_place(std::string&)’ 153 | inline void reverse_complement_in_place(std::string& seq) { | ^~~~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:153:13: note: ‘void xg::reverse_complement_in_place(std::string&)’ previously defined here 153 | inline void reverse_complement_in_place(std::string& seq) { | ^~~~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:172:7: error: redefinition of ‘class xg::XGFormatError’ 172 | class XGFormatError : public std::runtime_error { | ^~~~~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:172:7: note: previous definition of ‘class xg::XGFormatError’ 172 | class XGFormatError : public std::runtime_error { | ^~~~~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:181:7: error: redefinition of ‘class xg::XG’ 181 | class XG : public PathPositionHandleGraph, public SerializableHandleGraph, public VectorizableHandleGraph { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:181:7: note: previous definition of ‘class xg::XG’ 181 | class XG : public PathPositionHandleGraph, public SerializableHandleGraph, public VectorizableHandleGraph { | ^~ In file included from src/subcommand/../io/save_handle_graph.hpp:13, from src/subcommand/augment_main.cpp:29: /vg-v1.46.1.dep.change/include/xg.hpp:543:7: error: redefinition of ‘class xg::XGPath’ 543 | class XGPath { | ^~ In file included from src/subcommand/augment_main.cpp:25: /vg/include/dynamic/../xg.hpp:543:7: note: previous definition of ‘class xg::XGPath’ 543 | class XGPath { | ^~ make: *** [Makefile:884: obj/subcommand/augment_main.o] Error 1

[ld9866]

@jeizenga

[jeizenga]

The GitHub repository is the original code. Are you following the installation directions from our README? Also, what operating system and compiler version are you using?

[ld9866]

Of course, I was able to install version 1.46 from the original code, but the latest version of the installation encountered this problem, is it possible that the newly released features have dependencies? My system is Linux and Ubuntu 20.04

[jeizenga]

Something looks like it's gotten mixed up in your directory, which is causing you to have two copies of the XG headers at /vg-v1.46.1.dep.change/include/xg.hpp and /vg/include/xg.hpp. I'm not sure what your directory structure looks like, but possibly you are adding one of these to your include path? Another option would be to delete whichever of these directories you're not using, or to move it somewhere where the build system won't pick it up.

[ld9866]

Here，l move the path, and the problem is solved. However, a new problem occurred here，can you help me? Best yours,

. ./source_me.sh && /usr/bin/g++ -I/vg_2023_03_18/vg/include -isystem /vg_2023_03_18/vg/include -I. -I/vg_2023_03_18/vg/src -I/vg_2023_03_18/vg/src/unittest -I/vg_2023_03_18/vg/src/unittest/support -I/vg_2023_03_18/vg/src/subcommand -I/vg_2023_03_18/vg/include/dynamic -pthread -isystem /usr/include/cairo -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/pixman-1 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -O3 -Werror=return-type -std=c++14 -ggdb -g -I /vg_2023_03_18/vg/include -I/vg_2023_03_18/vg/include/dynamic -fopenmp -msse4.2 -MMD -MP -c -o obj/subcommand/sift_main.o src/subcommand/sift_main.cpp . ./source_me.sh && /usr/bin/g++ -I/vg_2023_03_18/vg/include -isystem /vg_2023_03_18/vg/include -I. -I/vg_2023_03_18/vg/src -I/vg_2023_03_18/vg/src/unittest -I/vg_2023_03_18/vg/src/unittest/support -I/vg_2023_03_18/vg/src/subcommand -I/vg_2023_03_18/vg/include/dynamic -pthread -isystem /usr/include/cairo -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/pixman-1 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -O3 -Werror=return-type -std=c++14 -ggdb -g -I /vg_2023_03_18/vg/include -I/vg_2023_03_18/vg/include/dynamic -fopenmp -msse4.2 -MMD -MP -c -o obj/subcommand/sim_main.o src/subcommand/sim_main.cpp src/subcommand/sim_main.cpp: In function ‘int main_sim(int, char*)’: src/subcommand/sim_main.cpp:660:11: error: ‘ReferencePathVectorizableOverlayHelper’ is not a member of ‘bdsg’; did you mean ‘PathVectorizableOverlayHelper’? 660 | bdsg::ReferencePathVectorizableOverlayHelper overlay_helper; | ^~~~~~~~~~ | PathVectorizableOverlayHelper src/subcommand/sim_main.cpp:661:77: error: ‘overlay_helper’ was not declared in this scope 661 | PathPositionHandleGraph xgidx = dynamic_cast<PathPositionHandleGraph*>(overlay_helper.apply(path_handle_graph.get())); | ^~~~~~ make: *** [Makefile:884: obj/subcommand/sim_main.o] Error 1

[jeizenga]

Try using git submodule update --init --recursive and building again.

[ld9866]

Dear developer: We tried according to your advice, but we find it is very difficult, because the use of "git clone -- recursive https://github.com/vgteam/vg.git" when they meet a lot of problems often failed due to network problems, Therefore, I manually installed many dependency packages for testing, but this time we could not use the same method to complete the installation. Could you please send your installation source code by email? When we used "git clone --recursive https://github.com/vgteam/vg.git", the code always failsed and never success. The git submodule update --init --recursive is also not work. Result: Cloning into 'vg'... remote: Enumerating objects: 151889, done. remote: Counting objects: 100% (2180/2180), done. remote: Compressing objects: 100% (413/413), done. remote: Total 151889 (delta 1804), reused 2124 (delta 1766), pack-reused 149709 Receiving objects: 100% (151889/151889), 130.65 MiB | 5.49 MiB/s, done. Resolving deltas: 100% (131251/131251), done. Submodule 'vgteam_bbhash' (https://github.com/vgteam/BBHash.git) registered for path 'deps/BBHash' Submodule 'deps/DYNAMIC' (https://github.com/vgteam/DYNAMIC) registered for path 'deps/DYNAMIC' Submodule 'deps/FlameGraph' (https://github.com/brendangregg/FlameGraph) registered for path 'deps/FlameGraph' Submodule 'deps/atomic_queue' (https://github.com/max0x7ba/atomic_queue.git) registered for path 'deps/atomic_queue' Submodule 'deps/backward-cpp' (https://github.com/adamnovak/backward-cpp.git) registered for path 'deps/backward-cpp' Submodule 'bash-tap' (https://github.com/illusori/bash-tap.git) registered for path 'deps/bash-tap' Submodule 'deps/dozeu' (https://github.com/vgteam/dozeu.git) registered for path 'deps/dozeu' Submodule 'deps/elfutils' (git://sourceware.org/git/elfutils.git) registered for path 'deps/elfutils' Submodule 'fastahack' (https://github.com/vgteam/fastahack.git) registered for path 'deps/fastahack' Submodule 'deps/fermi-lite' (https://github.com/vgteam/fermi-lite.git) registered for path 'deps/fermi-lite' Submodule 'deps/gbwt' (https://github.com/jltsiren/gbwt.git) registered for path 'deps/gbwt' Submodule 'deps/gbwtgraph' (https://github.com/jltsiren/gbwtgraph.git) registered for path 'deps/gbwtgraph' Submodule 'gcsa2' (https://github.com/jltsiren/gcsa2.git) registered for path 'deps/gcsa2' Submodule 'gssw' (https://github.com/vgteam/gssw.git) registered for path 'deps/gssw' Submodule 'deps/htslib' (https://github.com/samtools/htslib.git) registered for path 'deps/htslib' Submodule 'deps/ips4o' (https://github.com/vgteam/ips4o.git) registered for path 'deps/ips4o' Submodule 'deps/jemalloc' (https://github.com/jemalloc/jemalloc.git) registered for path 'deps/jemalloc' Submodule 'deps/kff-cpp-api' (https://github.com/Kmer-File-Format/kff-cpp-api.git) registered for path 'deps/kff-cpp-api' Submodule 'deps/libVCFH' (https://github.com/edawson/libVCFH.git) registered for path 'deps/libVCFH' Submodule 'deps/libbdsg' (https://github.com/vgteam/libbdsg.git) registered for path 'deps/libbdsg' Submodule 'deps/libdeflate' (https://github.com/ebiggers/libdeflate.git) registered for path 'deps/libdeflate' Submodule 'deps/libhandlegraph' (https://github.com/vgteam/libhandlegraph.git) registered for path 'deps/libhandlegraph' Submodule 'deps/libvgio' (https://github.com/vgteam/libvgio.git) registered for path 'deps/libvgio' Submodule 'lru_cache' (https://github.com/ekg/lru_cache.git) registered for path 'deps/lru_cache' Submodule 'deps/mio' (https://github.com/mandreyel/mio.git) registered for path 'deps/mio' Submodule 'deps/mmmultimap' (https://github.com/ekg/mmmultimap.git) registered for path 'deps/mmmultimap' Submodule 'deps/pinchesAndCacti' (https://github.com/benedictpaten/pinchesAndCacti.git) registered for path 'deps/pinchesAndCacti' Submodule 'progress_bar' (https://github.com/vgteam/cpp_progress_bar.git) registered for path 'deps/progress_bar' Submodule 'deps/raptor' (https://github.com/vgteam/raptor.git) registered for path 'deps/raptor' Submodule 'sdsl-lite' (https://github.com/vgteam/sdsl-lite.git) registered for path 'deps/sdsl-lite' Submodule 'sha1' (https://github.com/vog/sha1.git) registered for path 'deps/sha1' Submodule 'deps/snappy' (https://github.com/google/snappy.git) registered for path 'deps/snappy' Submodule 'deps/sonLib' (https://github.com/benedictpaten/sonLib.git) registered for path 'deps/sonLib' Submodule 'deps/sparsehash' (https://github.com/sparsehash/sparsehash.git) registered for path 'deps/sparsehash' Submodule 'deps/sparsepp' (https://github.com/greg7mdp/sparsepp.git) registered for path 'deps/sparsepp' Submodule 'deps/Complete-Striped-Smith-Waterman-Library' (https://github.com/vgteam/Complete-Striped-Smith-Waterman-Library.git) registered or path 'deps/ssw' Submodule 'deps/structures' (https://github.com/vgteam/structures.git) registered for path 'deps/structures' Submodule 'deps/sublinear-Li-Stephens' (https://github.com/yoheirosen/sublinear-Li-Stephens.git) registered for path 'deps/sublinear-Li-Stepens' Submodule 'deps/tabixpp' (https://github.com/ekg/tabixpp.git) registered for path 'deps/tabixpp' Submodule 'vcflib' (https://github.com/vcflib/vcflib.git) registered for path 'deps/vcflib' Submodule 'deps/xg' (https://github.com/vgteam/xg.git) registered for path 'deps/xg' Submodule 'doc/wiki' (https://github.com/vgteam/vg.wiki.git) registered for path 'doc/wiki' Submodule 'src/simde' (https://github.com/nemequ/simde-no-tests) registered for path 'src/simde' Cloning into '/Software/vg_2023_03_20/vg/deps/BBHash'... fatal: unable to access 'https://github.com/vgteam/BBHash.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/BBHash.git' into submodule path '/Software/vg_2023_03_20/vg/deps/BBHash' failed Failed to clone 'deps/BBHash'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/DYNAMIC'... fatal: unable to access 'https://github.com/vgteam/DYNAMIC/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/DYNAMIC' into submodule path '/Software/vg_2023_03_20/vg/deps/DYNAMIC' failed Failed to clone 'deps/DYNAMIC'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/FlameGraph'... remote: Enumerating objects: 1217, done.
remote: Counting objects: 100% (81/81), done.
remote: Compressing objects: 100% (49/49), done.
remote: Total 1217 (delta 43), reused 59 (delta 32), pack-reused 1136
Receiving objects: 100% (1217/1217), 1.93 MiB | 3.00 MiB/s, done. Resolving deltas: 100% (700/700), done. Cloning into '/Software/vg_2023_03_20/vg/deps/atomic_queue'... fatal: unable to access 'https://github.com/max0x7ba/atomic_queue.git/': GnuTLS recv error (-110): The TLS connection was non-properly termiated. fatal: clone of 'https://github.com/max0x7ba/atomic_queue.git' into submodule path '/Software/vg_2023_03_20/vg/deps/atomic_queue failed Failed to clone 'deps/atomic_queue'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/backward-cpp'... remote: Enumerating objects: 564, done.
remote: Counting objects: 100% (105/105), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 564 (delta 100), reused 100 (delta 100), pack-reused 459
Receiving objects: 100% (564/564), 472.86 KiB | 1.28 MiB/s, done. Resolving deltas: 100% (348/348), done. Cloning into '/Software/vg_2023_03_20/vg/deps/bash-tap'... fatal: unable to access 'https://github.com/illusori/bash-tap.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminate. fatal: clone of 'https://github.com/illusori/bash-tap.git' into submodule path '/Software/vg_2023_03_20/vg/deps/bash-tap' failed Failed to clone 'deps/bash-tap'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/dozeu'... fatal: unable to access 'https://github.com/vgteam/dozeu.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/dozeu.git' into submodule path '/Software/vg_2023_03_20/vg/deps/dozeu' failed Failed to clone 'deps/dozeu'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/elfutils'... remote: Enumerating objects: 38098, done.
remote: Counting objects: 100% (38098/38098), done.
remote: Compressing objects: 100% (8217/8217), done.
remote: Total 38098 (delta 31881), reused 35565 (delta 29840), pack-reused 0
Receiving objects: 100% (38098/38098), 21.11 MiB | 1.34 MiB/s, done. Resolving deltas: 100% (31881/31881), done. Cloning into '/Software/vg_2023_03_20/vg/deps/fastahack'... remote: Enumerating objects: 234, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 234 (delta 2), reused 5 (delta 2), pack-reused 227
Receiving objects: 100% (234/234), 57.11 KiB | 602.00 KiB/s, done. Resolving deltas: 100% (130/130), done. Cloning into '/Software/vg_2023_03_20/vg/deps/fermi-lite'... fatal: unable to access 'https://github.com/vgteam/fermi-lite.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminate. fatal: clone of 'https://github.com/vgteam/fermi-lite.git' into submodule path '/Software/vg_2023_03_20/vg/deps/fermi-lite' faild Failed to clone 'deps/fermi-lite'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/gbwt'... remote: Enumerating objects: 2255, done.
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 2255 (delta 4), reused 14 (delta 4), pack-reused 2237
Receiving objects: 100% (2255/2255), 664.25 KiB | 1.74 MiB/s, done. Resolving deltas: 100% (1597/1597), done. Cloning into '/Software/vg_2023_03_20/vg/deps/gbwtgraph'... remote: Enumerating objects: 2408, done.
remote: Counting objects: 100% (370/370), done.
remote: Compressing objects: 100% (138/138), done.
remote: Total 2408 (delta 246), reused 305 (delta 205), pack-reused 2038
Receiving objects: 100% (2408/2408), 656.02 KiB | 1.37 MiB/s, done. Resolving deltas: 100% (1638/1638), done. Cloning into '/Software/vg_2023_03_20/vg/deps/gcsa2'... remote: Enumerating objects: 1859, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 1859 (delta 21), reused 34 (delta 15), pack-reused 1813
Receiving objects: 100% (1859/1859), 977.89 KiB | 1.78 MiB/s, done. Resolving deltas: 100% (1337/1337), done. Cloning into '/Software/vg_2023_03_20/vg/deps/gssw'... fatal: unable to access 'https://github.com/vgteam/gssw.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/gssw.git' into submodule path '/Software/vg_2023_03_20/vg/deps/gssw' failed Failed to clone 'deps/gssw'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/htslib'... remote: Enumerating objects: 15775, done.
remote: Counting objects: 100% (54/54), done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 15775 (delta 21), reused 27 (delta 9), pack-reused 15721
Receiving objects: 100% (15775/15775), 11.86 MiB | 3.82 MiB/s, done. Resolving deltas: 100% (11427/11427), done. Cloning into '/Software/vg_2023_03_20/vg/deps/ips4o'... remote: Enumerating objects: 64, done.
remote: Total 64 (delta 0), reused 0 (delta 0), pack-reused 64
Receiving objects: 100% (64/64), 47.20 KiB | 478.00 KiB/s, done. Resolving deltas: 100% (34/34), done. Cloning into '/Software/vg_2023_03_20/vg/deps/jemalloc'... remote: Enumerating objects: 28173, done.
remote: Counting objects: 100% (66/66), done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 28173 (delta 22), reused 40 (delta 12), pack-reused 28107
Receiving objects: 100% (28173/28173), 11.56 MiB | 2.80 MiB/s, done. Resolving deltas: 100% (20630/20630), done. Cloning into '/Software/vg_2023_03_20/vg/deps/kff-cpp-api'... remote: Enumerating objects: 303, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 303 (delta 20), reused 30 (delta 13), pack-reused 262
Receiving objects: 100% (303/303), 102.46 KiB | 555.00 KiB/s, done. Resolving deltas: 100% (192/192), done. Cloning into '/Software/vg_2023_03_20/vg/deps/libVCFH'... fatal: unable to access 'https://github.com/edawson/libVCFH.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/edawson/libVCFH.git' into submodule path '/Software/vg_2023_03_20/vg/deps/libVCFH' failed Failed to clone 'deps/libVCFH'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/libbdsg'... remote: Enumerating objects: 10903, done.
remote: Counting objects: 100% (1328/1328), done.
remote: Compressing objects: 100% (298/298), done.
remote: Total 10903 (delta 968), reused 1248 (delta 933), pack-reused 9575
Receiving objects: 100% (10903/10903), 14.43 MiB | 9.57 MiB/s, done. Resolving deltas: 100% (7168/7168), done. Cloning into '/Software/vg_2023_03_20/vg/deps/libdeflate'... remote: Enumerating objects: 4022, done.
remote: Counting objects: 100% (559/559), done.
remote: Compressing objects: 100% (239/239), done.
remote: Total 4022 (delta 344), reused 409 (delta 285), pack-reused 3463
Receiving objects: 100% (4022/4022), 1.57 MiB | 3.26 MiB/s, done. Resolving deltas: 100% (2761/2761), done. Cloning into '/Software/vg_2023_03_20/vg/deps/libhandlegraph'... fatal: unable to access 'https://github.com/vgteam/libhandlegraph.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/libhandlegraph.git' into submodule path '/Software/vg_2023_03_20/vg/deps/libhandlegraph' failed Failed to clone 'deps/libhandlegraph'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/libvgio'... remote: Enumerating objects: 1243, done.
remote: Counting objects: 100% (121/121), done.
remote: Compressing objects: 100% (74/74), done.
remote: Total 1243 (delta 46), reused 65 (delta 35), pack-reused 1122
Receiving objects: 100% (1243/1243), 337.74 KiB | 1.17 MiB/s, done. Resolving deltas: 100% (727/727), done. Cloning into '/Software/vg_2023_03_20/vg/deps/lru_cache'... remote: Enumerating objects: 37, done.
remote: Total 37 (delta 0), reused 0 (delta 0), pack-reused 37
Receiving objects: 100% (37/37), 8.00 KiB | 8.00 MiB/s, done. Resolving deltas: 100% (10/10), done. Cloning into '/Software/vg_2023_03_20/vg/deps/mio'... error: RPC failed; curl 16 Error in the HTTP2 framing layer fatal: expected flush after ref listing fatal: clone of 'https://github.com/mandreyel/mio.git' into submodule path '/Software/vg_2023_03_20/vg/deps/mio' failed Failed to clone 'deps/mio'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/mmmultimap'... remote: Enumerating objects: 362, done.
remote: Counting objects: 100% (170/170), done.
remote: Compressing objects: 100% (112/112), done.
remote: Total 362 (delta 105), reused 119 (delta 58), pack-reused 192
Receiving objects: 100% (362/362), 85.47 KiB | 491.00 KiB/s, done. Resolving deltas: 100% (215/215), done. Cloning into '/Software/vg_2023_03_20/vg/deps/pinchesAndCacti'... remote: Enumerating objects: 1282, done.
remote: Total 1282 (delta 0), reused 0 (delta 0), pack-reused 1282
Receiving objects: 100% (1282/1282), 602.35 KiB | 1.57 MiB/s, done. Resolving deltas: 100% (825/825), done. Cloning into '/Software/vg_2023_03_20/vg/deps/progress_bar'... fatal: unable to access 'https://github.com/vgteam/cpp_progress_bar.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/cpp_progress_bar.git' into submodule path '/Software/vg_2023_03_20/vg/deps/progress_bar' failed Failed to clone 'deps/progress_bar'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/raptor'... remote: Enumerating objects: 34612, done.
remote: Counting objects: 100% (209/209), done.
remote: Compressing objects: 100% (93/93), done.
remote: Total 34612 (delta 125), reused 189 (delta 115), pack-reused 34403
Receiving objects: 100% (34612/34612), 27.03 MiB | 6.33 MiB/s, done. Resolving deltas: 100% (21031/21031), done. Cloning into '/Software/vg_2023_03_20/vg/deps/sdsl-lite'... fatal: unable to access 'https://github.com/vgteam/sdsl-lite.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/sdsl-lite.git' into submodule path '/Software/vg_2023_03_20/vg/deps/sdsl-lite' failed Failed to clone 'deps/sdsl-lite'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/sha1'... remote: Enumerating objects: 164, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 164 (delta 4), reused 15 (delta 2), pack-reused 145
Receiving objects: 100% (164/164), 35.98 KiB | 400.00 KiB/s, done. Resolving deltas: 100% (85/85), done. Cloning into '/Software/vg_2023_03_20/vg/deps/snappy'... remote: Enumerating objects: 1394, done.
remote: Counting objects: 100% (64/64), done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 1394 (delta 35), reused 44 (delta 30), pack-reused 1330
Receiving objects: 100% (1394/1394), 2.71 MiB | 1.57 MiB/s, done. Resolving deltas: 100% (946/946), done. Cloning into '/Software/vg_2023_03_20/vg/deps/sonLib'... remote: Enumerating objects: 5176, done.
remote: Counting objects: 100% (59/59), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 5176 (delta 25), reused 41 (delta 18), pack-reused 5117
Receiving objects: 100% (5176/5176), 1.59 MiB | 2.50 MiB/s, done. Resolving deltas: 100% (3326/3326), done. Cloning into '/Software/vg_2023_03_20/vg/deps/sparsehash'... remote: Enumerating objects: 1416, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 1416 (delta 3), reused 11 (delta 3), pack-reused 1404
Receiving objects: 100% (1416/1416), 812.04 KiB | 1.74 MiB/s, done. Resolving deltas: 100% (1005/1005), done. Cloning into '/Software/vg_2023_03_20/vg/deps/sparsepp'... fatal: unable to access 'https://github.com/greg7mdp/sparsepp.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/greg7mdp/sparsepp.git' into submodule path '/Software/vg_2023_03_20/vg/deps/sparsepp' failed Failed to clone 'deps/sparsepp'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/ssw'... remote: Enumerating objects: 1585, done.
remote: Total 1585 (delta 0), reused 0 (delta 0), pack-reused 1585
Receiving objects: 100% (1585/1585), 12.18 MiB | 5.68 MiB/s, done. Resolving deltas: 100% (987/987), done. Cloning into '/Software/vg_2023_03_20/vg/deps/structures'... remote: Enumerating objects: 320, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 320 (delta 16), reused 25 (delta 11), pack-reused 282
Receiving objects: 100% (320/320), 96.65 KiB | 460.00 KiB/s, done. Resolving deltas: 100% (169/169), done. Cloning into '/Software/vg_2023_03_20/vg/deps/sublinear-Li-Stephens'... remote: Enumerating objects: 1178, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 1178 (delta 0), reused 3 (delta 0), pack-reused 1174
Receiving objects: 100% (1178/1178), 3.44 MiB | 3.37 MiB/s, done. Resolving deltas: 100% (899/899), done. Cloning into '/Software/vg_2023_03_20/vg/deps/tabixpp'... remote: Enumerating objects: 191, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 191 (delta 25), reused 45 (delta 18), pack-reused 135
Receiving objects: 100% (191/191), 99.75 KiB | 561.00 KiB/s, done. Resolving deltas: 100% (78/78), done. Cloning into '/Software/vg_2023_03_20/vg/deps/vcflib'... fatal: unable to access 'https://github.com/vcflib/vcflib.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vcflib/vcflib.git' into submodule path '/Software/vg_2023_03_20/vg/deps/vcflib' failed Failed to clone 'deps/vcflib'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/xg'... error: RPC failed; curl 16 Error in the HTTP2 framing layer fatal: error reading section header 'shallow-info' fatal: clone of 'https://github.com/vgteam/xg.git' into submodule path '/Software/vg_2023_03_20/vg/deps/xg' failed Failed to clone 'deps/xg'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/doc/wiki'... remote: Enumerating objects: 3153, done.
remote: Total 3153 (delta 0), reused 0 (delta 0), pack-reused 3153
Receiving objects: 100% (3153/3153), 491.80 KiB | 1.34 MiB/s, done. Resolving deltas: 100% (2010/2010), done. Cloning into '/Software/vg_2023_03_20/vg/src/simde'... error: RPC failed; curl 16 Error in the HTTP2 framing layer fatal: expected flush after ref listing fatal: clone of 'https://github.com/nemequ/simde-no-tests' into submodule path '/Software/vg_2023_03_20/vg/src/simde' failed Failed to clone 'src/simde'. Retry scheduled Cloning into '/Software/vg_2023_03_20/vg/deps/BBHash'... fatal: unable to access 'https://github.com/vgteam/BBHash.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. fatal: clone of 'https://github.com/vgteam/BBHash.git' into submodule path '/Software/vg_2023_03_20/vg/deps/BBHash' failed Failed to clone 'deps/BBHash' a second time, aborting

[jeizenga]

Again, the GitHub repository is the source code. If a submodule update fails because of a network interruption, you can use git pull in the submodules subdirectory in deps/, or alternatively try git submodule update again.

[jeizenga]

Following up: I just made a PR to add a GFA-based pipeline to vg autoindex for vg mpmap and rpvg, so if you're still interested in going that direction, it should be possible now.

[ld9866]

Well, thank you for your patience. We will continue to test your method in subsequent studies, so far we have solved the problem and conducted follow-up analysis. Best yours