search

seqan / seqan3

The modern C++ library for sequence analysis. Contains version 3 of the library and API docs.
https://www.seqan.de
Other
406 stars 81 forks source link

Use OS/system packaged sdsl library #3162

Open kbipinkumar opened 1 year ago

kbipinkumar commented 1 year ago

Platform

Question

Is there a way to use sdsl library packaged in distribution repository? we have managed to package sdsl as package for archlinux . the said package install sdsl library and headers as below

pacman -pQl sdsl-lite-2.3.1-2-x86_64.pkg.tar.zst
sdsl-lite /usr/
sdsl-lite /usr/include/
sdsl-lite /usr/include/sdsl/
sdsl-lite /usr/include/sdsl/bit_vector_il.hpp
sdsl-lite /usr/include/sdsl/bit_vectors.hpp
sdsl-lite /usr/include/sdsl/bits.hpp
sdsl-lite /usr/include/sdsl/bp_support.hpp
sdsl-lite /usr/include/sdsl/bp_support_algorithm.hpp
sdsl-lite /usr/include/sdsl/bp_support_g.hpp
sdsl-lite /usr/include/sdsl/bp_support_gg.hpp
sdsl-lite /usr/include/sdsl/bp_support_sada.hpp
sdsl-lite /usr/include/sdsl/coder.hpp
sdsl-lite /usr/include/sdsl/coder_comma.hpp
sdsl-lite /usr/include/sdsl/coder_elias_delta.hpp
sdsl-lite /usr/include/sdsl/coder_elias_gamma.hpp
sdsl-lite /usr/include/sdsl/coder_fibonacci.hpp
sdsl-lite /usr/include/sdsl/config.hpp
sdsl-lite /usr/include/sdsl/construct.hpp
sdsl-lite /usr/include/sdsl/construct_bwt.hpp
sdsl-lite /usr/include/sdsl/construct_config.hpp
sdsl-lite /usr/include/sdsl/construct_isa.hpp
sdsl-lite /usr/include/sdsl/construct_lcp.hpp
sdsl-lite /usr/include/sdsl/construct_lcp_helper.hpp
sdsl-lite /usr/include/sdsl/construct_sa.hpp
sdsl-lite /usr/include/sdsl/construct_sa_se.hpp
sdsl-lite /usr/include/sdsl/csa_alphabet_strategy.hpp
sdsl-lite /usr/include/sdsl/csa_bitcompressed.hpp
sdsl-lite /usr/include/sdsl/csa_sada.hpp
sdsl-lite /usr/include/sdsl/csa_sampling_strategy.hpp
sdsl-lite /usr/include/sdsl/csa_wt.hpp
sdsl-lite /usr/include/sdsl/cst_fully.hpp
sdsl-lite /usr/include/sdsl/cst_iterators.hpp
sdsl-lite /usr/include/sdsl/cst_sada.hpp
sdsl-lite /usr/include/sdsl/cst_sct3.hpp
sdsl-lite /usr/include/sdsl/dac_vector.hpp
sdsl-lite /usr/include/sdsl/enc_vector.hpp
sdsl-lite /usr/include/sdsl/fast_cache.hpp
sdsl-lite /usr/include/sdsl/hyb_vector.hpp
sdsl-lite /usr/include/sdsl/int_vector.hpp
sdsl-lite /usr/include/sdsl/int_vector_buffer.hpp
sdsl-lite /usr/include/sdsl/int_vector_io_wrappers.hpp
sdsl-lite /usr/include/sdsl/int_vector_mapper.hpp
sdsl-lite /usr/include/sdsl/inv_perm_support.hpp
sdsl-lite /usr/include/sdsl/io.hpp
sdsl-lite /usr/include/sdsl/iterators.hpp
sdsl-lite /usr/include/sdsl/k2_treap.hpp
sdsl-lite /usr/include/sdsl/k2_treap_algorithm.hpp
sdsl-lite /usr/include/sdsl/k2_treap_helper.hpp
sdsl-lite /usr/include/sdsl/k2_tree.hpp
sdsl-lite /usr/include/sdsl/k2_tree_helper.hpp
sdsl-lite /usr/include/sdsl/lcp.hpp
sdsl-lite /usr/include/sdsl/lcp_bitcompressed.hpp
sdsl-lite /usr/include/sdsl/lcp_byte.hpp
sdsl-lite /usr/include/sdsl/lcp_dac.hpp
sdsl-lite /usr/include/sdsl/lcp_support_sada.hpp
sdsl-lite /usr/include/sdsl/lcp_support_tree.hpp
sdsl-lite /usr/include/sdsl/lcp_support_tree2.hpp
sdsl-lite /usr/include/sdsl/lcp_vlc.hpp
sdsl-lite /usr/include/sdsl/lcp_wt.hpp
sdsl-lite /usr/include/sdsl/louds_tree.hpp
sdsl-lite /usr/include/sdsl/memory_management.hpp
sdsl-lite /usr/include/sdsl/nearest_neighbour_dictionary.hpp
sdsl-lite /usr/include/sdsl/nn_dict_dynamic.hpp
sdsl-lite /usr/include/sdsl/qsufsort.hpp
sdsl-lite /usr/include/sdsl/ram_filebuf.hpp
sdsl-lite /usr/include/sdsl/ram_fs.hpp
sdsl-lite /usr/include/sdsl/rank_support.hpp
sdsl-lite /usr/include/sdsl/rank_support_scan.hpp
sdsl-lite /usr/include/sdsl/rank_support_v.hpp
sdsl-lite /usr/include/sdsl/rank_support_v5.hpp
sdsl-lite /usr/include/sdsl/raster_img.hpp
sdsl-lite /usr/include/sdsl/rle_vector.hpp
sdsl-lite /usr/include/sdsl/rmq_succinct_sada.hpp
sdsl-lite /usr/include/sdsl/rmq_succinct_sct.hpp
sdsl-lite /usr/include/sdsl/rmq_support.hpp
sdsl-lite /usr/include/sdsl/rmq_support_sparse_table.hpp
sdsl-lite /usr/include/sdsl/rrr_helper.hpp
sdsl-lite /usr/include/sdsl/rrr_vector.hpp
sdsl-lite /usr/include/sdsl/rrr_vector_15.hpp
sdsl-lite /usr/include/sdsl/sd_vector.hpp
sdsl-lite /usr/include/sdsl/sdsl_concepts.hpp
sdsl-lite /usr/include/sdsl/select_support.hpp
sdsl-lite /usr/include/sdsl/select_support_mcl.hpp
sdsl-lite /usr/include/sdsl/select_support_scan.hpp
sdsl-lite /usr/include/sdsl/sfstream.hpp
sdsl-lite /usr/include/sdsl/simple_sds.hpp
sdsl-lite /usr/include/sdsl/sorted_int_stack.hpp
sdsl-lite /usr/include/sdsl/sorted_multi_stack_support.hpp
sdsl-lite /usr/include/sdsl/sorted_stack_support.hpp
sdsl-lite /usr/include/sdsl/structure_tree.hpp
sdsl-lite /usr/include/sdsl/suffix_array_algorithm.hpp
sdsl-lite /usr/include/sdsl/suffix_array_helper.hpp
sdsl-lite /usr/include/sdsl/suffix_arrays.hpp
sdsl-lite /usr/include/sdsl/suffix_tree_algorithm.hpp
sdsl-lite /usr/include/sdsl/suffix_tree_helper.hpp
sdsl-lite /usr/include/sdsl/suffix_trees.hpp
sdsl-lite /usr/include/sdsl/uint128_t.hpp
sdsl-lite /usr/include/sdsl/uint256_t.hpp
sdsl-lite /usr/include/sdsl/uintx_t.hpp
sdsl-lite /usr/include/sdsl/util.hpp
sdsl-lite /usr/include/sdsl/vectors.hpp
sdsl-lite /usr/include/sdsl/vlc_vector.hpp
sdsl-lite /usr/include/sdsl/wavelet_trees.hpp
sdsl-lite /usr/include/sdsl/wm_int.hpp
sdsl-lite /usr/include/sdsl/wt_algorithm.hpp
sdsl-lite /usr/include/sdsl/wt_ap.hpp
sdsl-lite /usr/include/sdsl/wt_blcd.hpp
sdsl-lite /usr/include/sdsl/wt_gmr.hpp
sdsl-lite /usr/include/sdsl/wt_helper.hpp
sdsl-lite /usr/include/sdsl/wt_huff.hpp
sdsl-lite /usr/include/sdsl/wt_hutu.hpp
sdsl-lite /usr/include/sdsl/wt_int.hpp
sdsl-lite /usr/include/sdsl/wt_pc.hpp
sdsl-lite /usr/include/sdsl/wt_rlmn.hpp
sdsl-lite /usr/lib/
sdsl-lite /usr/lib/libsdsl.so
sdsl-lite /usr/lib/libsdsl.so.2.3.1
sdsl-lite /usr/lib/libsdsl.so.3
sdsl-lite /usr/lib/pkgconfig/
sdsl-lite /usr/lib/pkgconfig/sdsl-lite.pc

my query is whether it is possible to use the sdsl library from OS repo instead of checking out sdsl repo into submodules folder?

eseiler commented 1 year ago

Dear @kbipinkumar,

Thanks for reaching out!

It is possible to use a packaged version, but the version needs to be 3.0.1, which is available here: https://github.com/xxsds/sdsl-lite

The packaged debian version (https://packages.debian.org/sid/libseqan3-dev, https://salsa.debian.org/med-team/seqan3) doesn't package sdsl-lite v3, but uses the submodule.

Please let us know if you have any further questions.

kbipinkumar commented 1 year ago

Thanks for the input. We will attempt to build with submodule approach then.

kbipinkumar commented 1 year ago

hi i am attempting to build from sources as below

$ git clone https://github.com/seqan/seqan3.git
$ cd seqan3
$ git submodule update --init --recursive
Submodule 'cereal' (https://github.com/USCiLab/cereal) registered for path 'submodules/cereal'
Submodule 'sdsl-lite' (https://github.com/xxsds/sdsl-lite) registered for path 'submodules/sdsl-lite'
Cloning into '/build/seqan3/src/seqan3/submodules/cereal'...
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite'...
Submodule path 'submodules/cereal': checked out 'ebef1e929807629befafbb2918ea1a08c7194554'
Submodule path 'submodules/sdsl-lite': checked out '549721d74f346f540b0e7c2e276dba643405c16e'
Submodule 'external/cereal' (https://github.com/USCiLab/cereal.git) registered for path 'submodules/sdsl-lite/external/cereal'
Submodule 'external/googletest' (https://github.com/google/googletest.git) registered for path 'submodules/sdsl-lite/external/googletest'
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite/external/cereal'...
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite/external/googletest'...
Submodule path 'submodules/sdsl-lite/external/cereal': checked out '02eace19a99ce3cd564ca4e379753d69af08c2c8'
Submodule path 'submodules/sdsl-lite/external/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
$ cmake -B build  \
        -DCMAKE_BUILD_TYPE='None' \
        -DCMAKE_INSTALL_PREFIX='/usr' \
        -DCMAKE_SKIP_RPATH=true \
        -Wno-dev
$  cmake --build build

the build process errors out as below

-- Finding SeqAn3 and checking requirements:
--   SeqAn3 include dir found:   /build/seqan3/src/seqan3/include
--   Detected as running from a repository checkout…
--     …adding submodule include:  /build/seqan3/src/seqan3/submodules/cereal/include
--     …adding submodule include:  /build/seqan3/src/seqan3/submodules/sdsl-lite/include
--   C++ Standard-20 support:    via -std=c++20
--   C++ Concepts support:       via -std=c++20
--   Thread support:             builtin.
CMake Error at build_system/seqan3-config.cmake:113 (message):
  The SDSL library is required, but wasn't found.  Get it from
  https://github.com/xxsds/sdsl-lite
Call Stack (most recent call first):
  build_system/seqan3-config.cmake:319 (seqan3_config_error)
  CMakeLists.txt:32 (find_package)

-- Configuring incomplete, errors occurred!

i am sure git submodule update --init --recursive did fetch sdsl-lite but it seems to be not detected by cmake.

SGSSGene commented 1 year ago

Hey, I couldn't reproduce this on my arch linux machine. Can you tell me the exact git commit version, you are using? The line C++ Concepts support: via -std=c++20 is suspicious. It was removed 24th Oct 2022 ( #771c10e108c ) . We require c++20 now in any case.

starsareintherose commented 1 year ago

hi i am attempting to build from sources as below

$ git clone https://github.com/seqan/seqan3.git
$ cd seqan3
$ git submodule update --init --recursive
Submodule 'cereal' (https://github.com/USCiLab/cereal) registered for path 'submodules/cereal'
Submodule 'sdsl-lite' (https://github.com/xxsds/sdsl-lite) registered for path 'submodules/sdsl-lite'
Cloning into '/build/seqan3/src/seqan3/submodules/cereal'...
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite'...
Submodule path 'submodules/cereal': checked out 'ebef1e929807629befafbb2918ea1a08c7194554'
Submodule path 'submodules/sdsl-lite': checked out '549721d74f346f540b0e7c2e276dba643405c16e'
Submodule 'external/cereal' (https://github.com/USCiLab/cereal.git) registered for path 'submodules/sdsl-lite/external/cereal'
Submodule 'external/googletest' (https://github.com/google/googletest.git) registered for path 'submodules/sdsl-lite/external/googletest'
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite/external/cereal'...
Cloning into '/build/seqan3/src/seqan3/submodules/sdsl-lite/external/googletest'...
Submodule path 'submodules/sdsl-lite/external/cereal': checked out '02eace19a99ce3cd564ca4e379753d69af08c2c8'
Submodule path 'submodules/sdsl-lite/external/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
$ cmake -B build  \
        -DCMAKE_BUILD_TYPE='None' \
        -DCMAKE_INSTALL_PREFIX='/usr' \
        -DCMAKE_SKIP_RPATH=true \
        -Wno-dev
$  cmake --build build

the build process errors out as below

-- Finding SeqAn3 and checking requirements:
--   SeqAn3 include dir found:   /build/seqan3/src/seqan3/include
--   Detected as running from a repository checkout…
--     …adding submodule include:  /build/seqan3/src/seqan3/submodules/cereal/include
--     …adding submodule include:  /build/seqan3/src/seqan3/submodules/sdsl-lite/include
--   C++ Standard-20 support:    via -std=c++20
--   C++ Concepts support:       via -std=c++20
--   Thread support:             builtin.
CMake Error at build_system/seqan3-config.cmake:113 (message):
  The SDSL library is required, but wasn't found.  Get it from
  https://github.com/xxsds/sdsl-lite
Call Stack (most recent call first):
  build_system/seqan3-config.cmake:319 (seqan3_config_error)
  CMakeLists.txt:32 (find_package)

-- Configuring incomplete, errors occurred!

i am sure git submodule update --init --recursive did fetch sdsl-lite but it seems to be not detected by cmake.

Sorry for the late, I found this issue but already Merged the PR, therefore the error log can be found here. You can commit this pkg on our repo directly.

Full log is here https://build.bioarchlinux.org/api/pkg/seqan3/log/1688923248

SGSSGene commented 1 year ago

I can reproduce this. There seems to be an error with seqan3 3.2.0 release. Looking at the cmake documentation, I am not sure why it is working on the current main branch. We will investigate.

eseiler commented 1 year ago

I can reproduce this. There seems to be an error with seqan3 3.2.0 release. Looking at the cmake documentation, I am not sure why it is working on the current main branch. We will investigate.

It's because of https://github.com/seqan/seqan3/commit/ceb2f32e9cf6c83a530a3000ce5eae97ad9d7f61. The sdsl was missing a header (for gcc>=13). Hence, the include check failed. gcc13 was not available when 3.2.0 was released.

SDSL commit: https://github.com/xxsds/sdsl-lite/commit/607019d52b10bb8eb399e6fc9578eaf0a150f9c4

starsareintherose commented 1 year ago

Now I fix it at https://github.com/BioArchLinux/Packages/commit/ad3a29272a81368d3f70f261d157961eee443ad0

@kbipinkumar Could you help me check it if it's okay

kbipinkumar commented 1 year ago

Now I fix it at BioArchLinux/Packages@ad3a292

@kbipinkumar Could you help me check it if it's okay

yeas. the package is being built just fine.

starsareintherose commented 1 year ago

okay, you can close it now.

Thanks for the repo programmer.