phylo42 / IPK

Inference of phylo-k-mers
MIT License
4 stars 1 forks source link

xpas::load() hangs forever on some databases #8

Open blinard-BIOINFO opened 2 years ago

blinard-BIOINFO commented 2 years ago

I observed this weird behaviour a few times when loading tens of thousands of DB for clapas.

  1. xpas v0.3.1
  2. I call xpas::load() on some rps database
  3. it hangs forever, and never finishes to load.
  4. this happens only on the the cluster AND NOT on my local machine (same database file, it loads in 2 seconds).

To reproduce on lamarck :

# load env

conda activate /ngs/linard/conda/
source /ngs/nromashchenko/load_env.sh

# build problematic database

/beegfs/linard/CLAPPAS/clapas_core/lib/xpas/build/xpas-build-aa-pos \
 --merge-branches \
-k 5 -o 5.0 \
--ar-binary "$(which raxml-ng)" \
--refalign /beegfs/linard/CLAPPAS/33208/pruned/3B9RA/Ax/A0_nx0_la.align \
--reftree /beegfs/linard/CLAPPAS/33208/pruned/3B9RA/Tx/T0_nx0_la.tree.raxml.bestTree.rerooted \
--ar-dir /beegfs/linard/CLAPPAS/33208/pruned/3B9RA/Dx/A0_nx0_la/k5_o5.00/AR \
--workdir /beegfs/linard/CLAPPAS/33208/pruned/3B9RA/Dx/A0_nx0_la/k5_o5.00 \
--uncompressed

# compile target xpas_example_stats, then :

xpas_example_stats /beegfs/linard/CLAPPAS/33208/pruned/3B9RA/Dx/A0_nx0_la/k5_o5.00/DB_k5_o5.0.rps

# will hang forever with 100% CPU usage and no RAM increase ... 
blinard-BIOINFO commented 2 years ago

@nromashchenko

TEST 1 : 0) modified CMakelist to link xpas_aa_pos in recipe xpas_example_stats 1) binary xpas_example_stats built on lamarck 2) copy to local desktop computer 3) ldd shows

    linux-vdso.so.1 (0x00007ffd269dd000)
    libboost_serialization.so.1.74.0 => not found
    libboost_iostreams.so.1.74.0 => not found
    libboost_system.so.1.74.0 => not found
    libboost_filesystem.so.1.74.0 => not found
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa88bbc7000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa88b9e5000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa88b894000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa88b879000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa88b687000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fa88bc01000)

4) installation of boost 1.74 5) execution on same database file => hangs forever on local desktop computer, behaviour of lamarck reproduced

CONCLUSION ==> it is the binary built on lamarck side that is faulty. Issue related to boost 1.74 ?

TEST 2 : 0) modified CMakelist to link xpas_aa_pos in recipe xpas_example_stats 1) xpas_example_stats build on local machine with local boost version (1.71). 2) ldd shows

    linux-vdso.so.1 (0x00007ffc5e9b9000)
    libboost_serialization.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_serialization.so.1.71.0 (0x00007fa210dd7000)
    libboost_iostreams.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_iostreams.so.1.71.0 (0x00007fa210db0000)
    libboost_filesystem.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_filesystem.so.1.71.0 (0x00007fa210d92000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa210bb0000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa210b95000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa2109a1000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa21097e000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa210962000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fa21094f000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fa210926000)
    libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fa21087d000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa21072c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fa210e63000)

3) now it loads the database