search

nanoporetech / dorado

Oxford Nanopore's Basecaller
https://nanoporetech.com/
Other
488 stars 59 forks source link

DORADO 0.41 compile issue on Ubuntu 22.04 LTS #442

Closed cement-head closed 4 months ago

cement-head commented 11 months ago

dorado_error.txt Looks as though there's a bug with PyTorch that crashes the last steps..Error 1 and Error 2 when compiling at the very last step - see output below.


- Downloading pod5-0.2.2-Linux - done
-- Extracting pod5-0.2.2-Linux
-- Extracting pod5-0.2.2-Linux - done
-- Found CUDA: /usr/local/cuda-12.3 (found version "12.3") 
-- Found Python3: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
-- Downloading torch-2.0.0-ont.1-cxx11-abi-static-Linux from https://cdn.oxfordnanoportal.com/software/analysis/torch-2.0.0.1-linux-x64-ont-cxx11-abi.zip
-- Downloading torch-2.0.0-ont.1-cxx11-abi-static-Linux - done
-- Extracting torch-2.0.0-ont.1-cxx11-abi-static-Linux
-- Extracting torch-2.0.0-ont.1-cxx11-abi-static-Linux - done
CMake Warning at dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library foxi_loader_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:106 (append_torchlib_if_found)
  cmake/Torch.cmake:188 (find_package)
  CMakeLists.txt:60 (include)

CMake Warning at dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library fmt_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:106 (append_torchlib_if_found)
  cmake/Torch.cmake:188 (find_package)
  CMakeLists.txt:60 (include)

CMake Warning at dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library eigen_blas_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:113 (append_torchlib_if_found)
  cmake/Torch.cmake:188 (find_package)
  CMakeLists.txt:60 (include)

CMake Warning at dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library mkldnn_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)
  cmake/Torch.cmake:188 (find_package)
  CMakeLists.txt:60 (include)

CMake Warning at dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library asmjit_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/share/cmake/Torch/TorchConfig.cmake:123 (append_torchlib_if_found)
  cmake/Torch.cmake:188 (find_package)
  CMakeLists.txt:60 (include)

-- Found Torch: /home/cbfgws8/Programs/dorado/dorado/3rdparty/torch-2.0.0-ont.1-cxx11-abi-static-Linux/libtorch/lib/libtorch.a  
-- TORCH_BUILD_VERSION: 2.0.0-ont
-- Downloading openssl3-Linux-x86_64 from https://cdn.oxfordnanoportal.com/software/analysis/openssl3-linux-x86_64.zip
-- Downloading openssl3-Linux-x86_64 - done
-- Extracting openssl3-Linux-x86_64
-- Extracting openssl3-Linux-x86_64 - done```
tijyojwad commented 10 months ago

From the full logs it looks like the linker can't find some symbols for htslib

/usr/bin/ld: ../3rdparty/htslib/lib/libhts.a(bgzf.o): in function `bgzf_encode_level0_func':
/home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:1342: undefined reference to `libdeflate_crc32'
/usr/bin/ld: ../3rdparty/htslib/lib/libhts.a(bgzf.o): in function `bgzf_uncompress':
/home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:721: undefined reference to `libdeflate_alloc_decompressor'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:727: undefined reference to `libdeflate_deflate_decompress'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:728: undefined reference to `libdeflate_free_decompressor'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:735: undefined reference to `libdeflate_crc32'
/usr/bin/ld: ../3rdparty/htslib/lib/libhts.a(bgzf.o): in function `bgzf_compress':
/home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:604: undefined reference to `libdeflate_crc32'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:579: undefined reference to `libdeflate_alloc_compressor'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:584: undefined reference to `libdeflate_deflate_compress'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:596: undefined reference to `libdeflate_free_compressor'
/usr/bin/ld: /home/cbfgws8/Programs/dorado/cmake-build/htslib_build/htslib/bgzf.c:590: undefined reference to `libdeflate_free_compressor'

We ship pre-built binaries for dorado. The latest builds can be found here - https://github.com/nanoporetech/dorado#installation . That might be sufficient for your use case.

cement-head commented 10 months ago

@tijyojwad Yep, thanks - everntually did that (downloaded the pre-compiled binary).

Well, solved it - 18.04 doesn't have libdeflate, so I downloaded the packages from Focal Fossa (20.04).

URL: https://launchpad.net/ubuntu/+source/libdeflate/1.5-3/+build/18699119

cement-head commented 10 months ago

Nope, that did not solve the issue. I get the same problem on both an 18.04 machine and a 22.04 machine.

The true source of this problem is the ORDER of linking of the libraries.

tijyojwad commented 10 months ago

Hi @cement-head - can you elaborate on this issue? What change in ordering fixed it for you?

abadouh commented 8 months ago

Hi @cement-head , By any chance you've managed to solve the link issues?

vellamike commented 4 months ago

Closing due to inactivity.