search

Illumina / Isaac4

Isaac aligner version 4
Other
17 stars 3 forks source link

New AMD Server AMD EPYC 7532 configure errors with CMAKE_HAVE_LIBC_PTHREAD and errors with make #18

Closed BenoitFiset closed 2 years ago

BenoitFiset commented 2 years ago

Hi Roman,

new servers on Centos 8.4. Supports NUMA and has sse4_2

Configure finishes but there are error in CMakeError.log. Running make starts but dies with errors like c++/include/bgzf/BgzfReader.hh:98:19: error: expected ‘;’ at end of member declaration lower below you have the outputs of CMakeError.log and the output of make.

I tried different versions of gcc (10.3 and 9.3 and 8.4.0) and my own version of boost (which Isaac still doesn't like)... and the outcome is always the same. Do I have to go with and older version of gcc ? Would the PThread errors in configure cause the BgzfReader errors ?

Any clues ? Pointers ?

Thanks,

B

Command line for confgure:

../Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/configure --parallel=6 --verbose --with-numa --with-avx2 --prefix=/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19-bin

Configure finishes:

[SNIP]
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
[SNIP]

-- using compiler: gcc version 9.3.0
-- Found ccache: /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/ccache
-- Adding the c++    library subdirectory: bgzf
-- Adding the c++    library subdirectory: common
-- Adding the c++    library subdirectory: oligo
-- Adding the c++    library subdirectory: io
-- Adding the c++    library subdirectory: rta
-- Adding the c++    library subdirectory: flowcell
-- Adding the c++    library subdirectory: demultiplexing
-- Adding the c++    library subdirectory: xml
-- Adding the c++    library subdirectory: reference
-- Adding the c++    library subdirectory: bam
-- Adding the c++    library subdirectory: options
-- Adding the c++    library subdirectory: statistics
-- Adding the c++    library subdirectory: alignment
-- Adding the c++    library subdirectory: build
-- Adding the c++    library subdirectory: package
-- Adding the c++    library subdirectory: reports
-- Adding the c++    library subdirectory: workflow
-- Adding the c++    program subdirectory: bin
-- Adding the c++    program subdirectory: libexec
-- Found Doxygen: /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/doxygen (found version "1.8.16") found components: doxygen dot 
-- Doxygen: /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/doxygen. Dot: /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/dot.
-- Creating Doxygen config file: /home/bfiset/lw-project/Isaac_Build/c++/Doxyfile
-- Could NOT find Markdown (missing: MARKDOWN_EXECUTABLE) 
-- Markdown: MARKDOWN_EXECUTABLE-NOTFOUND.
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    iSAAC_DOCDIR
    iSAAC_INCLUDEDIR
    iSAAC_MANDIR

-- Build files have been written to: /home/bfiset/lw-project/Isaac_Build
The build directory /home/bfiset/lw-project/Isaac_Build was configured successfully

Type make at the top level of the root directory to build Isaac

But these errors in CMakeFiles/CMakeError.log

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Run Build Command(s):/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake cmTC_50487/fast && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake -f CMakeFiles/cmTC_50487.dir/build.make CMakeFiles/cmTC_50487.dir/build
gmake[1]: Entering directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_50487.dir/src.c.o
/cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/gcc   -DCMAKE_HAVE_LIBC_PTHREAD   -o CMakeFiles/cmTC_50487.dir/src.c.o   -c /home/bfiset/lw-project/Isaac_Build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_50487
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_50487.dir/link.txt --verbose=1
/cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/gcc  -DCMAKE_HAVE_LIBC_PTHREAD    -rdynamic CMakeFiles/cmTC_50487.dir/src.c.o  -o cmTC_50487 
CMakeFiles/cmTC_50487.dir/src.c.o:src.c:function main: error: undefined reference to 'pthread_create'
CMakeFiles/cmTC_50487.dir/src.c.o:src.c:function main: error: undefined reference to 'pthread_detach'
CMakeFiles/cmTC_50487.dir/src.c.o:src.c:function main: error: undefined reference to 'pthread_join'
collect2: error: ld returned 1 exit status
gmake[1]: *** [CMakeFiles/cmTC_50487.dir/build.make:87: cmTC_50487] Error 1
gmake[1]: Leaving directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:121: cmTC_50487/fast] Error 2

Source file was:
#include <pthread.h>

void* test_func(void* data)
{
  return data;
}

int main(void)
{
  pthread_t thread;
  pthread_create(&thread, NULL, test_func, NULL);
  pthread_detach(thread);
  pthread_join(thread, NULL);
  pthread_atfork(NULL, NULL, NULL);
  pthread_exit(NULL);

  return 0;
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/bfiset/lw-project/Isaac_Build/CMakeFiles/CMakeTmp

Run Build Command(s):/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake cmTC_7ec55/fast && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake -f CMakeFiles/cmTC_7ec55.dir/build.make CMakeFiles/cmTC_7ec55.dir/build
gmake[1]: Entering directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_7ec55.dir/CheckFunctionExists.c.o
/cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/gcc   -DCHECK_FUNCTION_EXISTS=pthread_create   -o CMakeFiles/cmTC_7ec55.dir/CheckFunctionExists.c.o   -c /cvmfs/soft.computecanada.ca/gentoo/2020/usr/share/cmake/M
odules/CheckFunctionExists.c
Linking C executable cmTC_7ec55
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_7ec55.dir/link.txt --verbose=1
/cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create    -rdynamic CMakeFiles/cmTC_7ec55.dir/CheckFunctionExists.c.o  -o cmTC_7ec55  -lpthreads 
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/x86_64-pc-linux-gnu/binutils-bin/2.33.1/ld.gold: error: cannot find -lpthreads
CMakeFiles/cmTC_7ec55.dir/CheckFunctionExists.c.o:CheckFunctionExists.c:function main: error: undefined reference to 'pthread_create'
collect2: error: ld returned 1 exit status
gmake[1]: *** [CMakeFiles/cmTC_7ec55.dir/build.make:87: cmTC_7ec55] Error 1
gmake[1]: Leaving directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:121: cmTC_7ec55/fast] Error 2

Determining if the include file cppunit/config-auto.h exists failed with the following output:
Change Dir: /home/bfiset/lw-project/Isaac_Build/CMakeFiles/CMakeTmp

Run Build Command(s):/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake cmTC_8ea56/fast && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/gmake -f CMakeFiles/cmTC_8ea56.dir/build.make CMakeFiles/cmTC_8ea56.dir/build
gmake[1]: Entering directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8ea56.dir/CheckIncludeFile.c.o
/cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/gcc    -o CMakeFiles/cmTC_8ea56.dir/CheckIncludeFile.c.o   -c /home/bfiset/lw-project/Isaac_Build/CMakeFiles/CMakeTmp/CheckIncludeFile.c
/home/bfiset/lw-project/Isaac_Build/CMakeFiles/CMakeTmp/CheckIncludeFile.c:1:10: fatal error: cppunit/config-auto.h: No such file or directory
    1 | #include <cppunit/config-auto.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
gmake[1]: *** [CMakeFiles/cmTC_8ea56.dir/build.make:66: CMakeFiles/cmTC_8ea56.dir/CheckIncludeFile.c.o] Error 1
gmake[1]: Leaving directory '/project/6014302/bfiset/Isaac_Build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:121: cmTC_8ea56/fast] Error 2

And running make goes kaboom !!!

make
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -S/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src -B/home/bfiset/lw-project/Isaac_Build --check-build-system CMakeFiles/Makefile.cmake 0
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_progress_start /home/bfiset/lw-project/Isaac_Build/CMakeFiles /home/bfiset/lw-project/Isaac_Build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/project/6014302/bfiset/Isaac_Build'
make -f CMakeFiles/iSAAC_OPT.dir/build.make CMakeFiles/iSAAC_OPT.dir/depend
make[2]: Entering directory '/project/6014302/bfiset/Isaac_Build'
cd /home/bfiset/lw-project/Isaac_Build && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src /home/bfiset/lw-project/Isaac_Build /home/bfiset/lw-project/Isaac_Build /home/bfiset/lw-project/Isaac_Build/CMakeFiles/iSAAC_OPT.dir/DependInfo.cmake --color=
Scanning dependencies of target iSAAC_OPT
make[2]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
make -f CMakeFiles/iSAAC_OPT.dir/build.make CMakeFiles/iSAAC_OPT.dir/build
make[2]: Entering directory '/project/6014302/bfiset/Isaac_Build'
make[2]: Nothing to be done for 'CMakeFiles/iSAAC_OPT.dir/build'.
make[2]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
[  0%] Built target iSAAC_OPT
make -f c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/build.make c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/depend
make[2]: Entering directory '/project/6014302/bfiset/Isaac_Build'
cd /home/bfiset/lw-project/Isaac_Build && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf /home/bfiset/lw-project/Isaac_Build /home/bfiset/lw-project/Isaac_Build/c++/lib/bgzf /home/bfiset/lw-project/Isaac_Build/c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/DependInfo.cmake --color=
Scanning dependencies of target isaac_bgzf
make[2]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
make -f c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/build.make c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/build
make[2]: Entering directory '/project/6014302/bfiset/Isaac_Build'
[  1%] Building CXX object c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/BgzfReader.cpp.o
cd /home/bfiset/lw-project/Isaac_Build/c++/lib/bgzf && /cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/ccache /cvmfs/soft.computecanada.ca/easybuild/software/2020/Core/gcccore/9.3.0/bin/g++   -I/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include -I/home/bfiset/lw-project/Isaac_Build/c++ -I/home/bfiset/lw-project/Isaac_Build/c++/lib/bgzf -I/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf -I/home/bfiset/lw-project/Isaac_Build/c++/common -isystem /cvmfs/soft.computecanada.ca/gentoo/2020/usr/include/libxml2 -isystem /home/bfiset/lw-project/Isaac_Build/bootstrap/include  -std=gnu++0x -fpermissive -fopenmp -Wall -Wextra -Wunused -Wno-long-long -Wsign-compare -Wpointer-arith -Wno-unused-local-typedefs -Wno-deprecated-declarations -DBOOST_SYSTEM_API_CONFIG_HPP -DBOOST_POSIX_API  -Wno-unused-parameter  -mavx2 -O2 -ftree-vectorize -finline-functions -fpredictive-commoning -fgcse-after-reload -funswitch-loops -ftree-slp-vectorize -fvect-cost-model -fipa-cp-clone -ftree-phiprop -ggdb    -o CMakeFiles/isaac_bgzf.dir/BgzfReader.cpp.o -c /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp
In file included from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:25:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:98:19: error: expected ‘;’ at end of member declaration
   98 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                   ^~~~~~
      |                         ;
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:98:29: error: expected identifier before ‘(’ token
   98 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                             ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:98:67: warning: ISO C++ forbids declaration of ‘OF’ with no type [-fpermissive]
   98 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                                                                   ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:105:17: error: variable or field ‘zfree’ declared void
  105 |     static void zfree OF((voidpf opaque, voidpf address))
      |                 ^~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:105:17: error: expected ‘;’ at end of member declaration
  105 |     static void zfree OF((voidpf opaque, voidpf address))
      |                 ^~~~~
      |                      ;
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:105:26: error: expected identifier before ‘(’ token
  105 |     static void zfree OF((voidpf opaque, voidpf address))
      |                          ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:105:57: warning: ISO C++ forbids declaration of ‘OF’ with no type [-fpermissive]
  105 |     static void zfree OF((voidpf opaque, voidpf address))
      |                                                         ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh: In member function ‘void isaac::bgzf::BgzfReader::reset()’:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:87:24: warning: invalid conversion from ‘voidpf’ {aka ‘void*’} to ‘alloc_func’ {aka ‘void* (*)(void*, unsigned int, unsigned int)’} [-fpermissive]
   87 |         strm_.zalloc = zalloc;
      |                        ^~~~~~
      |                        |
      |                        voidpf {aka void*}
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:88:23: error: ‘zfree’ was not declared in this scope; did you mean ‘free’?
   88 |         strm_.zfree = zfree;
      |                       ^~~~~
      |                       free
In file included from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Threads.hpp:26,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:29,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:25:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh: In member function ‘int isaac::bgzf::BgzfReader::OF(int (*)(voidpf, uInt, uInt))’:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:100:56: error: ‘opaque’ was not declared in this scope
  100 |         ISAAC_ASSERT_MSG(reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree, "Unexpected too many zalloc calls");
      |                                                        ^~~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Debug.hh:77:42: note: in definition of macro ‘ISAAC_ASSERT_MSG’
   77 | #define ISAAC_ASSERT_MSG(expr, msg) {if (expr) {} else \
      |                                          ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:101:63: error: ‘opaque’ was not declared in this scope
  101 |         ISAAC_ASSERT_MSG(sizeof(reinterpret_cast<BgzfReader*>(opaque)->zalbuffer[reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree]) >= items * size, "Unexpected buffer size passed to zalloc : size=" << size << " items=" << items);
      |                                                               ^~~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Debug.hh:77:42: note: in definition of macro ‘ISAAC_ASSERT_MSG’
   77 | #define ISAAC_ASSERT_MSG(expr, msg) {if (expr) {} else \
      |                                          ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:101:140: error: ‘items’ was not declared in this scope
  101 |         ISAAC_ASSERT_MSG(sizeof(reinterpret_cast<BgzfReader*>(opaque)->zalbuffer[reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree]) >= items * size, "Unexpected buffer size passed to zalloc : size=" << size << " items=" << items);
      |                                                                                                                                            ^~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Debug.hh:77:42: note: in definition of macro ‘ISAAC_ASSERT_MSG’
   77 | #define ISAAC_ASSERT_MSG(expr, msg) {if (expr) {} else \
      |                                          ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:101:148: error: ‘size’ was not declared in this scope
  101 |         ISAAC_ASSERT_MSG(sizeof(reinterpret_cast<BgzfReader*>(opaque)->zalbuffer[reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree]) >= items * size, "Unexpected buffer size passed to zalloc : size=" << size << " items=" << items);
      |                                                                                                                                                    ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Debug.hh:77:42: note: in definition of macro ‘ISAAC_ASSERT_MSG’
   77 | #define ISAAC_ASSERT_MSG(expr, msg) {if (expr) {} else \
      |                                          ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:101:148: note: suggested alternatives:
  101 |         ISAAC_ASSERT_MSG(sizeof(reinterpret_cast<BgzfReader*>(opaque)->zalbuffer[reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree]) >= items * size, "Unexpected buffer size passed to zalloc : size=" << size << " items=" << items);
      |                                                                                                                                                    ^~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/common/Debug.hh:77:42: note: in definition of macro ‘ISAAC_ASSERT_MSG’
   77 | #define ISAAC_ASSERT_MSG(expr, msg) {if (expr) {} else \
      |                                          ^~~~
In file included from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/range/functions.hpp:20,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/range/iterator_range_core.hpp:38,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/range/iterator_range.hpp:13,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/iostreams/traits.hpp:38,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/iostreams/pipeline.hpp:18,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/iostreams/detail/push.hpp:22,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/iostreams/filtering_stream.hpp:19,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:21:
/home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/range/size.hpp:55:5: note:   ‘boost::size’
   55 |     size(const SinglePassRange& rng)
      |     ^~~~
In file included from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/mpl/vector/aux_/size.hpp:17,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/mpl/vector/vector0.hpp:26,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/mpl/vector/vector10.hpp:18,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/mpl/vector/vector20.hpp:18,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/detail/returntype_deduction.hpp:13,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/traits.hpp:22,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/storage.hpp:26,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/../common/StaticVector.hh:24,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:27,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:25:
/home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/mpl/size_fwd.hpp:20:38: note:   ‘boost::mpl::size’
   20 | template< typename Sequence > struct size;
      |                                      ^~~~
In file included from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/detail/config.hpp:300,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/exception.hpp:19,
                 from /home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/storage.hpp:25,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/../common/StaticVector.hh:24,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:27,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:25:
/home/bfiset/lw-project/Isaac_Build/bootstrap/include/boost/numeric/ublas/detail/definitions.hpp:177:31: note:   ‘boost::numeric::ublas::dimension::size’
  177 |         typename E::size_type size (const E& e) {
      |                               ^~~~
In file included from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/lib/bgzf/BgzfReader.cpp:25:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:102:41: error: ‘opaque’ was not declared in this scope
  102 |         --reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree;
      |                                         ^~~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh: In member function ‘int isaac::bgzf::BgzfReader::OF(int (*)(voidpf, voidpf))’:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:109:41: error: ‘opaque’ was not declared in this scope
  109 |         ++reinterpret_cast<BgzfReader*>(opaque)->zalbufferFree;
      |                                         ^~~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.20.06.19/Isaac4-master/src/c++/include/bgzf/BgzfReader.hh:110:5: warning: no return statement in function returning non-void [-Wreturn-type]
  110 |     }
      |     ^
make[2]: *** [c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/build.make:66: c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/BgzfReader.cpp.o] Error 1
make[2]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
make[1]: *** [CMakeFiles/Makefile2:845: c++/lib/bgzf/CMakeFiles/isaac_bgzf.dir/all] Error 2
make[1]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
make: *** [Makefile:144: all] Error 2
rpetrovski commented 2 years ago

Hi, I can't help you with AMD. It appears that your libpthread is one of the problems. Please let me know if you find the fix.

I've checked that I can compile with gcc 9.3.1 under devtoolset-9 on Intel CentOS 7.7.1908. There are definitely some issues with gcc 10, but I don't have the capacity to deal with those.

I've added the fix for zalloc and zalfree. These should not produce the errors you have listed although it is still a puzzle why OF is not set. Accordint to zconf.h it should be dependent on STDC: 00132 00133 #ifndef OF / function prototypes / 00134 # ifdef STDC 00135 # define OF(args) args 00136 # else 00137 # define OF(args) () 00138 # endif 00139 #endif 00140

BenoitFiset commented 2 years ago

Hi Roman,

Thanks for the fix. One other file needs to be fixed for the "OF" issue. (see lower)

I fixed it locally on my end and make makes it to the end. make install is also ok. I used gcc 9.3.0.

I'll try to align some files and update if all goes well.

Quick question, what should I use instead of Isaac when I won't be able to compile it anymore (I don't like that thought :( !!!) ? Thinking down the road.. 

[100%] Built target mergeReferences
make[1]: Leaving directory '/project/6014302/bfiset/Isaac_Build'
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/bin/cmake -E cmake_progress_start /home/bfiset/lw-project/Isaac_Build/CMakeFiles 0

=============

The file to fix for the last OF issue is InflateGzipDecompressor.hh : 

Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:295:19: error: expected ‘;’ at end of member declaration
  295 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                   ^~~~~~
      |                         ;
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:295:29: error: expected identifier before ‘(’ token
  295 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                             ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:295:67: warning: ISO C++ forbids declaration of ‘OF’ with no type [-fpermissive]
  295 |     static voidpf zalloc OF((voidpf opaque, uInt items, uInt size))
      |                                                                   ^
In file included from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/FastqReader.hh:36,
                 from /home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/lib/io/FastqReader.cpp:24:
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:303:17: error: variable or field ‘zfree’ declared void
  303 |     static void zfree OF((voidpf opaque, voidpf address))
      |                 ^~~~~
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:303:17: error: expected ‘;’ at end of member declaration
  303 |     static void zfree OF((voidpf opaque, voidpf address))
      |                 ^~~~~
      |                      ;
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:303:26: error: expected identifier before ‘(’ token
  303 |     static void zfree OF((voidpf opaque, voidpf address))
      |                          ^
/home/bfiset/lw-project/Isaac4-Build-Isaac-04.01.02.22/Isaac4-master/src/c++/include/io/InflateGzipDecompressor.hh:303:57: warning: ISO C++ forbids declaration of ‘OF’ with no type [-fpermissive]
  303 |     static void zfree OF((voidpf opaque, voidpf address))
      |                                                         ^
BenoitFiset commented 2 years ago

Hi Roman,

Quick update did an align with the Example data in the build and all is good so far.

Will do a last update when I finish alignment of real data.

Test Command:

../Isaac4-Build-Isaac-04.01.02.22-bin/bin/isaac-align -r ../Isaac4-Build-Isaac-04.01.02.22-bin/share/Isaac-04.18.11.09/data/examples/PhiX/iGenomes/PhiX/NCBI/1993-04-28/Sequence/Chromosomes/phix.fa -b ../Isaac4-Build-Isaac-04.01.02.22-bin/share/Isaac-04.18.11.09/data/examples/PhiX/Fastq -f fastq --use-bases-mask y150,y150 -m40

End of output:

[SNIP]
022-02-01 16:12:17  [152e342097c0]  BAM file generated: /project/6014302/bfiset/Isaac_Test/./Aligned/Projects/default/default/sorted.bam
2022-02-01 16:12:17     [152e342097c0]  BAM index generated for /project/6014302/bfiset/Isaac_Test/./Aligned/Projects/default/default/sorted.bam
2022-02-01 16:12:17     [152e342097c0]  Generating Build statistics
2022-02-01 16:12:17     [152e342097c0]  Generating Build statistics done
2022-02-01 16:12:17     [152e342097c0]  Generating the BAM files done
2022-02-01 16:12:17     [152e342097c0]  md5 checksum for /project/6014302/bfiset/Isaac_Test/./Aligned/Projects/default/default/sorted.bam:8d1629827f1d524e7d028a53cdcaff68
2022-02-01 16:12:18     [152e342097c0]  Saving workflow state to "/project/6014302/bfiset/Isaac_Test/./Temp/AlignerState.txt"
2022-02-01 16:12:18     [152e342097c0]  Saving workflow state done to "/project/6014302/bfiset/Isaac_Test/./Temp/AlignerState.txt"
BenoitFiset commented 2 years ago

Hi Roman,

Works superbly !!! 26 Minutes to align a pair end 88G FASTQs with 64 CPUs and 250G of RAM. Can't ask for more.

Thanks again.

rpetrovski commented 2 years ago

Glad it worked out.

Answering your previous question, Illumina recommends using the DRAGEN platform. It uses FPGA to perform compute intensive bits and takes care of both alignment, variant calling and everything and anything your soul might desire. Should you not have access to FPGA hardware, there are options to run the data analysis in the cloud.

Alternatively, most of the DNA alignment implementation of DRAGEN is available as open source on github.

R.

BenoitFiset commented 2 years ago

Hi Roman,

did you see in the comments above that another file has to be modified (OF issue) to make this work ? The file to fix for the last OF issue is InflateGzipDecompressor.hh.

Are most of the features of Issac in Dragmap ? You mention of options, when no FPGA avail, to run analysis in the cloud, what are the options ?

Thanks,

B.

rpetrovski commented 2 years ago

Yep, pushed another change about 6 hours ago. Cheers for that.

Dragmap/DRAGEN use fundamentally different alignment algorithm. So, you will have different mappings in many situations. That said, high MAPQ alignments should be similar. At Illumina, we have mostly switched to DRAGEN which is why Isaac is not being developed anymore.

Of the notable Isaac things that are not available in Dragmap :

I'm not the right person to explain AWS DRAGEN. Let me know if you want to be referred to an appropriate CSR. If I google, I find this: https://aws.amazon.com/quickstart/architecture/illumina-dragen/

R.