CMake error during installation from source

[sunhongmin225]

Hi, I am trying the installation from source following the instruction in this document: https://github.com/CannyLab/tsne-cuda/blob/master/INSTALL.md

I've made sure to meet all the requirements you've written.

Especially, I've installed the latest version of Faiss (v1.7.2) using conda install -c pytorch faiss-gpu command. I've tested that python -c "import faiss" works without problems.

However, when I try the cMake .. command in build/, below error is returned.

-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The CUDA compiler identification is NVIDIA 10.1.243
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /usr/include (found version "10.1.243")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found GTest: /usr/lib/x86_64-linux-gnu/libgtest.a
-- Found ZMQ: /usr/lib/x86_64-linux-gnu/libzmq.so
CMake Error at /usr/local/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find FAISS (missing: FAISS_INCLUDE_DIR FAISS_GPU_INCLUDE_DIR
  FAISS_LIBRARY)
Call Stack (most recent call first):
  /usr/local/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  cmake/Modules/FindFAISS.cmake:21 (find_package_handle_standard_args)
  CMakeLists.txt:154 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeOutput.log".
See also "/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeError.log".

Below is the content of build/CMakeFiles/CMakeError.log.

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make -f Makefile cmTC_47669/fast && /usr/bin/make  -f CMakeFiles/cmTC_47669.dir/build.make CMakeFiles/cmTC_47669.dir/build
make[1]: Entering directory '/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_47669.dir/src.c.o
/usr/bin/cc -DCMAKE_HAVE_LIBC_PTHREAD  -fPIE -o CMakeFiles/cmTC_47669.dir/src.c.o -c /home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_47669
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_47669.dir/link.txt --verbose=1
/usr/bin/cc CMakeFiles/cmTC_47669.dir/src.c.o -o cmTC_47669
/usr/bin/ld: CMakeFiles/cmTC_47669.dir/src.c.o: in function `main':
src.c:(.text+0x3e): undefined reference to `pthread_create'
/usr/bin/ld: src.c:(.text+0x4a): undefined reference to `pthread_detach'
/usr/bin/ld: src.c:(.text+0x56): undefined reference to `pthread_cancel'
/usr/bin/ld: src.c:(.text+0x67): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_47669.dir/build.make:99: cmTC_47669] Error 1
make[1]: Leaving directory '/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_47669/fast] Error 2

Source file was:
#include <pthread.h>

static void* test_func(void* data)
{
  return data;
}

int main(void)
{
  pthread_t thread;
  pthread_create(&thread, NULL, test_func, NULL);
  pthread_detach(thread);
  pthread_cancel(thread);
  pthread_join(thread, NULL);
  pthread_atfork(NULL, NULL, NULL);
  pthread_exit(NULL);

  return 0;
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make -f Makefile cmTC_fff13/fast && /usr/bin/make  -f CMakeFiles/cmTC_fff13.dir/build.make CMakeFiles/cmTC_fff13.dir/build
make[1]: Entering directory '/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_fff13.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=pthread_create -fPIE -o CMakeFiles/cmTC_fff13.dir/CheckFunctionExists.c.o -c /usr/local/share/cmake-3.22/Modules/CheckFunctionExists.c
Linking C executable cmTC_fff13
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_fff13.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_fff13.dir/CheckFunctionExists.c.o -o cmTC_fff13  -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_fff13.dir/build.make:99: cmTC_fff13] Error 1
make[1]: Leaving directory '/home2/sunhong/tsne-cuda/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:127: cmTC_fff13/fast] Error 2

Could you help me out how to fix this error? Thanks.

[DavidMChan]

The issue here is that our cmake script can't find the conda installation of FAISS (since it's not installed in the standard location, due to the conda environment). If you want to get this working, you need to locate the include and library files in conda, and set the required paths using FAISS_INCLUDE_DIR, FAIS_GPU_INCLUDE_DIR and FAISS_LIBRARY.

If you just want to use tsnecuda with conda, I recommend 'conda install tsnecuda -c conda-forge' or if you want to build tsnecuda from scratch, install faiss from source as well.

[sunhongmin225]

Thanks for a quick reply. However, I couldn't find the location of the Faiss library files. May I ask you to kindly inform me of the contents of FAISS_INCLUDE_DIR, FAIS_GPU_INCLUDE_DIR and FAISS_LIBRARY, by echoing them in your system? I am using an Ubuntu 20.04.4 LTS machine.

[DavidMChan]

If they are in the standard location, then they will be found by our script, however, when using a conda install, this path will be unique to your machine (depending on your conda env name, conda install location, username, etc.) so I can't provide any insight about where they're located.

[sunhongmin225]

I've followed your advice -- installed Faiss from source, and have successfully installed tsne-cuda. Thanks for your help.