facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
31.44k stars 3.64k forks source link

[regression] cannot build GPU support for windows in v1.7.2 #2191

Open h-vetinari opened 2 years ago

h-vetinari commented 2 years ago

When building for conda-forge, I get:

 (base) D:\bld\faiss-split_1642028268414\work\_build_generic\faiss>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe" -gencode=arch=compute_52,code=\"sm_52,compute_52\" -gencode=arch=compute_60,code=\"sm_60,compute_60\" -gencode=arch=compute_61,code=\"sm_61,compute_61\" -gencode=arch=compute_70,code=\"sm_70,compute_70\" -gencode=arch=compute_75,code=\"sm_75,compute_75\" -gencode=arch=compute_80,code=\"compute_80,compute_80\" -gencode=arch=compute_80,code=\"sm_80,compute_80\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64" -x cu   -I"D:\bld\faiss-split_1642028268414\work" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include"     --keep-dir x64\Release -maxrregcount=0  --machine 64 --compile -cudart static -Xfatbin=-compress-all -std=c++17 -Xcompiler="/EHsc -Ob2"    -D_WINDOWS -DNDEBUG -DFAISS_MAIN_LIB -DFINTEGER=int -D"CMAKE_INTDIR=\"Release\"" -Dfaiss_EXPORTS -D_WINDLL -D_MBCS -DWIN32 -D_WINDOWS -DNDEBUG -DFAISS_MAIN_LIB -DFINTEGER=int -D"CMAKE_INTDIR=\"Release\"" -Dfaiss_EXPORTS -Xcompiler "/EHsc /W1 /nologo /O2 /Fdfaiss.dir\Release\vc142.pdb /FS   /MD /GR" -o faiss.dir\Release\GpuIcmEncoder.obj "D:\bld\faiss-split_1642028268414\work\faiss\gpu\GpuIcmEncoder.cu" 
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\functional(920): error : no instance of function template "std::_Invoker_ret<_Rx, false>::_Call [with _Rx=Concurrency::details::_Unit_type]" matches the argument list [D:\bld\faiss-split_1642028268414\work\_build_generic\faiss\faiss.vcxproj]
              argument types are: (std::decay_t<std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>>)
            detected during:
              instantiation of "_Rx std::_Func_impl_no_alloc<_Callable, _Rx, _Types...>::_Do_call(_Types &&...) [with _Callable=std::decay_t<std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>>, _Rx=Concurrency::details::_Unit_type, _Types=<>]" 
  (896): here
              instantiation of "std::_Func_impl_no_alloc<_Callable, _Rx, _Types...>::_Func_impl_no_alloc(_Other &&) [with _Callable=std::decay_t<std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>>, _Rx=Concurrency::details::_Unit_type, _Types=<>, _Other=std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>, <unnamed>=0]" 
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\xmemory(283): here
              instantiation of "_Ty *std::_Global_new<_Ty,_Types...>(_Types &&...) [with _Ty=std::_Func_impl_no_alloc<std::decay_t<std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>>, Concurrency::details::_Unit_type>, _Types=<std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>>]" 
  (1012): here
              instantiation of "void std::_Func_class<_Ret, _Types...>::_Reset(_Fx &&) [with _Ret=Concurrency::details::_Unit_type, _Types=<>, _Fx=std::remove_reference_t<lambda []()->Concurrency::details::_Unit_type &>]" 
  (1143): here
              instantiation of "std::function<_Fty>::function(_Fx) [with _Fty=Concurrency::details::_Unit_type (), _Fx=lambda []()->Concurrency::details::_Unit_type, <unnamed>=int]" 
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\ppltasks.h(2348): here

  1 error detected in the compilation of "D:/bld/faiss-split_1642028268414/work/faiss/gpu/GpuIcmEncoder.cu".
  GpuIcmEncoder.cu

AFAICT, GpuIcmEncoder.{cu,h} was added between 1.7.1 & 1.7.2, in #1978. CC @KinglittleQ

The error seems to only appear for CUDA=11.{0,1}; for 10.2 and 11.2 the build succeeds.

h-vetinari commented 2 years ago

Ping @mdouze @wickedfoo @beauby @KinglittleQ

zeyus commented 11 months ago

I have a possibly related issue with CUDA 12.2

 3>CudaBuildCore:

         (R:\envs\mamba\faiss) R:\github_projects\faiss\build\faiss>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\bin\nvcc.exe"  --use-local-env -ccbin "C:\tools\BuildTools\VC
         \Tools\MSVC\14.38.33130\bin\HostX64\x64" -x cu   -IR:\github_projects\faiss -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\include" -I"C:\Program Files\NVIDIA GPU Co
         mputing Toolkit\CUDA\v12.2\include"     --keep-dir x64\Release  -maxrregcount=0   --machine 64 --compile -cudart static -std=c++17 --generate-code=arch=compute_61,code=[compute_6
         1,sm_61] -Xfatbin=-compress-all --expt-extended-lambda --expt-relaxed-constexpr -Xcompiler="/EHsc -Ob2"   -D_WINDOWS -DNDEBUG -DFAISS_MAIN_LIB -DFINTEGER=int -D"CMAKE_INTDIR=\"Re
         lease\"" -Dfaiss_EXPORTS -D_WINDLL -D_MBCS -DWIN32 -D_WINDOWS -DNDEBUG -DFAISS_MAIN_LIB -DFINTEGER=int -D"CMAKE_INTDIR=\"Release\"" -Dfaiss_EXPORTS -Xcompiler "/EHsc /W1 /nologo
         /O2 /FS   /MD " -Xcompiler "/Fdfaiss.dir\Release\vc143.pdb" -o faiss.dir\Release\template_faissgpuCodec01_faissgpuL2Distance_128_1_1.obj "R:\github_projects\faiss\build\faiss\gpu
         \template_faissgpuCodec01_faissgpuL2Distance_128_1_1.cu"
     3>R:/github_projects/faiss\faiss/gpu/impl/PQCodeDistances-inl.cuh(539): error : expected an expression [R:\github_projects\faiss\build\faiss\faiss.vcxproj]
                   auto outDistancesCodeViewCols = outCodeDistancesView.view<2>({coarseIndices.getSize(0) * coarseIndices.getSize(1), outCodeDistances.getSize(2) * outCodeDistances.getSiz
         e(3)});
                                                                                ^

         1 error detected in the compilation of "R:/github_projects/faiss/faiss/gpu/impl/IVFPQ.cu".
         IVFPQ.cu

cmake command:

cmake -DFAISS_ENABLE_GPU=ON -DFAISS_ENABLE_PYTHON=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DFAISS_OPT_LEVEL=avx2 -DCMAKE_CUDA_ARCHITECTURES="61" -DCUDAToolkit_ROOT="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2" -DBLA_VENDOR=NVHPC "-DMKL_LIBRARIES=R:/github_projects/vcpkg/packages/intel-mkl_x64-windows/lib/intel64/mkl_intel_thread_dll.lib;R:/github_projects/vcpkg/packages/intel-mkl_x64-windows/lib/intel64/mkl_intel_ilp64_dll.lib;R:/github_projects/vcpkg/packages/intel-mkl_x64-windows/lib/intel64/mkl_core_dll.lib" "-DBLAS_LIBRARIES=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cublas.lib;C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cublasLt.lib" -DFAISS_ENABLE_C_API=ON -DBUILD_TESTING=OFF -DPython_LIBRARIES=R:/envs/mamba/faiss/libs/python311.lib -DPython_EXECUTABLE=R:/envs/mamba/faiss/python.exe "-DCMAKE_TOOLCHAIN_FILE=R:/github_projects/vcpkg/scripts/buildsystems/vcpkg.cmake" -B build .

cmake --build build --config Release --target faiss -j4 --verbose