I spent 4 days trying to compile the C++ library of this library, and this is the only solution that worked.

[EternalSaga]

Here is the Dockerfile which can build the C++ library successfully, but it requires several workarounds for successful building. Please make it easier for C++ inference. I'd appreciate any suggestions for streamlining the process.

FROM nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04 AS build_stage
ENV TZ=Asia/Shanghai
ENV DEBIAN_FRONTEND=noninteractive
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update && apt install -y lsb-release wget software-properties-common gnupg 
RUN sed -i 's/archive.ubuntu.com/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list
RUN apt update && apt install lsb-release wget software-properties-common gnupg -y && apt update && apt upgrade -y && apt install -y vim wget gdb pkg-config autoconf automake build-essential git libpoco-dev uuid-dev libncurses5-dev python3-dev python3-pip \
    libtool libc6 libc6-dev unzip libnuma1 libnuma-dev libturbojpeg0-dev apt-utils \
    && python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip

WORKDIR /workspace
RUN git clone https://github.com/traveller59/spconv.git && git clone https://github.com/FindDefinition/cumm
# set environmnet variables for building cumm
ENV CUMM_CUDA_VERSION=12.2
ENV CUMM_DISABLE_JIT=1
ENV SPCONV_DISABLE_JIT=1
ENV CUMM_INCLUDE_PATH=/workspace/cumm/include
ENV CUMM_CUDA_ARCH_LIST="8.9"
# install pccm for building cumm wheel
RUN pip3 install wheel pccm -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN cd /workspace/cumm && git checkout v0.5.3 && python3 setup.py bdist_wheel && pip3 install dist/cumm_cu122-0.5.3-cp310-cp310-linux_x86_64.whl
# remove cumm depandance for building spconv
COPY pyproject.toml /workspace/spconv/pyproject.toml
ENV PATH=/usr/local/cuda-12.2/bin${PATH:+:${PATH}}
ENV LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
RUN apt install cmake -y
# It must build spconv wheel first, then use cmake to build c++ shared library.
# for cmake, I have to add gcc flags to make it could find headers and cudart.so
RUN cd /workspace/spconv && python3 setup.py bdist_wheel && python3 -m spconv.gencode --include=./cpp/include --src=./cpp/src && \
    cd cpp/ && mkdir build && cd src && cmake -DCMAKE_CXX_FLAGS="-I/workspace/spconv/cpp/include -L/usr/local/cuda/lib64/" . -B ../build && \
    cmake --build ../build --config Release -j 8

FROM scratch AS export
COPY --from=build_stage /workspace/spconv/cpp .

Please notice that the pyproject.toml file should be prepared first which used to be copied into the building container.

Then run:

docker build -t spconv:x86 --output ./out .

You can get the libspconv.so in your output direcotry.

[barrydoooit]

Hi @EternalSaga, Thanks for the script. I tried to put your commands into a bash script like:

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

#git clone https://github.com/FindDefinition/cumm.git $SCRIPT_DIR/cumm

export CUMM_CUDA_VERSION=11.4 # cuda version, required but only used for flag selection when build libspconv.
export CUMM_DISABLE_JIT=1
export SPCONV_DISABLE_JIT=1
export CUMM_INCLUDE_PATH=$SCRIPT_DIR/cumm/include # if you use cumm as a subdirectory, you need this to find cumm includes.
export CUMM_CUDA_ARCH_LIST="8.6" # cuda arch flags

pip3 install wheel pccm -i https://pypi.tuna.tsinghua.edu.cn/simple
cd $SCRIPT_DIR/cumm && git checkout v0.3.7 && python3 setup.py bdist_wheel # cumm has already been installed

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

cd $SCRIPT_DIR/spconv && python3 setup.py bdist_wheel && python3 -m spconv.gencode --include=./cpp/include --src=./cpp/src && \
    cd cpp/  && cd src && cmake -DCMAKE_CXX_FLAGS="-I/builds/libspconv/spconv/cpp/include -L/usr/local/cuda/lib64/" . -B ../build && \
    cmake --build ../build --config Release -j 8

but got the following error about accessing the header files (though they do exist). Have you ever encountered such issues? I suppose there would be some additional settings in CMakeLists.

-- Configuring done -- Generating done -- Build files have been written to: /builds/libspconv/spconv/cpp/build [ 1%] Building CXX object spconvlib/cumm/conv/main/ConvMainUnitTest/CMakeFiles/spconvlib_cumm_conv_main_ConvMainUnitTest.dir/ConvMainUnitTest_extract_mnk.cc.o [ 2%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/StaticAllocator/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_StaticAllocator.dir/StaticAllocator_StaticAllocator.cc.o [ 2%] Building CXX object spconvlib/cumm/gemm/main/GemmMainUnitTest/CMakeFiles/spconvlib_cumm_gemm_main_GemmMainUnitTest.dir/GemmMainUnitTest_get_all_algo_desp.cc.o [ 3%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/ExternalAllocatorGuard/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_ExternalAllocatorGuard.dir/ExternalAllocatorGuard_ExternalAllocatorGuard.cc.o [ 4%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_ExternalAllocator.dir/ExternalAllocator_zeros_guard.cc.o [ 4%] Building CXX object spconvlib/spconv/csrc/sparse/convops/GemmTuneResult/CMakeFiles/spconvlib_spconv_csrc_sparse_convops_GemmTuneResult.dir/GemmTuneResult_is_valid.cc.o [ 5%] Building CXX object spconvlib/cumm/common/CompileInfo/CMakeFiles/spconvlib_cumm_common_CompileInfo.dir/CompileInfo_get_compiled_cuda_arch.cc.o [ 6%] Building CXX object spconvlib/spconv/csrc/sparse/convops/gemmops/GemmTunerSimple/CMakeFiles/spconvlib_spconv_csrc_sparse_convops_gemmops_GemmTunerSimple.dir/GemmTunerSimple_GemmTunerSimple.cc.o /builds/libspconv/spconv/cpp/src/spconvlib/cumm/common/CompileInfo/CompileInfo_get_compiled_cuda_arch.cc:1:10: fatal error: spconvlib/cumm/common/CompileInfo.h: No such file or directory 1 | #include <spconvlib/cumm/common/CompileInfo.h> | ^~~~~~~~~ /builds/libspconv/spconv/cpp/src/spconvlib/cumm/conv/main/ConvMainUnitTest/ConvMainUnitTest_extract_mnk.cc:1:10: fatal error: spconvlib/cumm/conv/main/ConvMainUnitTest.h: No such file or directory 1 | #include <spconvlib/cumm/conv/main/ConvMainUnitTest.h> | ^~~~~~~~~~~~~ /builds/libspconv/spconv/cpp/src/spconvlib/cumm/gemm/main/GemmMainUnitTest/GemmMainUnitTest_get_all_algo_desp.cc:1:10: fatal error: spconvlib/cumm/gemm/main/GemmMainUnitTest.h: No such file or directory 1 | #include <spconvlib/cumm/gemm/main/GemmMainUnitTest.h> | ^~~~~~~~~~~~~ /builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/alloc/StaticAllocator/StaticAllocator_StaticAllocator.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/alloc/StaticAllocator.h: No such file or directory 1 | #include <spconvlib/spconv/csrc/sparse/alloc/StaticAllocator.h> | ^~~~~~~~~~~~~~ /builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator/ExternalAllocator_zeros_guard.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator.h: No such file or directory 1 | #include <spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator.h> | ^~~~~~~~~~~~ compilation terminated. /builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/convops/GemmTuneResult/GemmTuneResult_is_valid.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/convops/GemmTuneResult.h: No such file or directory 1 | #include <spconvlib/spconv/csrc/sparse/convops/GemmTuneResult.h> | ^~~~~~~~~~~~~~~

[barrydoooit]

I found out that I forgot to prepare spconv/CMakeLists.txt. It works like a charm with this repo's example:

cmake_minimum_required(VERSION 3.20)

project(spconv LANGUAGES CXX CUDA)
include_directories(/builds/libspconv/include)

add_subdirectory(src)
# tell parent spconv2 include path.
set(SPCONV2_INCLUDE_PATH /builds/libspconv/spconv/include PARENT_SCOPE)

install (TARGETS spconv
         ARCHIVE DESTINATION lib
         LIBRARY DESTINATION lib
         RUNTIME DESTINATION bin)

[EternalSaga]

@barrydoooit Thank you for your suggestion. I didn't notice that there is a CMakeLists.txt in this library. I use the cmake cli arguments ("-DCMAKE_CXX_FLAGS="-I/workspace/spconv/cpp/include -L/usr/local/cuda/lib64/") as a workaround. If you use my solution, please make sure you use the correct include directory path and the cuda library path.

[EternalSaga]

A little bit improvements with barrydoooit's suggestion.

FROM nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04 AS basebuild
ENV TZ=Asia/Shanghai
ENV DEBIAN_FRONTEND=noninteractive
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update && apt install -y lsb-release wget software-properties-common gnupg 
RUN sed -i 's/archive.ubuntu.com/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list
RUN apt update && apt install lsb-release wget software-properties-common gnupg -y && apt update && apt upgrade -y && apt install -y vim wget gdb pkg-config autoconf automake build-essential git libpoco-dev uuid-dev libncurses5-dev python3-dev python3-pip \
    libtool libc6 libc6-dev unzip libnuma1 libnuma-dev libturbojpeg0-dev apt-utils \
    && python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1A127079A92F09ED && apt-add-repository 'deb https://apt.kitware.com/ubuntu/ jammy main' && apt update && apt install -y cmake
WORKDIR /workspace

FROM basebuild as build_stage
RUN git clone https://github.com/traveller59/spconv.git && git clone https://github.com/FindDefinition/cumm
# set environmnet variables for building cumm
ENV CUMM_CUDA_VERSION=12.2
ENV CUMM_DISABLE_JIT=1
ENV SPCONV_DISABLE_JIT=1
ENV CUMM_INCLUDE_PATH=/workspace/cumm/include
ENV CUMM_CUDA_ARCH_LIST="8.9"
# install pccm for building cumm wheel
RUN pip3 install wheel pccm -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN cd /workspace/cumm && git checkout v0.5.3 && python3 setup.py bdist_wheel && pip3 install dist/cumm_cu122-0.5.3-cp310-cp310-linux_x86_64.whl
# remove cumm depandance for building spconv
COPY pyproject.toml /workspace/spconv/pyproject.toml
ENV PATH=/usr/local/cuda-12.2/bin${PATH:+:${PATH}}
ENV LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
RUN cd /workspace/spconv && mkdir ./cpp

# from example/libspconv/spconv/CMakeLists.txt
COPY CMakeLists.txt /workspace/spconv/cpp/CMakeLists.txt
RUN cd /workspace/spconv && python3 setup.py bdist_wheel && python3 -m spconv.gencode --include=./cpp/include --src=./cpp/src
RUN cd /workspace/spconv/cpp/ && mkdir build && cmake . -B ./build && \
    cmake --build ./build --config Release --parallel $(nproc) --target install
RUN mkdir -p ./libspconv/lib && mkdir -p ./libspconv/include
# make sure the cumm headers have been moved out with spconv heads
RUN mv /usr/local/lib/libspconv.so ./libspconv/lib && mv /workspace/spconv/cpp/include ./libspconv/ && mv /workspace/cumm/include/tensorview ./libspconv/include/

FROM basebuild
RUN apt install -y gcc-12 g++-12
RUN wget https://apt.llvm.org/llvm.sh && chmod +x ./llvm.sh && ./llvm.sh 18 -s && rm ./llvm.sh 
COPY --from=build_stage /workspace/libspconv /usr/local/libspconv