search

SyneRBI / SIRF-SuperBuild

SIRF CMake SuperBuild
http://www.ccpsynerbi.ac.uk
Apache License 2.0
15 stars 17 forks source link

Build NiftyPET>=2 #494

Open paskino opened 3 years ago

paskino commented 3 years ago

Building NiftyPET>=2 will allow dropping python2 support, see https://github.com/UCL/STIR/issues/795, https://github.com/SyneRBI/SIRF-SuperBuild/issues/484

NiftyPET binaries could be installed via pip, or by integrating it in the SuperBuild.

Current issues for integration in SuperBuild:

paskino commented 3 years ago

Interestingly enough NiftyPET doesn't seem to build on my Ubuntu18.04 with python3:

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build/nipet && /usr/local/cuda/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(148): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(222): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(302): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(368): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(374): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(395): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(457): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(463): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(483): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(566): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(573): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(600): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

However, on another machine with CentOS and conda python it works. I checked the imports and they seem to be in order.

KrisThielemans commented 3 years ago

Create a NiftyPET issue? Or possibly @casperdcl knows immediatelly...

casperdcl commented 3 years ago

Your python3 also needs to have numpy installed (if you pip install "nipet>=2" this would be done for you)

paskino commented 3 years ago

@casperdcl I have numpy installed! That's installed with pip to the local user.

casperdcl commented 3 years ago

Yes but I presume you're using some sort of custom CMake wrapper so you need to find_package() first... have a look at https://github.com/NiftyPET/NIPET/blob/master/niftypet/CMakeLists.txt

casperdcl commented 3 years ago

Wait hang on... your output shows you're picking up numpy. My guess is it's just really old. Try pip install --upgrade --user numpy

paskino commented 3 years ago

Not that I don't want to do it, but this is what I've got currently: a pip installed local version of numpy 1.19.5

numpy.version
<module 'numpy.version' from '/home/ofn77899/.local/lib/python3.6/site-packages/numpy/version.py'>
>>> numpy.version.version
'1.19.5'
paskino commented 3 years ago

Just to make this clear @casperdcl I'm not building NiftyPET as part of SIRF-SuperBuild yet, but rather just the package itself using the instructions on the NiftyPET repo.

In the process I found out that the CUDA compiler must already be in the ${PATH} if CUDA is defined as one of the project's LANGUAGES.

Also, 

python3 -m pip install --upgrade --user numpy
Requirement already satisfied: numpy in ./.local/lib/python3.6/site-packages (1.19.5)
paskino commented 3 years ago

I downgraded numpy to 1.18.

I added a message(WARNING to the NiftyPET CMakeLists.txt to see that Python is found and includes are there, but it still fails in the same way.

Do you manage to build on ubuntu 18.04 with stock python3?

 cmake ../niftypet/ -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- The CUDA compiler identification is NVIDIA 10.1.168
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found Python3: /usr/bin/python3.6 (found version "3.6.9") found components: Interpreter Development NumPy Development.Module Development.Embed
CMake Warning at CMakeLists.txt:11 (message):
  Found /usr/include/python3.6m
  /home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include

-- Found CUDAToolkit: /usr/local/cuda/include (found version "10.1.168")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- CUDA architectures: 30
-- Build type: Release
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build
casperdcl commented 3 years ago

Curious.

I also don't understand your problem. You say that you've already pip installed nipet (so it's already built) and you're just using find_package & link libs functionality, yet you post a build error.

paskino commented 3 years ago

I haven't pip installed nipet, I've cloned the repo and started building as you suggest in the readme:


# ... or build & install directly with cmake
mkdir build && cd build
cmake ../niftypet && cmake --build . && cmake --install . --prefix /my/install/dir

For me it fails at --build stage.

Maybe this will help

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build/nipet && /usr/local/cuda/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
casperdcl commented 3 years ago

ah right.

Well this is what I've done on a clean Ubuntu machine (18.04 and 20.04):

sudo snap install --classic cmake
wget https://raw.githubusercontent.com/pypa/get-pip/master/get-pip.py
python3 get-pip.py --user
python3 -m pip install --user 'numpy==1.19.5'  # no need to specify version. just testing...
git clone https://github.com/NiftyPET/NIPET nipet
mkdir nipet/build
cd nipet/build
cmake ../niftypet && cmake --build .

If that doesn't work for you, then please do open an issue at https://github.com/NiftyPET/NIPET

paskino commented 3 years ago

So, 

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

I ran


sudo apt update
sudo apt upgrade
sudo apt install python3-dev
sudo apt install nvidia-cuda-toolkit
sudo snap install --classic cmake
wget https://raw.githubusercontent.com/pypa/get-pip/master/get-pip.py
python3 get-pip.py --user
python3 -m pip install --user 'numpy==1.19.5'  # no need to specify version. just testing...
git clone https://github.com/NiftyPET/NIPET nipet
mkdir nipet/build
cd nipet/build
cmake ../niftypet && cmake --build .

Error

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/nipet/build/nipet && /usr/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/nipet/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/nipet/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/nipet/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(84): error: expected a "}"

/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(89): warning: parsing restarts here after previous syntax error

/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(450): error: identifier "NPY_NTYPES_ABI_COMPATIBLE" is undefined

2 errors detected in the compilation of "/tmp/tmpxft_00005c59_00000000-6_aux_module.cpp1.ii".

I then tried to install CUDA Toolkit from NVidia, and got 10.1. With it it works, both with conda and stock python.

Maybe 9.1, which is installed by Ubuntu is not sufficient for NiftyPET?

paskino commented 3 years ago

There must be more than this going on. On my old machine it still fails with the old message, even after upgrading to 10.1 from NVIDIA, rather than from ubuntu.