fail to build release 5.2rc03 on ArchLinux

hubutui commented 3 years ago

Error log:

CMake Error at Wrapping/CMakeUtilityFunctions.cmake:40 (math):
  math cannot parse the expression: ")

  set(ITK_WRAP_VECTOR_COMPONENTS + 1": syntax error, unexpected
  exp_CLOSEPARENT (1).
Call Stack (most recent call first):
  Wrapping/WrapBasicTypes.cmake:55 (INCREMENT)
  Wrapping/ConfigureWrapping.cmake:131 (include)
  Wrapping/CMakeLists.txt:97 (include)

CMake Warning (dev) at Wrapping/CMakeUtilityFunctions.cmake:40 (math):
  Unexpected character in expression at position 1: "

Call Stack (most recent call first):
  Wrapping/WrapBasicTypes.cmake:63 (INCREMENT)
  Wrapping/ConfigureWrapping.cmake:131 (include)
  Wrapping/CMakeLists.txt:97 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at Wrapping/WrapBasicTypes.cmake:73 (message):
  ITK_WRAP_VECTOR_COMPONENTS must include ITK_WRAP_IMAGE_DIMS
Call Stack (most recent call first):
  Wrapping/ConfigureWrapping.cmake:131 (include)
  Wrapping/CMakeLists.txt:97 (include)

See the complete build log here

thewtex commented 3 years ago

How is the build invoked?

hubutui commented 3 years ago

  export CC=clang
  export CXX=clang++
  python setup.py build

I use clang instead of gcc, because I used to build python-itk 5.1.2 with clang.

thewtex commented 3 years ago

Does the same issue occur with the v5.2.0 tag (the current release branch)?

hubutui commented 3 years ago

yes, check the latest build log for v5.2.0 here.

thewtex commented 3 years ago

@hubutui , please try this branch https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/156

hubutui commented 3 years ago

I got this error when building with clang 11.1.0:

CMake Error at /usr/share/cmake-3.20/Modules/CheckIPOSupported.cmake:63 (message):
  IPO is not supported (check failed to compile).

check the complete build log here

I could build itk with gcc 10.2.0. I create pkg for ArchLinux with:

python setup.py build
python setup.py install --root="${pkgdir}" --optimize=1 --skip-build

But I fail to read example image with itk.imread(), error output:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
    img = itk.imread('/home/hubutui/tmpdir/MRBrainTumor2.nrrd')
  File "/usr/lib/python3.9/site-packages/itk/support/extras.py", line 957, in imread
    reader = template_reader_type.New(**kwargs)
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 660, in New
    return self._NewImageReader(
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 191, in _NewImageReader
    ImageType = itk.Image[PixelType, dimension]
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 529, in __getitem__
    raise itk.TemplateTypeError(self, key)
itk.support.extras.TemplateTypeError: itk.Image is not wrapped for input type `itk.SS, int`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.Image.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.Image[itk.RGBPixel[itk.UC], int].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.RGBPixel[itk.UC]
itk.RGBAPixel[itk.UC]
itk.Vector[itk.F,2]
itk.Vector[itk.F,4]
itk.CovariantVector[itk.F,2]
itk.CovariantVector[itk.F,4]
itk.SS
itk.UC
itk.US
itk.F
itk.D
itk.complex[itk.F]
itk.Vector[itk.D,2]
itk.Vector[itk.D,4]
itk.SI
itk.UI
itk.UL
itk.ULL
itk.B
itk.FixedArray[itk.F,2]
itk.FixedArray[itk.D,2]
itk.Offset[2]
itk.SymmetricSecondRankTensor[itk.D,2]
itk.Vector[itk.D,2]
itk.Vector[itk.D,4]
itk.Vector[itk.F,1]
itk.CovariantVector[itk.D,2]
itk.CovariantVector[itk.D,4]
itk.NormalBandNode[itk.Image[itk.F,2]]
itk.NormalBandNode[itk.Image[itk.D,2]]

and itk.Image.GetTypes() gives me:

<itkTemplate itk::Image>
Options:
  [<itkCType bool>, 2]
  [<itkCType double>, 2]
  [<itkCType float>, 2]
  [<itkCType signed int>, 2]
  [<itkCType signed short>, 2]
  [<itkCType unsigned char>, 2]
  [<itkCType unsigned int>, 2]
  [<itkCType unsigned long long>, 2]
  [<itkCType unsigned long>, 2]
  [<itkCType unsigned short>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorD2'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorD4'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorF2'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorF4'>, 2]
  [<class 'itk.itkFixedArrayPython.itkFixedArrayD2'>, 2]
  [<class 'itk.itkFixedArrayPython.itkFixedArrayF2'>, 2]
  [<class 'itk.itkOffsetPython.itkOffset2'>, 2]
  [<class 'itk.itkRGBAPixelPython.itkRGBAPixelUC'>, 2]
  [<class 'itk.itkRGBPixelPython.itkRGBPixelUC'>, 2]
  [<class 'itk.itkSparseFieldFourthOrderLevelSetImageFilterPython.itkNormalBandNodeID2'>, 2]
  [<class 'itk.itkSparseFieldFourthOrderLevelSetImageFilterPython.itkNormalBandNodeIF2'>, 2]
  [<class 'itk.itkSymmetricSecondRankTensorPython.itkSymmetricSecondRankTensorD2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD2'>, 3]
  [<class 'itk.itkVectorPython.itkVectorD4'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD4'>, 3]
  [<class 'itk.itkVectorPython.itkVectorF1'>, 2]
  [<class 'itk.itkVectorPython.itkVectorF2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorF4'>, 2]
  [<class 'itk.stdcomplexPython.stdcomplexF'>, 2]

Did I use the wrong method to build and create pkg for python-itk?

thewtex commented 3 years ago

IPO is not supported (check failed to compile).

@hubutui please test this patch: https://github.com/InsightSoftwareConsortium/ITK/pull/2498

To limit the size of the package, only a limited number of types are available in ITK Python. To print the supported types, run the following command in your python environment:

The default build does not enable support for all pixel types. To add support for more pixels type, enable CMake configurations such as:

https://github.com/InsightSoftwareConsortium/ITKPythonPackage/blob/e019220857b69439dd2f359423ad03f3823070a6/scripts/internal/manylinux-build-wheels.sh#L126-L128

hubutui commented 3 years ago

OK, I could now build python-itk with pull 156 and pull 2498 patches applied.

cd ${srcdir}/ITK-5.2.0
patch -p1 -i https://github.com/InsightSoftwareConsortium/ITK/pull/2498
cd ${srcdir}/ITKPythonPackage-5.2.0
patch -p1 -i https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/156
  python setup.py build -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \
    -DITK_SOURCE_DIR="${srcdir}/ITK-5.2.0"

However, I still fail to read the example image with itk.imread() with a similar error message as above.

Are these wheels from PyPI build with:

    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \

? I could confirm that if I install from PyPi, I could read the example image.

thewtex commented 3 years ago

itk.Image is not wrapped for input type itk.SS, int

This indicated the signed short pixel type, which is wrapped by default.

[, 2]

The error message says 2D signed short images are wrapped. However, I do not see 3D listed.

-DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \

As with #156, we may need to remove the quotes.

hubutui commented 3 years ago

@thewtex I'm confused, if I run

python setup.py build -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING=2;3;4 \
    -DITK_SOURCE_DIR="${srcdir}/ITK-5.2.0"

doesn't it means:

python setup.py build -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING=2
3
4 -DITK_SOURCE_DIR="${srcdir}/ITK-5.2.0"

?

thewtex commented 3 years ago

@hubutui yes, how about "-DITK_WRAP_IMAGE_DIMS:STRING=2;3;4"

hubutui commented 3 years ago

I could build python-itk with:

cd ${srcdir}/ITK-5.2.0
patch -p1 -i https://github.com/InsightSoftwareConsortium/ITK/pull/2498
cd ${srcdir}/ITKPythonPackage-5.2.0
patch -p1 -i https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/156
  python setup.py build -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \
    -DITK_SOURCE_DIR="${srcdir}/ITK-5.2.0"

However, it seems that -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" is not passed to ITK build stage, so itk.Image.GetTypes() shows no 3D type supported.

hubutui commented 3 years ago

I check ${srcdir}/ITKPythonPackage-5.2.0/_skbuild/linux-x86_64-3.9/cmake-build/ITKb/CMakeCache.txt, and found ITK_WRAP_IMAGE_DIMS:STRING=2, if I set "-DITK_WRAP_IMAGE_DIMS:STRING=2;3;4".

ITK_WRAP_IMAGE_DIMS:STRING=2 is also found in ${srcdir}/ITKPythonPackage-5.2.0/_skbuild/linux-x86_64-3.9/cmake-build/ITKb/CMakeCache.txt if I set -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4".

hubutui commented 3 years ago

I check the build log, and log messages mentioned ITK_WRAP_IMAGE_DIMS in build stage:

-- SuperBuild - Searching for python[OK]
-- SuperBuild -   DOXYGEN_EXECUTABLE: /usr/bin/doxygen
-- ITK CMake Cache Args -   -DITK_WRAP_IMAGE_DIMS=2;3;4;-DITK_WRAP_double=ON;-DITK_WRAP_unsigned_short=ON
-- SuperBuild -
...
...
Configuring Project
  Working directory:
    /build/python-itk/src/ITKPythonPackage-5.2.0/_skbuild/linux-x86_64-3.9/cmake-build
  Command:
    cmake /build/python-itk/src/ITKPythonPackage-5.2.0 -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/build/python-itk/src/ITKPythonPackage-5.2.0/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python -DPYTHON_VERSION_STRING:STRING=3.9.3 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.9 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/libpython3.9.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/usr/lib/python3.9/site-packages/skbuild/resources/cmake -DITK_WRAP_unsigned_short:BOOL=ON -DITK_WRAP_double:BOOL=ON '-DITK_WRAP_IMAGE_DIMS:STRING=2;3;4' -DITK_SOURCE_DIR=/build/python-itk/src/ITK-5.2.0 -DCMAKE_BUILD_TYPE:STRING=Release

hubutui commented 3 years ago

update: I rebuild with v5.2.0.post1, now ITK_WRAP_IMAGE_DIMS:STRING=2,3,4 is found in ${srcdir}/ITKPythonPackage-5.2.0/_skbuild/linux-x86_64-3.9/cmake-build/ITKb/CMakeCache.txt as expected. However, it seems that there is something wrong with the install step.

python setup.py install --root="${pkgdir}" --optimize=1 --skip-build

Nothing is installed except an __init__.py file. The install tree looks like this:

.
└── usr
    └── lib
        └── python3.9
            └── site-packages
                ├── itk
                │   └── __init__.py
                └── itk-5.2.0.post1-py3.9.egg-info
                    ├── dependency_links.txt
                    ├── not-zip-safe
                    ├── PKG-INFO
                    ├── requires.txt
                    ├── SOURCES.txt
                    └── top_level.txt

check the complete build log here

thewtex commented 3 years ago

@hubutui thanks for the update.

Does python setup.py bdist_wheel, python -m pip install dist/*.whl work?

hubutui commented 3 years ago

_itkver=5.2.0
  python setup.py bdist_wheel -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \
    -DITK_SOURCE_DIR="${srcdir}/ITK-${_itkver}"
  PIP_CONFIG_FILE=/dev/null pip install --isolated --root="${pkgdir}" --ignore-installed --no-deps dist/*.whl

No, it didn't work. Check the complete build log here.

thewtex commented 3 years ago

No, it didn't work. Check the complete build log here.

A quick looks, and I am having a hard time finding the error. Any pointers?

It seems to end with

-> Checking packages

and the fils that were built in the package, which look to be in their correct location:

usr/lib/python3.9/site-packages/itk/Configuration/ITKIOIPL_sn <

hubutui commented 3 years ago

I check the build wheel file, it's less than 1MB. According to https://github.com/InsightSoftwareConsortium/ITKPythonPackage/blob/f7fb03173bbde8e5a6a7462e0b478cc7287737a3/scripts/internal/manylinux-build-wheels.sh#L51-L79 I rebuid with these commands:

# build step
  python setup.py bdist_wheel -- \
    -DITK_WRAP_unsigned_short:BOOL=ON \
    -DITK_WRAP_double:BOOL=ON \
    -DITK_WRAP_IMAGE_DIMS:STRING="2;3;4" \
    -DITKPythonPackage_ITK_BINARY_REUSE=OFF \
    -DITKPythonPackage_WHEEL_NAME="itk" \
    -DITK_SOURCE_DIR="${srcdir}/ITK-${_itkver}"
# package (or install) step
  PIP_CONFIG_FILE=/dev/null pip install --isolated --root="${pkgdir}" --ignore-installed --no-deps dist/*.whl
  python -O -m compileall "${pkgdir}/usr/lib"

I could now build the wheel file, but the file itkConfig.py is installed to /usr, not /usr/lib/python3.9/site-packages. This lead to import error:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
    import itk
  File "/usr/lib/python3.9/site-packages/itk/__init__.py", line 26, in <module>
    from itkConfig import ITK_GLOBAL_VERSION_STRING as __version__
ModuleNotFoundError: No module named 'itkConfig'

I manually move /usr/itkConfig.py to /usr/lib/python3.9/site-packages/itkConfig.py, then I got error when I try to read an image:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
    img = itk.imread('/home/hubutui/tmpdir/MRBrainTumor2.nrrd')
  File "/usr/lib/python3.9/site-packages/itk/support/extras.py", line 955, in imread
    reader = template_reader_type.New(**kwargs)
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 660, in New
    return self._NewImageReader(
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 191, in _NewImageReader
    ImageType = itk.Image[PixelType, dimension]
  File "/usr/lib/python3.9/site-packages/itk/support/template_class.py", line 529, in __getitem__
    raise itk.TemplateTypeError(self, key)
itk.support.extras.TemplateTypeError: itk.Image is not wrapped for input type `itk.SS, int`.

To limit the size of the package, only a limited number of
types are available in ITK Python. To print the supported
types, run the following command in your python environment:

    itk.Image.GetTypes()

Possible solutions:
* If you are an application user:
** Convert your input image into a supported format (see below).
** Contact developer to report the issue.
* If you are an application developer, force input images to be
loaded in a supported pixel type.

    e.g.: instance = itk.Image[itk.RGBPixel[itk.UC], int].New(my_input)

* (Advanced) If you are an application developer, build ITK Python yourself and
turned to `ON` the corresponding CMake option to wrap the pixel type or image
dimension you need. When configuring ITK with CMake, you can set
`ITK_WRAP_${type}` (replace ${type} with appropriate pixel type such as
`double`). If you need to support images with 4 or 5 dimensions, you can add
these dimensions to the list of dimensions in the CMake variable
`ITK_WRAP_IMAGE_DIMS`.

Supported input types:

itk.RGBPixel[itk.UC]
itk.RGBAPixel[itk.UC]
itk.Vector[itk.F,2]
itk.Vector[itk.F,4]
itk.CovariantVector[itk.F,2]
itk.CovariantVector[itk.F,4]
itk.SS
itk.UC
itk.US
itk.F
itk.D
itk.complex[itk.F]
itk.Vector[itk.D,2]
itk.Vector[itk.D,4]
itk.SI
itk.UI
itk.UL
itk.ULL
itk.B
itk.FixedArray[itk.F,2]
itk.FixedArray[itk.D,2]
itk.Offset[2]
itk.SymmetricSecondRankTensor[itk.D,2]
itk.Vector[itk.D,2]
itk.Vector[itk.D,4]
itk.Vector[itk.F,1]
itk.CovariantVector[itk.D,2]
itk.CovariantVector[itk.D,4]
itk.NormalBandNode[itk.Image[itk.F,2]]
itk.NormalBandNode[itk.Image[itk.D,2]]

and itk.Image.GetTypes outputs:

<itkTemplate itk::Image>
Options:
  [<itkCType bool>, 2]
  [<itkCType double>, 2]
  [<itkCType float>, 2]
  [<itkCType signed int>, 2]
  [<itkCType signed short>, 2]
  [<itkCType unsigned char>, 2]
  [<itkCType unsigned int>, 2]
  [<itkCType unsigned long long>, 2]
  [<itkCType unsigned long>, 2]
  [<itkCType unsigned short>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorD2'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorD4'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorF2'>, 2]
  [<class 'itk.itkCovariantVectorPython.itkCovariantVectorF4'>, 2]
  [<class 'itk.itkFixedArrayPython.itkFixedArrayD2'>, 2]
  [<class 'itk.itkFixedArrayPython.itkFixedArrayF2'>, 2]
  [<class 'itk.itkOffsetPython.itkOffset2'>, 2]
  [<class 'itk.itkRGBAPixelPython.itkRGBAPixelUC'>, 2]
  [<class 'itk.itkRGBPixelPython.itkRGBPixelUC'>, 2]
  [<class 'itk.itkSparseFieldFourthOrderLevelSetImageFilterPython.itkNormalBandNodeID2'>, 2]
  [<class 'itk.itkSparseFieldFourthOrderLevelSetImageFilterPython.itkNormalBandNodeIF2'>, 2]
  [<class 'itk.itkSymmetricSecondRankTensorPython.itkSymmetricSecondRankTensorD2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD2'>, 3]
  [<class 'itk.itkVectorPython.itkVectorD4'>, 2]
  [<class 'itk.itkVectorPython.itkVectorD4'>, 3]
  [<class 'itk.itkVectorPython.itkVectorF1'>, 2]
  [<class 'itk.itkVectorPython.itkVectorF2'>, 2]
  [<class 'itk.itkVectorPython.itkVectorF4'>, 2]
  [<class 'itk.stdcomplexPython.stdcomplexF'>, 2]

see also the complete build log here. The lines between Starting build() and Starting package() are the log for build step, and lines after Starting package() are the log for install step.

hubutui commented 3 years ago

I also try to build the python binding with only ITK source, not using this repo.

  cmake_opts=(
    -DBUILD_SHARED_LIBS=ON
    -DBUILD_TESTING=OFF
    -DCMAKE_BUILD_TYPE=Release
    -DCMAKE_C_COMPILER=gcc
    -DCMAKE_CXX_COMPILER=g++
    -DCMAKE_INSTALL_PREFIX=/usr
    -DCMAKE_SKIP_INSTALL_RPATH=ON
    -DITK_BUILD_DEFAULT_MODULES=ON
    -DITK_USE_MKL=ON
    -DITK_USE_SYSTEM_CASTXML=ON
    -DITK_USE_SYSTEM_DOUBLECONVERSION=OFF
    -DITK_USE_SYSTEM_EIGEN=ON
    -DITK_USE_SYSTEM_EXPAT=ON
    -DITK_USE_SYSTEM_FFTW=ON
    -DITK_USE_SYSTEM_GDCM=ON
    -DITK_USE_SYSTEM_GOOGLETEST=ON
    -DITK_USE_SYSTEM_HDF5=ON
    -DITK_USE_SYSTEM_JPEG=ON
    -DITK_USE_SYSTEM_PNG=ON
    -DITK_USE_SYSTEM_SWIG=ON
    -DITK_USE_SYSTEM_TIFF=ON
    -DITK_USE_SYSTEM_ZLIB=ON
    -DITK_WRAP_unsigned_short=ON
    -DITK_WRAP_double=ON
    -DITK_WRAP_IMAGE_DIMS="2;3;4"
    -DITK_WRAP_PYTHON=ON
)
git clone https://github.com/InsightSoftwareConsortium/ITK.git
_pkgname=ITK
  cmake -B "build" -S "${srcdir}/${_pkgname}" \
    ${cmake_opts[@]} \
    -DITK_USE_GPU=ON
  make -C "${srcdir}/build"
  make -C "${srcdir}/build" DESTDIR="${pkgdir}" install

everything works great. I could build and install both ITK and its python binding.

InsightSoftwareConsortium / ITKPythonPackage

fail to build release 5.2rc03 on ArchLinux #154