search

InsightSoftwareConsortium / ITKPythonPackage

A setup script to generate ITK Python Wheels
https://itkpythonpackage.readthedocs.io
Apache License 2.0
63 stars 21 forks source link

COMP: Consolidate Linux ARM build script #236

Closed tbirdso closed 1 year ago

tbirdso commented 1 year ago

Several quality-of-life fixes and updates for ARM support:

Applies changes introduced by @thewtex in https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/234.

Tests have passed at https://github.com/InsightSoftwareConsortium/ITKSplitComponents/actions/runs/3736106376/jobs/6340059855. Note that ARM builds on Github Actions runners are significantly slower than x86/x64 builds due to emulation. ITKSplitComponents Linux jobs previously completed in ~10 minutes and now complete in ~30 minutes with the addition of ARM wheels.

co-authored-by: Matt McCormick matt.mccormick@kitware.com

tbirdso commented 1 year ago

Testing in https://github.com/InsightSoftwareConsortium/ITKSplitComponents/pull/67

tbirdso commented 1 year ago

After navigating several incremental issues applying build scripts for ARM module builds I've encountered a wrapping compilation issue:

FAILED: /work/ITK-cp38-cp38-manylinux_2_28_aarch64/Wrapping/castxml_inputs/itkSplitComponentsImageFilter.xml 
cd /work/_skbuild/linux-aarch64-3.8/cmake-build/Wrapping/Modules/SplitComponents && /work/_skbuild/linux-aarch64-3.8/cmake-build/Wrapping/Generators/CastXML/castxml/bin/castxml -o /work/ITK-cp38-cp38-manylinux_2_28_aarch64/Wrapping/castxml_inputs/itkSplitComponentsImageFilter.xml --castxml-gccxml --castxml-start _wrapping_ --castxml-cc-gnu "(" /opt/rh/gcc-toolset-12/root/usr/bin/c++ -std=c++14 ")" -w -c @/work/ITK-cp38-cp38-manylinux_2_28_aarch64/Wrapping/castxml_inputs/.castxml.inc /work/ITK-cp38-cp38-manylinux_2_28_aarch64/Wrapping/castxml_inputs/itkSplitComponentsImageFilter.cxx
...
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/aarch64-redhat-linux/12/../../../../include/c++/12/bits/new_allocator.h:158:2: error: call to '__builtin_operator_delete' selects non-usual deallocation function
        _GLIBCXX_OPERATOR_DELETE(_GLIBCXX_SIZED_DEALLOC(__p, __n));
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Failed build with logs is at https://github.com/tbirdso/ITKSplitComponents/actions/runs/3604194121/jobs/6073281299

As this appears to occur in the first applied CastXML step I will open an issue on that repo for help.

thewtex commented 1 year ago

@tbirdso I am wondering if this is an incompatibility with the ITKPythonBuilds tarball (built with the gcc-toolset-11) and the gcc-toolset-12 in the manylinux aarch64 latest image?

I recommend rebasing on master to integrate https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/237. Then rebuild the tarball on blaster. This will take a while. Then test the module wheel script locally with the built tarball.

Since we will need updated itk-* aarch64 wheels, I suggest we defer this update to 5.3.1.

tbirdso commented 1 year ago

I am wondering if this is an incompatibility with the ITKPythonBuilds tarball (built with the gcc-toolset-11) and the gcc-toolset-12 in the manylinux aarch64 latest image?

Thanks @thewtex , I think this is spot on. I have consolidated discussion in https://github.com/InsightSoftwareConsortium/ITK/issues/3785.

tbirdso commented 1 year ago

Re-running testing at https://github.com/InsightSoftwareConsortium/ITKSplitComponents/pull/67

tbirdso commented 1 year ago

Ready for review as CI jobs are passing. However, it should be noted that Python packaging jobs build but do not load and test wheels. Once ARM Linux wheel artifacts are available from the ITKSplitComponents test run it would be good to verify that those can be used on an ARM platform. It may take me several days to get back to testing those, or someone else can test to verify.

Steps to test:

  1. Pull ITKSplitComponents Linux ARM wheel artifacts from https://github.com/tbirdso/ITKSplitComponents/actions/runs/3736220631
  2. Obtain access to a Linux ARM machine, such as running an ARM instance on AWS EC2
  3. Install the corresponding Python wheel
  4. Install ITK with pip install itk==v5.3.0
  5. Put the ITKSplitComponents wheel on the machine and install the local wheel in the Python environment
  6. In a Python shell try loading a filter from the module: import itk; itk.SplitComponentsImageFilter.GetTypes()
  7. (Optional) Try reproducing example functionality from the ITKSplitComponents example notebook

EDIT: Python load test failed; see comments below.

tbirdso commented 1 year ago

EDIT: On review it looks like there was an issue in Python 3.11 packages resulting in only ARM wheel artifacts being made available from the run, will need to investigate.

tbirdso commented 1 year ago

Getting back to the 3.11 log, it looks like the issue may have simply been that I force pushed ITKPythonPackage updates too quickly.

Updating build scripts to InsightSoftwareConsortium/ITKPythonPackage@d45219286484dafb746591e3801f4e860dfd29ec
Cloning into 'IPP-tmp'...
~/work/ITKSplitComponents/ITKSplitComponents/IPP-tmp ~/work/ITKSplitComponents/ITKSplitComponents
fatal: reference is not a tree: d45219286484dafb746591e3801f4e860dfd29ec

Will re-run testing in ITKSplitComponents.

EDIT: Testing at https://github.com/InsightSoftwareConsortium/ITKSplitComponents/actions/runs/3759019847/jobs/6388068372

EDIT: Build succeeded on retry.

tbirdso commented 1 year ago

When attempting to install locally, load, and test on an ARM Linux instance I am seeing the following failure:

>>> itk.SplitComponentsImageFilter.GetTypes()
Loading ITKPyBase... done
Loading ITKCommon... done
Loading SplitComponents... Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.8/site-packages/itk/support/lazy.py", line 138, in __getattribute__
    base.itk_load_swig_module(module, namespace)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/itk/support/base.py", line 132, in itk_load_swig_module
    l_module = loader.load(swig_module_name)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/itk/support/base.py", line 291, in load
    l_spec.loader.exec_module(l_module)  # pytype: disable=attribute-error
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ubuntu/.local/lib/python3.8/site-packages/itk/support/../SplitComponentsPython.py", line 13, in <module>
    from . import _SplitComponentsPython
ImportError: libtbb.so.12: cannot open shared object file: No such file or directory

It looks like the TBB library is not being packaged with the ITKSplitComponents wheel as expected. I have compared the contents of the 2_28_aarch64 and _2_28_x86_64 wheels and confirmed that libtbb is included in the x86_64 wheel but not the aarch64 wheel. Will investigate next week.

tbirdso commented 1 year ago

Successfully repaired, loaded, and used an ITKSplitComponents ARM wheel on a local ARM machine with the most recent patch. Will test in ITKSplitComponents CI.

EDIT: Testing at https://github.com/tbirdso/ITKSplitComponents/actions/runs/3823026183/jobs/6503756014

EDIT2: Verified that the ITKSplitComponents aarch64 resulting wheel is packaged with TBB and that the ITKSplitComponents example procedure (minus viewing) can be run on an aarch64 test machine. aarch64 build updates are good to go. 🟢

tbirdso commented 1 year ago

@dzenanz When you have a moment would you please re-review these changes before merge?

tbirdso commented 1 year ago

@dzenanz Thanks! Moving forward with merge.