Building wheel suitable for extensions

[Korijn]

Me and @berendkleinhaneveld were investigating if we could build extensions against this package. It seems to be a runtime-only package, which is great, but we need the development files, such as headers and the wrapPython executable.

It looks like overriding the option -DVTK_INSTALL_NO_DEVELOPMENT:BOOL=ON to OFF should do the trick. So we tried that, and more development files ended up in the wheel. However, we are not seeing any header files!

We've been investigating all day, but it's unclear to us what is causing this. Do you guys have any suggestions for where we should be looking?

[sanguinariojoe]

+1

header files, and eventually all the cmake helpers, shall be added

[sanguinariojoe]

Applying the following patch the package is installing the header files and the cmake module files (inside the vtk python package folder). As main drawback, it is creating another vtk-8.1.0.data folder with the verdict stuff.

diff --git CMakeLists.txt CMakeLists.txt
index a9cf106..cf587dd 100644
--- CMakeLists.txt
+++ CMakeLists.txt
@@ -212,7 +212,9 @@ if(VTKPythonPackage_SUPERBUILD)
       -DVTK_INSTALL_RUNTIME_DIR:PATH=${install_dir}
       -DVTK_INSTALL_LIBRARY_DIR:PATH=${install_dir}
       -DVTK_INSTALL_ARCHIVE_DIR:PATH=${install_dir}
-      -DVTK_INSTALL_NO_DEVELOPMENT:BOOL=ON
+      -DVTK_INSTALL_INCLUDE_DIR:PATH=${install_dir}/include/
+      -DVTK_INSTALL_PACKAGE_DIR:PATH=${install_dir}/cmake/
+      -DVTK_INSTALL_NO_DEVELOPMENT:BOOL=OFF

       # Rendering options
       -DVTK_Group_Qt:BOOL=OFF  # XXX Enabled this later
@@ -281,7 +283,7 @@ else()
   #------------------------------------------------------
   #-----------------------------------------------------------------------------

-  set(components "RuntimeLibraries")
+  set(components "RuntimeLibraries;Development")

   #-----------------------------------------------------------------------------
   # Install VTK components

To make it fully extensible we just need to add the rest of VTK Cmake files.

[jcfr]

@Korijn @berendkleinhaneveld Thanks for summarizing your experiments and reaching out :+1:

It seems to be a runtime-only package

Indeed. It wasn't developed to be used as a C++ SDK

we could build extensions against this package. It seems to be a runtime-only package, which is great, but we need the development files, such as headers and the wrapPython executable.

The main challenge is that ultimately only python modules with stripped symbols will be distributed. Only the symbol of the init function will be exposed.

To streamline the building of extension, I instead suggest an approach like what is done for the ITK module where an archived build tree re-usable from Appveyor, TravisCI or CircleCI is provided:

• https://github.com/InsightSoftwareConsortium/ITKModuleTemplate/blob/master/.circleci/config.yml#L76

• https://raw.githubusercontent.com/InsightSoftwareConsortium/ITKPythonPackage/master/scripts/dockcross-manylinux-download-cache-and-build-module-wheels.sh

@thewtex: What do you think ?

[thewtex]

Publishing the header files is only part of the requirements for creating modules that would build against, link to, and use the VTK wheels. It will not be sufficient.

Wheels do not officially support distributing native shared libraries, but there are workarounds that can be applied to make it happen. However, using native libraries from a second package is another issue (the loader / linker, which operates in a different way across platforms, needs to be able to find the libraries, ...).

There are also ABI / API / linked filepath issues. This means that when the VTK package updates, any modules that where built against it would break. As @jcfr mentioned, we have mitigated this issue in ITK by static linking. However, there is work that needs to be done to see if it is possible to use static linking for the VTK Python package.