Use system libraries

[Korijn]

I've compiled a list of third party libraries that we might be able to supply with conda-forge packages. This will have the added benefit of bringing down build time.

I used the following searches to track all the libraries that can be easily configured using the CMake parmeter VTK_USE_SYSTEM_LIBRARIES:BOOL=ON:

• https://github.com/Kitware/VTK/search?p=2&q=vtk_module_third_party&utf8=%E2%9C%93
• https://github.com/Kitware/VTK/search?utf8=%E2%9C%93&q=vtk_module_python_package

Here is the list:

 # VTK third party libraries
 # some are probably optional...
 - freetype 2.4.7
 - gl2ps 1.3.9
 - expat 1.95.2
 - glew  # seems heavily modified, couldn't find a version
 - jpeg 6b
 - libogg 1.1.4 
 - libtheora 1.1.1
 - libpng 1.0.65
 - tiff 3.8.2
 - zlib 1.2.8
 - libxml12 2.6.27  # seem heavily customized
 - libproj4 4.4
 - diy2  # [not win] couldnt find a version
 - netcdf 4.1.2  # seems modified
 - jsoncpp 0.7.0
 - xdmf2  # unclear which version
 - xdmf3  # unclear
 - hdf5 1.8.13
 # indirect dependencies
 - boost 1.61.*  # required by xdmf2 and xdmf3
 - mpi4py 1.3  # [not win]
 # VTK third party Python libraries (probably only necessary for web addon?)
 - autobahn 0.8.14
 - six 1.7.2
 - twisted 14.0.0
 - zope.interface 4.1.1

A number of things remain unclear:

• Some of these libraries have been modified (e.g. bugfixes) but this is not clearly indicated.
• Some have no version number and are simply snapshots of remote repositories.
• Some have indirect dependencies, without listing their required version number.
• Some are probably only required when building VTK with specific add-ons.

Todo:

• Determine which packages we have available
• Determine which packages are strictly necessary for our build and which are not

cc @jakirkham @ccordoba12 @ivoflipse @grlee77 @patricksnape @demarle

[jakirkham]

libogg libtheora

Maybe we should start packaging audio and video players too. 😄

[Korijn]

@jakirkham I'm quite surprised we needed freeglut btw, since it isn't in this list. Or is it a replacement for glew?

[demarle]

I think freeglut comes in with the RenderingExternal (adaptor to a specific VR environment).

Yes, turn off VTK_Group_Web to remove autobahn, six, twisted and zope dependencies.

Boost is used in a couple of modules, all including xdmf3 are optional. At least for xdmf3, and probably for all of them, it is a header only dependency. Xdmf2 does not depend on it at all.

hope that helps

[demarle]

@aashish24 you and the UVCDAT guys just condified VTK (and the rest of UVCDAT). Do you have any pointers/requests for VTK's conda packaging?

[aashish24]

Here is the link to our conda recipe for VTK that is working well for the climate project we are involved with https://github.com/UV-CDAT/conda-recipes/tree/master/vtk. Note that some of the names / variables are project specific.

[Korijn]

@aashish24 thanks for sharing! I think we can pick up a thing or two from your recipe. I have a couple of questions:

• I see you are picking different rendering configurations, cocao for os x, x for linux (or mesa) and opengl as a backend. Are these default configurations or should we include them here as well?
• Is mesa or X preferable as a default?
• Do you mind if we also add a mesa recipe to conda-forge, based on your recipe?
• Why did you disable "OSMesa Gallium llvmpipe state-tracker", as it appears to be much more powerful according to the VTK docs?

[demarle]

For mac, strongly recommend cocoa. We haven't been supporting X (or carbon fwiw) on Mac for a while now.

The OpenGL2 backend is fairly new and is more demanding of the features provided by the OpenGL implementation (decent vendor graphics drivers or mesa > 11 roughly). We will be deprecating and then removing the legacy backend over the next couple of VTK releases. My recommendation is to get OpenGL1 backend working well first so that you know you have a good VTK build, then switch to OpenGL2 and see what issues turn up with the more stringent OpenGL requirements.

[Korijn]

We are actually working on a mesa (latest) package as well, so that might work out just fine with the opengl2 backend. Anyway, thanks again for your input. :)

[aashish24]

I see you are picking different rendering configurations, cocao for os x, x for linux (or mesa) and opengl as a backend. Are these default configurations or should we include them here as well?

I think it would be nice to include them.

Is mesa or X preferable as a default?

On linux X is preferable then mesa.

Do you mind if we also add a mesa recipe to conda-forge, based on your recipe?

Sure, please go ahead. This recipe is written by someone else but I don't see a reason why you cannot add a similar recipe to forge.

Why did you disable "OSMesa Gallium llvmpipe state-tracker", as it appears to be much more powerful according to the VTK docs?

I would have to ask the original author about it.

[Korijn]

This work is pretty much done, as can be read in #4. The following dependencies were added:

    - future  # used in the generated python wrappers
    # VTK Third Party dependencies
    - zlib
    - freetype
    - hdf5  # [unix]
    - libxml2
    - libpng
    - jpeg
    - libtiff
    - jsoncpp  # [unix]
    - expat

These were the problems encountered along the way:

• jsoncpp is not available on Windows
• hdf5 package cannot be detected by VTK's CMake on Windows
• netcdf package cannot be detected by VTK's CMake on all platforms
• sqlite package cannot be replaced since there is no VTK_USE_SYSTEM_SQLITE option

Other packages simply aren't available on conda-forge (yet).

Some other changes:

• Use Cocoa on OS X
• Release build on Windows
• Floating point exceptions disabled
• Rendering backend fixed to OpenGL (instead of the default OpenGL2) for broader support
• vtkRenderingMatplotlib module enabled

[aashish24]

Nice @Korijn !! thanks for working on this.

[Korijn]

4 was merged after pinning the packages to the "correct" versions as used more broadly by conda-forge.