Mesa support

[jan-janssen]

With jupyter becoming more and more popular, it might be reasonable to provide a vtk version based on the mesa driver: https://anaconda.org/anaconda/mesa to be able to run vtk headless.

I found the corresponding options on: http://docs.enthought.com/mayavi/mayavi/tips.html#using-vtk-with-mesa-for-pure-software-rendering

So I guess, all that has to be done is add:

-DVTK_OPENGL_HAS_OSMESA:BOOL=ON \
-DVTK_USE_OFFSCREEN:BOOL=ON \
-DVTK_USE_GL2PS:BOOL=ON \
-DOSMESA_INCLUDE_DIR:PATH=/path/to/Mesa-10.5.4/include \
-DOSMESA_LIBRARY:FILEPATH=/path/to/Mesa-10.5.4/lib64/libOSMesa.so \
-DOPENGL_INCLUDE_DIR:PATH=/path/to/Mesa-10.5.4/include \
-DOPENGL_gl_LIBRARY:FILEPATH=/path/to/Mesa-10.5.4/lib64/libGL.so \
-DOPENGL_glu_LIBRARY:FILEPATH=/path/to/Mesa-10.5.4/lib64/libGLU.so \

I am just not sure how to create the dynamic links to the Mesa package.

[ivoflipse]

@maxyme can you chime in here? As you've been working on this already for our own package

[Cadair]

I would love to see this, we would like to use vtk on our cluster which would be much better headless. @solardrew

[Maxyme]

We have add similar requirements for our own projects. You can look here for the recipe using osmesa libraries and ospray: https://github.com/ClinicalGraphics/conda-recipes/tree/master/vtk and here if you plan on building your own osmesa and mesa libraries: https://github.com/ClinicalGraphics/conda-recipes/tree/master/osmesa and here to build ospray: https://github.com/ClinicalGraphics/conda-recipes/tree/master/ospray

Please note that osmesa is linked to a custom version of LLVM with built with _DLLVM_BUILD_LLVMDYLIB=ON which may differ from the llvm recipe on conda-forge. (from: http://www.linuxfromscratch.org/blfs/view/cvs/general/llvm.html)

builds libraries as static and links all of them into an unique shared one. This is the recommended way of building a shared library.

All of these are available as pre-build packages on anaconda: https://anaconda.org/clinicalgraphics/vtk https://anaconda.org/clinicalgraphics/osmesa https://anaconda.org/clinicalgraphics/ospray

The plan is to eventually PR for conda-forge for these recipes. Also, please note, the osmesa linking appears to only currently work in Linux and not in windows. (not tested for os x)

[Cadair]

Thanks @Maxyme your packages will probably be enough to get us started. It would be nice to see these things in conda-forge in time.

[Cadair]

It does seem that the coda-forge llvm package does not have that flag: https://github.com/conda-forge/llvmdev-feedstock/blob/master/recipe/build.sh

[Maxyme]

No, it does not. I'll point you to our package and recipe: https://anaconda.org/clinicalgraphics/llvm https://github.com/ClinicalGraphics/conda-recipes/tree/master/llvm

[jan-janssen]

I have to admit, I am still confused. I installed the packages from clinicalgraphics but still mayavi inside jupyter does not work in headless mode failing with an message about missing Display variable.

When checking the repository from clinicalgraphics, I am especially confused about the following lines - I basically expected the opposite.

 -DVTK_USE_X:BOOL=ON \
 -DVTK_USE_OFFSCREEN:BOOL=OFF \

@Maxyme: maybe you can help me ...

[Cadair]

@kwotuveang3k4bk yeah I get the same, it still needs an X server even though it uses osmesa. I would expect the settings as you described them.

[Korijn]

VTK 7.1 supports switching between on and offscreen when built against osmesa. If you build with VTK_USE_OFFSCREEN you disable that switching capability.

Make sure to call render_window.SetOffscreenRendering(True) to switch to osmesa rendering.

[Cadair]

Interesting, thanks. I need to work out how to enable that for mayavi. All the mayavi ways to enable offscreen seem to trigger a "can't find X server" error before I can enable offscreen.

[jan-janssen]

Thanks a lot for the explanation, I tested the VTK off screen capabilities and it is working:

#! ./local/bin/python

from vtk import (vtkSphereSource, vtkPolyDataMapper, vtkActor, vtkRenderer,
        vtkRenderWindow, vtkWindowToImageFilter, vtkPNGWriter)

sphereSource = vtkSphereSource()
mapper = vtkPolyDataMapper()
mapper.SetInputConnection(sphereSource.GetOutputPort())

actor = vtkActor()
actor.SetMapper(mapper)

renderer = vtkRenderer()
renderWindow = vtkRenderWindow()
renderWindow.SetOffScreenRendering(1)
renderWindow.AddRenderer(renderer)

renderer.AddActor(actor)
renderer.SetBackground(1, 1, 1)

renderWindow.Render()

windowToImageFilter = vtkWindowToImageFilter()
windowToImageFilter.SetInput(renderWindow)
windowToImageFilter.Update()

writer = vtkPNGWriter()
writer.SetFileName("sphere.png")
writer.SetInputConnection(windowToImageFilter.GetOutputPort())
writer.Write()

But when I test the mayavi off screen example, it is not working:

# Author: Prabhu Ramachandran <prabhu@aero.iitb.ac.in>
# Copyright (c) 2007, Enthought, Inc.
# License: BSD Style.

from os.path import join, abspath, dirname

# The offscreen Engine.
from mayavi.api import OffScreenEngine

# Usual MayaVi imports
from mayavi.scripts.util import get_data_dir
from mayavi.sources.api import VTKXMLFileReader
from mayavi.modules.api import Outline, ScalarCutPlane, Streamline

def main():
    # Create the MayaVi offscreen engine and start it.
    e = OffScreenEngine()
    # Starting the engine registers the engine with the registry and
    # notifies others that the engine is ready.
    e.start()

    # Create a new scene.
    win = e.new_scene()

    # Now setup a normal MayaVi pipeline.
    src = VTKXMLFileReader()
    src.initialize(join(get_data_dir(dirname(abspath(__file__))),
                        'fire_ug.vtu'))
    e.add_source(src)
    e.add_module(Outline())
    e.add_module(ScalarCutPlane())
    e.add_module(Streamline())
    win.scene.isometric_view()
    # Change the size argument to anything you want.
    win.scene.save('offscreen.png', size=(800, 800))

if __name__ == '__main__':
    main()

So I guess that is a topic we should discuss with the mayavi developers. https://github.com/enthought/mayavi/issues/477

[Cadair]

This issue should probably remain open as a feature request for including Mesa support in the conda-forge package.

[Korijn]

Indeed.

[doutriaux1]

We are working with this with @aashish24 @danlipsa with kitware. We ported mesalib to conda-forge and made it provide the mesalib feature. We would love to be able to using the official vtk from conda-forge but we do need BOTH version available as well I have the recipe working and if yoiu use the uvcdat channel you can download vtk with mesalib support. In order to makesure we don't break vtk we called our package vtk-cdat it can be obtained with or w/o mesalib

conda install vtk-cdat -c conda-forge -c uvcdat

or for the mesalib enabled

conda install vtk-cdat mesalib -c conda-forge -c uvcdat

My question is how to get conda-forge to build both versions short of creating 2 recipes?

[doutriaux1]

note we called our package mesalib and it does provides the latest version of mesa library both on Linux and OSX

[Korijn]

Boy, wouldn't it be a lot simpler if VTK linked to a backend at runtime depending on runtime configuration options?

Also, I feel like the name mesalib might be too broad: what type of mesa build does it contain? I'm guessing it's OSMesa, so why not call it osmesalib?

Anyway, I think the best way forward is indeed a slight variation on the recipe. We've already done this to support both py27 and py3k by creating a second master branch. This was due to the build taking too long and hitting timeouts. If you go through the issue and pr history you'll find it is a recurring topic, and no doubt you'll have some fun with it if you move forward with this. :)

In the scenario we add mesalib builds on this feedstock, you will have to add 2 more branches and make your recipe changes there. We'll have to remember to maintain all four of them in the future though.

However, since you will want to name this new variation on VTK differently, e.g. vtk-osmesa, the feedstock name will not match. So I think a fork might make more sense.That way you can merge changes made here into your fork every now and then. @jakirkham What do you think?

[LunarLanding]

https://github.com/conda-forge/vtk-feedstock/commit/77f0e66b9cc63d575aebfac993bf330a17ddab00 disables the osmesa build, so this should be reopened.