VTK issues with drake_visualizer when ros-indigo-tmc-desktop-full installed

[athackst]

When trying to run the drake_visualizer via

roscd
..
./install/bin/drake_visualizer

The following error appears:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/toyota/dev/drake_catkin_workspace/install/lib/python2.7/dist-packages/director/drakevisualizer.py", line 5, in <module>
    import director.objectmodel as om
  File "/home/toyota/dev/drake_catkin_workspace/install/lib/python2.7/dist-packages/director/objectmodel.py", line 4, in <module>
    from director.propertyset import PropertySet, PropertyAttributes, PropertyPanelHelper
  File "/home/toyota/dev/drake_catkin_workspace/install/lib/python2.7/dist-packages/director/propertyset.py", line 2, in <module>
    from director.fieldcontainer import FieldContainer
  File "/home/toyota/dev/drake_catkin_workspace/install/lib/python2.7/dist-packages/director/fieldcontainer.py", line 1, in <module>
    import vtk
  File "/usr/lib/python2.7/dist-packages/vtk/__init__.py", line 39, in <module>
    from vtkCommonCore import *
  File "/usr/lib/python2.7/dist-packages/vtk/vtkCommonCore.py", line 1, in <module>
    from vtkCommonCorePython import *
ImportError: No module named vtkCommonCorePython

[athackst]

It appears the correct version (5.8) should be linked:

ldd ./install/bin/drake-visualizer | grep libvtkCommon
    libvtkCommon.so.5.8 => /usr/lib/libvtkCommon.so.5.8 (0x00007f44fe8b6000)

However it looks like ros-indigo-tmc-desktop-full also installs version 6

$ dpkg -l | grep vtk
ii  libvtk-java                                           5.8.0-14.1ubuntu3                                   amd64        Visualization Toolkit - A high level 3D visualization library - java
ii  libvtk5-dev                                           5.8.0-14.1ubuntu3                                   amd64        VTK header files for building C++ code
ii  libvtk5-qt4-dev                                       5.8.0-14.1ubuntu3                                   amd64        Visualization Toolkit - A high level 3D visualization library - Qt devel
ii  libvtk5.8                                             5.8.0-14.1ubuntu3                                   amd64        Visualization Toolkit - A high level 3D visualization library - runtime
ii  libvtk5.8-qt4                                         5.8.0-14.1ubuntu3                                   amd64        Visualization Toolkit - A high level 3D visualization library - Qt runtime
ii  python-vtk                                            5.8.0-14.1ubuntu3                                   amd64        Python bindings for VTK
ii  tcl-vtk                                               5.8.0-14.1ubuntu3                                   amd64        Tcl bindings for VTK
ii  tmc-libvtk6                                           6.0.0-6                                             amd64        Visualization Toolkit - A high level 3D visualization library - runtime
ii  tmc-libvtk6-dev                                       6.0.0-6                                             amd64        VTK header files for building C++ code
ii  tmc-libvtk6-java                                      6.0.0-6                                             amd64        Visualization Toolkit - A high level 3D visualization library - java
ii  tmc-python-vtk6                                       6.0.0-6                                             amd64        Python bindings for VTK
ii  tmc-tcl-vtk6                                          6.0.0-6                                             amd64        Tcl bindings for VTK
ii  tmc-vtk6                                              6.0.0-6                                             amd64        Binaries for VTK6

[athackst]

Temporary solution: uninstall tmc's versions of vtc

sudo apt-get remove tmc-vtk6 tmc-libvtk6 tmc-libvtk6-java

However, this seems to remove a significant portion of HSR code

$ sudo apt-get remove tmc-vtk6 tmc-libvtk6 tmc-libvtk6-java 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  clang-format-3.6 evemu-tools evtest festival festlex-cmu festlex-poslex
  festvox-kallpc16k hdf5-helpers joystick libarmadillo4 libarpack2
  libasound2-dev libavahi-client-dev libavahi-common-dev libblas-dev
  libcaca-dev libcgal-dev libcgal10 libdap-dev libdap11 libdapclient3
  libdapserver7 libdbus-1-dev libegl1-mesa-dev libepsilon1 libestools2.1
  libevemu1 libfreexl1 libgdal-dev libgdal1h libgeos-dev libgif-dev
  libgles2-mesa-dev libglew-dev libgoogle-perftools4 libgsl0-dev libhdf4-0-alt
  libhdf4-alt-dev libhdf5-dev libjsoncpp-dev libkml0 liblapack-dev libllvm3.6
  libmirclient-dev libmirclient7 libmirclientplatform-mesa libmirprotobuf-dev
  libmirprotobuf0 libode-dev libode1 libogdi3.2 libproj0 libpulse-dev
  libqt5concurrent5 libqt5opengl5-dev libsdl-image1.2 libsdl-image1.2-dev
  libsdl1.2-dev libslang2-dev libsnappy1 libspatialite-dev libspatialite5
  libtcmalloc-minimal4 libunwind8 libv8-3.14.5 libwayland-dev libwebp-dev
  libwebpdemux1 libxerces-c-dev mircommon-dev mongodb mongodb-clients
  mongodb-dev mongodb-server proj-bin proj-data python-bson python-bson-ext
  python-enum34 python-gridfs python-gst0.10 python-mock python-pymongo
  python-pymongo-ext python-sqlalchemy python-sqlalchemy-ext qt5-qmake
  qtbase5-dev qtbase5-dev-tools ros-indigo-arm-navigation
  ros-indigo-audio-common-msgs ros-indigo-battery-notifier
  ros-indigo-constraint-aware-spline-smoother ros-indigo-control-toolbox
  ros-indigo-controller-interface ros-indigo-controller-manager
  ros-indigo-controller-manager-msgs ros-indigo-csm
  ros-indigo-diff-drive-controller ros-indigo-effort-controllers
  ros-indigo-fcl ros-indigo-force-torque-sensor-controller
  ros-indigo-forward-command-controller ros-indigo-gazebo-ros-control
  ros-indigo-gmapping ros-indigo-gripper-action-controller
  ros-indigo-hardware-interface ros-indigo-hector-compressed-map-transport
  ros-indigo-hector-geotiff ros-indigo-hector-geotiff-plugins
  ros-indigo-hector-imu-attitude-to-tf ros-indigo-hector-map-server
  ros-indigo-hector-map-tools ros-indigo-hector-mapping
  ros-indigo-hector-marker-drawing ros-indigo-hector-nav-msgs
  ros-indigo-hector-slam ros-indigo-hector-slam-launch
  ros-indigo-hector-trajectory-server
  ros-indigo-household-objects-database-msgs ros-indigo-hsr-analytic-ik
  ros-indigo-hsr-ant-manipulation ros-indigo-hsr-ant-planner-plugins
  ros-indigo-hsr-common ros-indigo-hsr-description ros-indigo-hsr-kinematics
  ros-indigo-hsr-msgs ros-indigo-hsrb-admin-wui ros-indigo-hsrb-analytic-ik
  ros-indigo-hsrb-angle-sensors-alignment ros-indigo-hsrb-base-controllers
  ros-indigo-hsrb-bumper ros-indigo-hsrb-common ros-indigo-hsrb-common-config
  ros-indigo-hsrb-common-launch ros-indigo-hsrb-control
  ros-indigo-hsrb-controller-config-gazebo ros-indigo-hsrb-controllers
  ros-indigo-hsrb-dashboard ros-indigo-hsrb-description ros-indigo-hsrb-gazebo
  ros-indigo-hsrb-gazebo-bringup ros-indigo-hsrb-gazebo-launch
  ros-indigo-hsrb-gazebo-plugins ros-indigo-hsrb-grasp-planner-plugins
  ros-indigo-hsrb-gripper-controller ros-indigo-hsrb-hardware-interface
  ros-indigo-hsrb-impedance-control ros-indigo-hsrb-imu-sensor-protocol
  ros-indigo-hsrb-interactive-teleop-config
  ros-indigo-hsrb-interactive-teleop-plugins ros-indigo-hsrb-interface-py
  ros-indigo-hsrb-interfaces ros-indigo-hsrb-io-monitor
  ros-indigo-hsrb-manipulation ros-indigo-hsrb-manipulation-launch
  ros-indigo-hsrb-mock-hardware ros-indigo-hsrb-moveit
  ros-indigo-hsrb-moveit-config ros-indigo-hsrb-moveit-customik-tutorial
  ros-indigo-hsrb-moveit-plugins ros-indigo-hsrb-moveit-tutorial
  ros-indigo-hsrb-omnibase-filter ros-indigo-hsrb-planner-plugins
  ros-indigo-hsrb-pseudo-endeffector-controller ros-indigo-hsrb-robot-hardware
  ros-indigo-hsrb-robot-mount-wui ros-indigo-hsrb-rqt
  ros-indigo-hsrb-rqt-joint-impedance ros-indigo-hsrb-rviz-simulator
  ros-indigo-hsrb-servomotor-protocol ros-indigo-hsrb-simulator
  ros-indigo-hsrb-teleop ros-indigo-hsrb-teleop-twist-joy-config
  ros-indigo-hsrb-trajectory-filter ros-indigo-hsrb-user-wui
  ros-indigo-hsrb-voice-wui ros-indigo-hsrb-web-server
  ros-indigo-hsrb-web-tools ros-indigo-hsrb-web-tools-config
  ros-indigo-imu-sensor-controller ros-indigo-interactive-marker-proxy
  ros-indigo-joint-limits-interface ros-indigo-joint-normalization-filters
  ros-indigo-joint-state-controller ros-indigo-joint-trajectory-controller
  ros-indigo-joy ros-indigo-laser-scan-matcher ros-indigo-libccd
  ros-indigo-manipulation-msgs ros-indigo-map-server ros-indigo-mjpeg-server
  ros-indigo-move-base-msgs ros-indigo-moveit-core
  ros-indigo-moveit-fake-controller-manager ros-indigo-moveit-kinematics
  ros-indigo-moveit-msgs ros-indigo-moveit-planners-ompl
  ros-indigo-moveit-ros-manipulation ros-indigo-moveit-ros-move-group
  ros-indigo-moveit-ros-perception ros-indigo-moveit-ros-planning
  ros-indigo-moveit-ros-planning-interface
  ros-indigo-moveit-ros-robot-interaction ros-indigo-moveit-ros-visualization
  ros-indigo-moveit-ros-warehouse ros-indigo-moveit-simple-controller-manager
  ros-indigo-object-recognition-msgs ros-indigo-octomap-msgs ros-indigo-ompl
  ros-indigo-openslam-gmapping ros-indigo-position-controllers
  ros-indigo-realtime-tools ros-indigo-ros-controllers
  ros-indigo-ros-web-video ros-indigo-rosapi ros-indigo-rosauth
  ros-indigo-rosbridge-library ros-indigo-rosbridge-server
  ros-indigo-rqt-joint-trajectory-controller
  ros-indigo-scan-to-cloud-converter ros-indigo-sound-play
  ros-indigo-spline-smoother ros-indigo-srdfdom ros-indigo-tarp3
  ros-indigo-tarp3-urdf ros-indigo-teleop-twist-joy ros-indigo-tf2-eigen
  ros-indigo-tf2-web-republisher ros-indigo-tmc-action-driver
  ros-indigo-tmc-action-manager ros-indigo-tmc-action-move-base
  ros-indigo-tmc-action-planner ros-indigo-tmc-actions
  ros-indigo-tmc-attached-object-publisher ros-indigo-tmc-base-behavior
  ros-indigo-tmc-base-path-follower ros-indigo-tmc-base-path-planner
  ros-indigo-tmc-base-simulator ros-indigo-tmc-base-visualizer
  ros-indigo-tmc-camera-parameters ros-indigo-tmc-catkin-env
  ros-indigo-tmc-cmake-modules ros-indigo-tmc-collision-detector
  ros-indigo-tmc-collision-detector-node ros-indigo-tmc-collision-environment
  ros-indigo-tmc-collision-map-server
  ros-indigo-tmc-collision-trajectory-filter ros-indigo-tmc-common
  ros-indigo-tmc-common-msgs ros-indigo-tmc-control-msgs
  ros-indigo-tmc-database ros-indigo-tmc-depthcloud-encoder
  ros-indigo-tmc-dev-tools ros-indigo-tmc-dictionary
  ros-indigo-tmc-eigen-bridge ros-indigo-tmc-eigen-utils
  ros-indigo-tmc-env-database ros-indigo-tmc-env-db
  ros-indigo-tmc-force-torque-sensor-compensation ros-indigo-tmc-gazebo
  ros-indigo-tmc-gazebo-plugins ros-indigo-tmc-gazebo-worlds
  ros-indigo-tmc-geometric-shapes-msgs ros-indigo-tmc-geometry-msgs
  ros-indigo-tmc-grasp-planner ros-indigo-tmc-grasp-state-recognizer
  ros-indigo-tmc-grid-map-server ros-indigo-tmc-hardware-interface
  ros-indigo-tmc-hmi-msgs ros-indigo-tmc-image-compressor
  ros-indigo-tmc-impedance-control ros-indigo-tmc-interactive-grasp-planner
  ros-indigo-tmc-interactive-teleop ros-indigo-tmc-laser-2d-localizer
  ros-indigo-tmc-linemod ros-indigo-tmc-linemod-node ros-indigo-tmc-lint
  ros-indigo-tmc-manipulation ros-indigo-tmc-manipulation-action
  ros-indigo-tmc-manipulation-msgs ros-indigo-tmc-manipulation-tests
  ros-indigo-tmc-manipulation-types ros-indigo-tmc-manipulation-types-bridge
  ros-indigo-tmc-manipulation-util ros-indigo-tmc-mapping-msgs
  ros-indigo-tmc-marker-based-localizer ros-indigo-tmc-mesh-data-server
  ros-indigo-tmc-motions ros-indigo-tmc-move-base ros-indigo-tmc-msgs
  ros-indigo-tmc-navigation ros-indigo-tmc-navigation-msgs
  ros-indigo-tmc-object-information-publisher ros-indigo-tmc-object-models
  ros-indigo-tmc-object-topology-viewer ros-indigo-tmc-obstacle-grid-mapper
  ros-indigo-tmc-omni-path-follower ros-indigo-tmc-perception
  ros-indigo-tmc-perspective-transformer
  ros-indigo-tmc-placement-area-detector ros-indigo-tmc-planning-msgs
  ros-indigo-tmc-point-cloud ros-indigo-tmc-point-cloud-accumulator
  ros-indigo-tmc-point-cloud-extractor ros-indigo-tmc-point-cloud-util
  ros-indigo-tmc-pose-integrator ros-indigo-tmc-potential-maps
  ros-indigo-tmc-pseudo-velocity-controller ros-indigo-tmc-realtime-tools
  ros-indigo-tmc-rectifier ros-indigo-tmc-remote-exec
  ros-indigo-tmc-road-traversable-recognizer
  ros-indigo-tmc-robot-collision-detector
  ros-indigo-tmc-robot-kinematics-model ros-indigo-tmc-robot-planner
  ros-indigo-tmc-robot-planner-plugins ros-indigo-tmc-robot-rrt-planner-node
  ros-indigo-tmc-ros-tools ros-indigo-tmc-roslaunch-tools
  ros-indigo-tmc-rplanner ros-indigo-tmc-rviz-simulator
  ros-indigo-tmc-safe-pose-changer ros-indigo-tmc-sanyo-battery
  ros-indigo-tmc-simulation ros-indigo-tmc-slam ros-indigo-tmc-speaker
  ros-indigo-tmc-spherical-point-creator
  ros-indigo-tmc-stereo-marker-recognizer ros-indigo-tmc-stl-loader
  ros-indigo-tmc-suction ros-indigo-tmc-tarp3 ros-indigo-tmc-task
  ros-indigo-tmc-task-integration-test ros-indigo-tmc-task-manager
  ros-indigo-tmc-task-msgs ros-indigo-tmc-teleop ros-indigo-tmc-teleop-base
  ros-indigo-tmc-teleop-msgs ros-indigo-tmc-teleop-trajectory
  ros-indigo-tmc-trajectory-offsetter ros-indigo-tmc-utils
  ros-indigo-tmc-viewpoint-controller
  ros-indigo-tmc-virtual-point-cloud-publisher ros-indigo-tmc-vision-msgs
  ros-indigo-tmc-vision-util ros-indigo-tmc-web-helper
  ros-indigo-tmc-web-login ros-indigo-trajectory-filter-server
  ros-indigo-transmission-interface ros-indigo-urdfdom-py
  ros-indigo-velocity-controllers ros-indigo-warehouse-ros sqlite3 tmc-connect
  tmc-libopencv-calib3d-dev tmc-libopencv-calib3d3.0 tmc-libopencv-core-dev
  tmc-libopencv-core3.0 tmc-libopencv-features2d-dev
  tmc-libopencv-features2d3.0 tmc-libopencv-flann-dev tmc-libopencv-flann3.0
  tmc-libopencv-hal-dev tmc-libopencv-highgui-dev tmc-libopencv-highgui3.0
  tmc-libopencv-imgcodecs-dev tmc-libopencv-imgcodecs3.0
  tmc-libopencv-imgproc-dev tmc-libopencv-imgproc3.0 tmc-libopencv-ml-dev
  tmc-libopencv-ml3.0 tmc-libopencv-objdetect-dev tmc-libopencv-objdetect3.0
  tmc-libopencv-photo-dev tmc-libopencv-photo3.0 tmc-libopencv-shape3.0
  tmc-libopencv-stitching-dev tmc-libopencv-stitching3.0
  tmc-libopencv-superres-dev tmc-libopencv-superres3.0 tmc-libopencv-ts-dev
  tmc-libopencv-video-dev tmc-libopencv-video3.0 tmc-libopencv-videoio-dev
  tmc-libopencv-videoio3.0 tmc-libopencv-videostab-dev
  tmc-libopencv-videostab3.0 tmc-libopencv3.0-java tmc-libopencv3.0-jni
  tmc-nodejs tmc-opencv-data tmc-serve-static tmc-webrtc-io xdot
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  ros-indigo-cv3-bridge ros-indigo-tmc-desktop-full ros-indigo-tmc-ikemen
  tmc-libopencv-dev tmc-libopencv-viz-dev tmc-libopencv-viz3.0 tmc-libvtk6
  tmc-libvtk6-dev tmc-libvtk6-java tmc-python-opencv tmc-python-vtk6
  tmc-tcl-vtk6 tmc-vtk6
0 upgraded, 0 newly installed, 13 to remove and 4 not upgraded.
After this operation, 187 MB disk space will be freed.
Do you want to continue? [Y/n] y

[jwnimmer-tri]

/CC @david-german-tri @jamiesnape @mwoehlke-kitware Is this something we can make improvements on?

[liangfok]

One thing worth trying is to re-install TMC's HSR packages then configure Director to build VTK5 from source: http://drake.mit.edu/from_source_ros_kinetic.html#step-5-build-the-workspace.

[kunimatsu-tri]

I ran into this problem before. The problem is the conflict between python-vtk6 and tmc-python-vtk6, and due to the TMC side. As @liangfok suggested, as long as you don't use python-vtk6, it works.

I've already reported this to TMC.

[liangfok]

Also see Step 6 in the webpage linked to below, which manually tweaks a couple environment variables to find VTK5:

http://drake.mit.edu/from_source_ros_kinetic.html#step-6-modify-environment-variables-to-allow-vtk5-libraries-to-be-found

I recall @patmarion might have a fix for this.

[athackst]

OK, that seems to have worked. I recommend adding it to the indigo install steps as well.

[liangfok]

Great! Will do ASAP.

[kunimatsu-tri]

Update:

I found that this was not due to the conflict between tmc-python-vtk6 and python-vtk6, but this is python-vtk6's bug.

Yes, tmc-python-vtk6 introduces the error:

ImportError: No module named vtkCommonCorePython

as @athackst reported, but if I install python-vtk6 via apt-get, it produces the same error when I import vtk.

(trusty)kunimatsu@ubuntu:~$ dpkg -l | grep vtk
ii  libvtk5.8                              5.8.0-14.1ubuntu3                       amd64        Visualization Toolkit - A high level 3D visualization library - runtime
ii  libvtk6                                6.0.0-6                                 amd64        Visualization Toolkit - A high level 3D visualization library - runtime
rc  python-vtk                             5.8.0-14.1ubuntu3                       amd64        Python bindings for VTK
ii  python-vtk6                            6.0.0-6                                 amd64        Python bindings for VTK
ii  tcl-vtk                                5.8.0-14.1ubuntu3                       amd64        Tcl bindings for VTK
(trusty)kunimatsu@ubuntu:~$ ipython
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
Type "copyright", "credits" or "license" for more information.

IPython 1.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import vtk
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-b7e11aadda62> in <module>()
----> 1 import vtk

/usr/lib/python2.7/dist-packages/vtk/__init__.py in <module>()
     37 
     38 # --------------------------------------
---> 39 from vtkCommonCore import *
     40 from vtkCommonMath import *
     41 from vtkCommonMisc import *

/usr/lib/python2.7/dist-packages/vtk/vtkCommonCore.py in <module>()
----> 1 from vtkCommonCorePython import *

ImportError: No module named vtkCommonCorePython

The some people have found this bug already: https://bugs.launchpad.net/ubuntu/+source/vtk6/+bug/1404045

http://vtk.1045678.n5.nabble.com/python-vtk6-ubuntu-package-Fail-to-import-vtk-issue-has-been-reported-on-launchpad-td5729915.html

I'm still not sure what "generate_links.sh" means. Is anyone from Kitware knows the workaround for this?

[david-german-tri]

@kunimatsu-tri If I'm understanding those links correctly, the problem is coming from Ubuntu packaging, not from VTK itself. Is that right?

[david-german-tri]

@liangfok @athackst if fiddling the environment variables to prefer VTK5 over VTK6 is an adequate workaround for the Ubuntu packaging problem, is the only task left on this issue a documentation one?

[liangfok]

Yes, I believe the only task left for this issue is to document the need to modify a couple environment variables. I will submit a PR shortly.

@kunimatsu-tri, are you unable to work around the "ImportError: No module named vtkCommonCorePython" error using the following two environment variable changes:

export LD_LIBRARY_PATH=$HOME/dev/drake_catkin_workspace/install/lib/vtk-5.10:$LD_LIBRARY_PATH
export PYTHONPATH=$HOME/dev/drake_catkin_workspace/build/drake/externals/director/src/vtk-build/Wrapping/Python:/home/liang/dev/drake_catkin_workspace/build/drake/externals/director/src/vtk-build/bin:$PYTHONPATH

[athackst]

That's not entirely true. Just setting the environment variable didn't work for me. I also had to set the flag to compile vtk from source.

[kunimatsu-tri]

@david-german-tri It seems you're right, but I'm still not sure if it's ONLY the packaging problem.

I can't find vtkCommonCorePython.py anywhere. I don't know if this is packaging problem or build configuration problem.

[liangfok]

@athackst: Got it, thanks. I'll include both the flag and the environment variable changes in the PR that resolves this issue.

[kunimatsu-tri]

@liangfok I'm not building drake as a catkin package. Do you know the workaround for such users?

[liangfok]

@kunimatsu-tri, I believe the workaround is the same, but the paths need to be modified. For example, something like:

cd drake-distro/build/externals/director
cmake . -DUSE_SYSTEM_VTK=OFF
cd drake-distro/build
ninja (or cmake)
cd drake-distro
export LD_LIBRARY_PATH=`pwd`/build/install/lib/vtk-5.10:$LD_LIBRARY_PATH
export PYTHONPATH=`pwd`/build/externals/director/src/vtk-build/Wrapping/Python:`pwd`/build/externals/director/src/vtk-build/bin:$PYTHONPATH

[kunimatsu-tri]

@liangfok That worked for me, I needed to modify the last line though:

export PYTHONPATH=`pwd`/build/externals/director/src/vtk-build/Wrapping/Python:`pwd`/build/externals/director/src/vtk-build/bin:$PYTHONPATH

[liangfok]

@kunimatsu-tri: Thanks! I've updated my post and will also update #4765.