start-jsk / jsk_apc

Other
36 stars 35 forks source link

Fix errors in building grasp_fusion and instance_occlsegm #2699

Closed pazeshun closed 5 years ago

pazeshun commented 5 years ago

Depends on #2698 Updated version of #2697

Add Cython to requirements for installing pycocotools

Problem & Solution

When building jsk_apc with .travis.rosinstall and .travis.rosinstall.kinetic, I encountered the following error at instance_occlsegm:

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-W46fSl/pycocotools/

This is because Cython is not automatically installed before installing pycocotools:

$ pip install --user pycocotools
Collecting pycocotools
  Downloading https://files.pythonhosted.org/packages/96/84/9a07b1095fd8555ba3f3d519517c8743c2554a245f9476e5e39869f948d2/pycocotools-2.0.0.tar.gz (1.5MB)
    100% |████████████████████████████████| 1.5MB 308kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-c36GAZ/pycocotools/setup.py", line 2, in <module>
        from Cython.Build import cythonize
    ImportError: No module named Cython.Build

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-c36GAZ/pycocotools/

This is fixed in latest pycocotools but not released (see https://github.com/cocodataset/cocoapi/issues/327). So I tried to install pycocotools from source in catkin_virtualenv, but failed due to https://github.com/locusrobotics/catkin_virtualenv/issues/48. Therefore, I add Cython to requirements.txt of instance_occlsegm. Cython installation completes before pycocotools thanks to alphabetical order.

Why travis doesn't catch this problem?

Because docker container used in travis already includes Cython:

$ sudo docker ps -a
CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                          PORTS               NAMES
186863aa12b0        jskrobotics/jsk_apc:kinetic-latest   "/ros_entrypoint.sh …"   30 minutes ago      Exited (0) About a minute ago                       sharp_jackson
...
root@186863aa12b0:/# pip show cython
Name: Cython
Version: 0.23.4
Summary: The Cython compiler for writing C extensions for the Python language.
Home-page: http://cython.org/
Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al.
Author-email: cython-devel@python.org
License: UNKNOWN
Location: /usr/lib/python2.7/dist-packages
Requires:

Avoid skipping installed python requirements which is incompatible

Problem

I encountered the following red lines at instance_occlsegm and grasp_fusion after solving the above issue:

opencv-python-headless 4.1.1.26 has requirement numpy>=1.11.1, but you'll have numpy 1.11.0 which is incompatible.
imgaug 0.3.0 has requirement numpy>=1.15, but you'll have numpy 1.11.0 which is incompatible.
imgaug 0.3.0 has requirement scikit-image>=0.14.2, but you'll have scikit-image 0.10.1 which is incompatible.
pandas 0.24.2 has requirement numpy>=1.12.0, but you'll have numpy 1.11.0 which is incompatible.
opencv-python-headless 4.1.1.26 has requirement numpy>=1.11.1, but you'll have numpy 1.11.0 which is incompatible.
imgaug 0.3.0 has requirement numpy>=1.15, but you'll have numpy 1.11.0 which is incompatible.
imgaug 0.3.0 has requirement scikit-image>=0.14.2, but you'll have scikit-image 0.10.1 which is incompatible.
pandas 0.24.2 has requirement numpy>=1.12.0, but you'll have numpy 1.11.0 which is incompatible.

(Related to https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025 ?)

Solution

numpy and scikit-image are installed via apt, so I add argument --upgrade --upgrade-strategy=eager to pip running in catkin_virtualenv. By this, latest numpy and scikit-image are installed in catkin_virtualenv. I wanted to use --upgrade-strategy=only-if-needed, but when I used that, I encountered another error which I comment in CMakeLists.txt.

pazeshun commented 5 years ago

@knorth55 I think this PR affects environments in catkin_virtualenv only and doesn't damage system (e.g., rosdep install). Could you review this?