search

start-jsk / jsk_apc

Other
36 stars 35 forks source link

catkin_virtualenv fails to construct environment #2716

Closed pazeshun closed 4 years ago

pazeshun commented 4 years ago

https://github.com/start-jsk/jsk_apc/pull/2708#issuecomment-644462660 When adding --verbose to catkin build, I got the following error log:

Collecting labelme>=2.9
Collecting labelme>=2.9
  Using cached https://files.pythonhosted.org/packages/41/70/e6ece2d9b10d8def5f9d755cbcd544ac513092ba3057826f81afa91b4ac9/labelme-4.5.0.tar.gz
  Using cached https://files.pythonhosted.org/packages/41/70/e6ece2d9b10d8def5f9d755cbcd544ac513092ba3057826f81afa91b4ac9/labelme-4.5.0.tar.gz
  Installing build dependencies ...   Installing build dependencies ... -done
  Getting requirements to build wheel ...done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpoGmpCu
       cwd: /tmp/pip-install-WVwLxQ/labelme
  Complete output (18 lines):
  Traceback (most recent call last):
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
      main()
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 91, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 243, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 15, in <module>
      sys.path.extend(site.getsitepackages())
  AttributeError: 'module' object has no attribute 'getsitepackages'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/build/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpoGmpCu Check the logs for full command output.
error
  ERROR: Command errored out with exit status 1:
   command: /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmplUy7zE
       cwd: /tmp/pip-install-rsKzmO/labelme
  Complete output (18 lines):
  Traceback (most recent call last):
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
      main()
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 91, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 243, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 15, in <module>
      sys.path.extend(site.getsitepackages())
  AttributeError: 'module' object has no attribute 'getsitepackages'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmplUy7zE Check the logs for full command output.

Now I'm investigating.

pazeshun commented 4 years ago

Related to https://github.com/pypa/pipenv/issues/3824 and https://github.com/pypa/virtualenv/issues/737???

pazeshun commented 4 years ago

After catkin build again (this doesn't fail but a generated environment is incomplete), I could reproduce the error by pip install labelme==4.5.0:

travis@509ecb423325:/$ source ~/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/bin/activate
(venv) travis@509ecb423325:/$ pip install labelme==4.5.0          
/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting labelme==4.5.0
  Using cached https://files.pythonhosted.org/packages/41/70/e6ece2d9b10d8def5f9d755cbcd544ac513092ba3057826f81afa91b4ac9/labelme-4.5.0.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmplzKEWm
       cwd: /tmp/pip-install-6CBQG5/labelme
  Complete output (18 lines):
  Traceback (most recent call last):
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
      main()
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 91, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 243, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 15, in <module>
      sys.path.extend(site.getsitepackages())
  AttributeError: 'module' object has no attribute 'getsitepackages'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/bin/python /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmplzKEWm Check the logs for full command output.
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

However, pip install labelme==4.4.0 succeeded:

(venv) travis@509ecb423325:/$ pip install labelme==4.4.0
/home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/travis/.cache/pip/wheels/9f/6e/b1/24df8354395e0a6433883005d31e158a97a01b99088e77da3b/labelme-4.4.0-cp27-none-any.whl
Collecting qtpy
  Using cached https://files.pythonhosted.org/packages/cd/fd/9972948f02e967b691cc0ca1f26124826a3b88cb38f412a8b7935b8c3c72/QtPy-1.9.0-py2.py3-none-any.whl
Processing /home/travis/.cache/pip/wheels/bd/b7/5e/9a0ce5c6120fd8e2c2ee5cdbb5974f65ef1912779e1724dc12/imgviz-0.12.3-cp27-none-any.whl
Requirement already satisfied: termcolor in /usr/lib/python2.7/dist-packages (from labelme==4.4.0) (1.1.0)
Requirement already satisfied: Pillow>=2.8.0 in /usr/local/lib/python2.7/dist-packages (from labelme==4.4.0) (6.2.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python2.7/dist-packages (from labelme==4.4.0) (2.2.5)
Requirement already satisfied: PyYAML in /usr/local/lib/python2.7/dist-packages (from labelme==4.4.0) (5.3.1)
Requirement already satisfied: numpy in /usr/local/lib/python2.7/dist-packages (from labelme==4.4.0) (1.16.6)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (2.4.7)
Requirement already satisfied: backports.functools-lru-cache in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (1.6.1)
Requirement already satisfied: subprocess32 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (3.5.4)
Requirement already satisfied: pytz in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (2020.1)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (1.15.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python2.7/dist-packages (from matplotlib->labelme==4.4.0) (0.10.0)
Requirement already satisfied: setuptools in /home/travis/ros/ws_jsk_apc/devel/.private/instance_occlsegm/share/instance_occlsegm/venv/lib/python2.7/site-packages (from kiwisolver>=1.0.1->matplotlib->labelme==4.4.0) (44.1.1)
Installing collected packages: qtpy, imgviz, labelme
Successfully installed imgviz-0.12.3 labelme-4.4.0 qtpy-1.9.0
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

My current assumption is that some change between labelme v4.4.0 and v4.5.0 (https://github.com/wkentaro/labelme/commit/19433fcabfd00b382bf50d511d1f5afc9f4d5df0?) forces to use site.getsitepackages() when labelme is installed. However, site.getsitepackages() is not implemeted on virtualenv, so installation fails. I'll investigate more to find a solution. cc @wkentaro

pazeshun commented 4 years ago

wkentaro/labelme@19433fc?

This seems wrong. Trying pip install -e . tells me installation fails after https://github.com/wkentaro/labelme/commit/5c9808446a179176c721bd7a59d621b41c95898c.

pazeshun commented 4 years ago

Now I've found the problem point where site.getsitepackages() is used: https://github.com/wkentaro/labelme/commit/5c9808446a179176c721bd7a59d621b41c95898c#diff-2eeaed663bd0d25b7e608891384b7298R15

pazeshun commented 4 years ago

This issue seems fixed by https://github.com/wkentaro/labelme/pull/688 and now travis tests of the master branch succeeded.