search

qcr / benchbot

BenchBot is a tool for seamlessly testing & evaluating semantic scene understanding tools in both realistic 3D simulation & on real robots
BSD 3-Clause "New" or "Revised" License
110 stars 12 forks source link

"ERROR: Exception: " about pip #1

Closed liucsg closed 4 years ago

liucsg commented 4 years ago

Hi, Could you tell me how to solve the error occurred while installing , as follows，

Running command: "cmake /benchbot/ros_ws/src -DCATKIN_DEVEL_PREFIX=/benchbot/ros_ws/devel -DCMAKE_INSTALL_PREFIX=/benchbot/ros_ws/install -G Unix Makefiles" in "/benchbot/ros_ws/build"

Running command: "make -j20 -l20" in "/benchbot/ros_ws/build"

/benchbot /benchbot/isaac_sdk /benchbot Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Hit:4 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease Hit:5 http://packages.ros.org/ros/ubuntu bionic InRelease Get:6 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Hit:7 https://packages.lunarg.com/vulkan bionic InRelease Ign:2 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu1804/x86_64 InRelease Hit:8 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu1804/x86_64 Release Get:10 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1161 kB] Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1365 kB] Fetched 2778 kB in 36s (76.4 kB/s) Reading package lists... [INFO] Installing packages - git git-lfs git-review build-essential g++ debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - git git-lfs git-review build-essential g++ [INFO] Installing packages - pkg-config zip zlib1g-dev unzip curl [INFO] Successfully installed packages - pkg-config zip zlib1g-dev unzip curl [INFO] Installing packages - uuid-dev [INFO] Successfully installed packages - uuid-dev [INFO] Installing packages - python python-dev [INFO] Successfully installed packages - python python-dev [INFO] Installing packages - libx11-dev libxcursor-dev libxrandr-dev libxinerama-dev [INFO] Successfully installed packages - libx11-dev libxcursor-dev libxrandr-dev libxinerama-dev [INFO] Installing packages - sox debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - sox [INFO] Installing packages - gcc-aarch64-linux-gnu g++-aarch64-linux-gnu debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - gcc-aarch64-linux-gnu g++-aarch64-linux-gnu [INFO] Installing packages - libvpx5 libtwolame-dev libtheora-bin libspeex1 debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - libvpx5 libtwolame-dev libtheora-bin libspeex1 [INFO] Installing packages - libopus0 libmp3lame0 libvdpau1 openexr libmpg123-0 debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - libopus0 libmp3lame0 libvdpau1 openexr libmpg123-0 [INFO] Installing packages - python3-pip latexmk texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended texlive-luatex texlive-xetex debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - python3-pip latexmk texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended texlive-luatex texlive-xetex [INFO] Installing packages - libudev-dev debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - libudev-dev [INFO] Installing packages - libi2c-dev debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - libi2c-dev [INFO] Installing packages - libhidapi-libusb0 libturbojpeg debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - libhidapi-libusb0 libturbojpeg [INFO] Installing packages - python3-dev libssl-dev [INFO] Successfully installed packages - python3-dev libssl-dev [INFO] Installing packages - python-tk libopenmpi-dev [INFO] Successfully installed packages - python-tk libopenmpi-dev [INFO] Installing packages - python3-sphinx python3-sphinx-rtd-theme debconf: delaying package configuration, since apt-utils is not installed [INFO] Successfully installed packages - python3-sphinx python3-sphinx-rtd-theme Collecting pip Downloading https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB) Installing collected packages: pip Successfully installed pip-20.0.2 Collecting launchpadlib Downloading launchpadlib-1.10.10.tar.gz (196 kB) ERROR: Exception: Traceback (most recent call last): File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 425, in _error_catcher yield File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 507, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read data = self.__fp.read(amt) File "/usr/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/usr/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/usr/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/usr/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 186, in _main status = self.run(options, args) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 331, in run resolver.resolve(requirement_set) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve discovered_reqs.extend(self._resolve_one(requirement_set, req)) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one abstract_dist = self._get_abstract_dist_for(req_to_install) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for abstract_dist = self.preparer.prepare_linked_requirement(req) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement hashes=hashes, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 287, in unpack_url hashes=hashes, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 159, in unpack_http_url link, downloader, temp_dir.path, hashes File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 303, in _download_http_url for chunk in download.chunks: File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/utils/ui.py", line 160, in iter for x in it: File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/network/utils.py", line 39, in response_chunks decode_content=False, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 564, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 529, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/usr/lib/python3.6/contextlib.py", line 99, in exit self.gen.throw(type, value, traceback) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 430, in _error_catcher raise ReadTimeoutError(self._pool, None, "Read timed out.") pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. Collecting setuptools Downloading setuptools-46.1.3-py3-none-any.whl (582 kB) Installing collected packages: setuptools WARNING: The scripts easy_install and easy_install-3.6 are installed in '/home/benchbot/.local/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed setuptools-46.1.3 Collecting tensorflow-gpu==1.13.1 Downloading tensorflow_gpu-1.13.1-cp36-cp36m-manylinux1_x86_64.whl (345.2 MB) ERROR: Exception: Traceback (most recent call last): File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 425, in _error_catcher yield File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 507, in read data = self._fp.read(amt) if not fp_closed else b"" File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read data = self.__fp.read(amt) File "/usr/lib/python3.6/http/client.py", line 459, in read n = self.readinto(b) File "/usr/lib/python3.6/http/client.py", line 503, in readinto n = self.fp.readinto(b) File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into return self.read(nbytes, buffer) File "/usr/lib/python3.6/ssl.py", line 874, in read return self._sslobj.read(len, buffer) File "/usr/lib/python3.6/ssl.py", line 631, in read v = self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 186, in _main status = self.run(options, args) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 331, in run resolver.resolve(requirement_set) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve discovered_reqs.extend(self._resolve_one(requirement_set, req)) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one abstract_dist = self._get_abstract_dist_for(req_to_install) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for abstract_dist = self.preparer.prepare_linked_requirement(req) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement hashes=hashes, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 287, in unpack_url hashes=hashes, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 159, in unpack_http_url link, downloader, temp_dir.path, hashes File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 303, in _download_http_url for chunk in download.chunks: File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/utils/ui.py", line 160, in iter for x in it: File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_internal/network/utils.py", line 39, in response_chunks decode_content=False, File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 564, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 529, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/usr/lib/python3.6/contextlib.py", line 99, in exit self.gen.throw(type, value, traceback) File "/home/benchbot/.local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 430, in _error_catcher raise ReadTimeoutError(self._pool, None, "Read timed out.") pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. Collecting baselines Cloning git://github.com/openai/baselines.git (to revision a071fa7630288ee00c86e140c5e674061f64690d) to /tmp/pip-install-6bz3t95w/baselines Running command git clone -q git://github.com/openai/baselines.git /tmp/pip-install-6bz3t95w/baselines

btalb commented 4 years ago

Thanks for trying out our software @liucsg !

We have seen the above error on rare occasions (& sporadically) on our QUT internet connection, particularly with the tensorflow-gpu package (as it is a large network download). Often simply running the install process again is sufficient to get past the issue (pip will fail once, then not again).

The root cause of the issue seems to be that pip offers no reliability guarantees under network instabilities.

I have spent the last couple of days trying to reproduce this error under any condition that I can create: pip2, pip3, computers on our QUT network, computers outside the QUT network, inside Docker images, directly on the host, etc. I haven't been able to deliberately cause the issue at all, even with bad options like pip --default-timeout=1 install tensorflow-gpu.

Given I can't reproduce, I can't offer solid suggestions on how to fix at the moment. In your output above the error is being thrown twice (while installing both launchpadlib & tensorflow-gpu). Given launchpadlib is an extremely small package, I believe it is network configuration related. Can I get some more information to help you try & isolate the cause:

Aside from the above information, can I suggest trying the following to see if the network errors are sporadic & we can get around them:

  1. Retry installation from that step: 
    ./install
  2. Retry installation from scratch: 
    ../install --force-clean
  3. Check if there are any pip installation issues directly on your machine: 
    pip3 install tensorflow-gpu==1.13.1
  4. Check if pip installation issues occur in a fresh Docker image: 
    docker run -it ubuntu:bionic /bin/bash -c 'apt update && apt install -y python3-pip && pip3 install tensorflow-gpu==1.13.1'
  5. A lot of similar issues I could find reference the --default-timeout option for pip: https://askubuntu.com/questions/905196/pip-fails-with-readtimeouterror. See if setting that to a large value can help fix the issue locally on your machine. If that's the case, I can look at how to include it in the install process.
liucsg commented 4 years ago

Hi, btalb, Thanks for your detailed answers ! I have two choices of network to install benchbot, as following, 1) wired network with proxy; 2) corporate wifi

It is unstable when used corporate wifi, problem occured as above, although i have tried many times. Now, I select wired network to install.There is a proxy between my compueter and remote destinations, so i have to use my proxy in Dockerfile. I am solving a problem occured in simulator.Dockfile when installing ROS Melodic, as following,

Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [57.5 kB] Err:10 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu1804/x86_64 Release Certificate verification failed: The certificate is NOT trusted. The certificate issuer is unknown. Could not handshake: Error in the certificate verification. [IP: 172.30.65.47 8080] Reading package lists... E: The repository 'http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release' no longer has a Release file.

Is there any other manners to install benchbot for my problems due to the network?

btalb commented 4 years ago

In that case, it unfortunately sounds like an issue with the underlying connection.

Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [57.5 kB] Err:10 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu1804/x86_64 Release Certificate verification failed: The certificate is NOT trusted. The certificate issuer is unknown. Could not handshake: Error in the certificate verification. [IP: 172.30.65.47 8080] Reading package lists... E: The repository 'http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release' no longer has a Release file.

This error is again due to a connectivity issue: performing the handshake verifying the certificate for the Nvidia CUDA repository is failing. It would seem to be a routing issues as address 172.30.65.47:8080 is a local IP.

Is there any other manners to install benchbot for my problems due to the network?

Unfortunately we can't provide an install method that avoids network connections to 3rd party locations. The install process downloads ~17GB of software from 100s of different 3rd party sources; providing methods that essentially bypassed installation from these locations would inevitably breach licensing agreements.