search

RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.32k stars 1.26k forks source link

mac install_prereqs.sh calls pip2 and pip3, but does not install them. #11631

Closed RussTedrake closed 5 years ago

RussTedrake commented 5 years ago

As a result, the script can fail (in a way was silent enough for me to miss in my haste) at https://github.com/RobotLocomotion/drake/blob/master/setup/mac/binary_distribution/install_prereqs.sh#L29-L30 if one of them is not installed apriori.

cc @EricCousineau-TRI

jamiesnape commented 5 years ago

It does install them. Type brew info python and brew info python@2 for more details.

EricCousineau-TRI commented 5 years ago

Tracing it out, setup/mac: https://github.com/RobotLocomotion/drake/blob/2e603a70ff6a08e7597dc5a83902a789dd1117bc/setup/mac/binary_distribution/Brewfile#L22 https://github.com/Homebrew/homebrew-core/blob/6ad0a51caf70354d83f440ad6e7259664a58693e/Formula/python.rb#L40-L43

It's still unclear what parts failed that made, so it's still kinda unclear what action to take.

@russtedrake If pip@2 is still not installed, can you rerun and post the full output of your provisioning script? (and if it's not part of Drake, can you post a link to that script?)

RussTedrake commented 5 years ago

It's definitely reproducible and still failing:

Russs-MacBook-Pro:drake russt$ ./setup/mac/install_prereqs.sh
+ source ./setup/mac/binary_distribution/install_prereqs.sh
++ set -euxo pipefail
++ [[ 501 -eq 0 ]]
++ command -v conda
++ command -v /usr/local/bin/brew
++ /usr/local/bin/brew update
Updated 5 taps (dreal-deps/ibex, homebrew/cask-versions, homebrew/core, homebrew/bundle and dreal/dreal).
==> New Formulae
scala@2.12                                                                          swig@3
==> Updated Formulae
dreal-deps/ibex/ibex@2.7.4 ✔     encfs                            imagemagick@6                    nomad                            sops
dreal/dreal/dreal ✔              envconsul                        jdupes                           opa                              telegraf
glib ✔                           exiftool                         jfrog-cli-go                     paket                            terraform
armadillo                        flow                             jhipster                         phpunit                          terragrunt
atlantis                         fluxctl                          joplin                           procs                            tokei
aws-sdk-cpp                      folly                            knot                             pulumi                           vultr
bitrise                          gibo                             libev                            pybind11                         webpack
braid                            glooctl                          librealsense                     pyenv                            whois
calicoctl                        gmic                             libsigc++                        rbspy                            wildfly-as
certbot                          go                               lmod                             scala                            wtf
cfn-lint                         godep                            macvim                           scalariform                      yelp-tools
chakra                           graph-tool                       mesa                             scons                            you-get
circleci                         gst-plugins-good                 mighttpd2                        scrcpy
citus                            hlint                            minio                            serverless
doctl                            imagemagick                      minio-mc                         ship
==> Deleted Formulae
scala@2.10                                                                          swig@3.04
++ /usr/local/bin/brew uninstall --force embree ospray vtk@8.1
++ /usr/local/bin/brew uninstall --force ipopt
++ /usr/local/bin/brew bundle --file=./setup/mac/binary_distribution/Brewfile
Using dreal/dreal
Using robotlocomotion/director
Using boost
Using cmake
Installing dreal/dreal/dreal
Using gcc
Using gflags
Using glew
Installing glib
Using robotlocomotion/director/ipopt@3.12
Using libyaml
Using lz4
Using nlopt
Error: Could not symlink lib/python2.7/site-packages/numpy/LICENSE.txt
Target /usr/local/lib/python2.7/site-packages/numpy/LICENSE.txt
already exists. You may want to remove it:
  rm '/usr/local/lib/python2.7/site-packages/numpy/LICENSE.txt'

To force the link and overwrite all conflicting files:
  brew link --overwrite numpy

To list all files that would be deleted:
  brew link --overwrite --dry-run numpy
Linking /usr/local/Cellar/numpy/1.16.4...
Using numpy
Using openblas
Using protobuf
Using python
Using python@2
Using scipy
Using tinyxml
Using tinyxml2
Using robotlocomotion/director/vtk@8.2
Using yaml-cpp
Using zeromq
Homebrew Bundle complete! 24 Brewfile dependencies now installed.
++ /usr/local/bin/pip2 install --upgrade --requirement ./setup/mac/binary_distribution/requirements.txt
./setup/mac/binary_distribution/install_prereqs.sh: line 29: /usr/local/bin/pip2: No such file or directory
RussTedrake commented 5 years ago

Somehow on this machine, I do have python2, but not pip2.

jamiesnape commented 5 years ago

Try brew link --overwrite numpy && brew reinstall python && brew reinstall python@2.

RussTedrake commented 5 years ago 
Russs-MacBook-Pro:build russt$ brew reinstall python@2
==> Reinstalling python@2
Error: Xcode alone is not sufficient on Mojave.
Install the Command Line Tools:
  xcode-select --install

Russs-MacBook-Pro:build russt$ xcode-select --install
xcode-select: note: install requested for command line developer tools
Russs-MacBook-Pro:build russt$ brew reinstall python@2
==> Reinstalling python@2
==> Downloading https://homebrew.bintray.com/bottles/python@2-2.7.16.mojave.bottle.1.tar.gz
==> Downloading from https://akamai.bintray.com/cb/cb90a15faf89116993fd85c330069965aefe2d5c6b189a594868d03a34c94aaa?__gda__=exp=1560451257~hmac=8d13ac245e4af3bbc986c4c
######################################################################## 100.0%
==> Pouring python@2-2.7.16.mojave.bottle.1.tar.gz
==> /usr/local/Cellar/python@2/2.7.16/bin/python -s setup.py --no-user-cfg install --force --verbose --single-version-externally-managed --record=installed.txt --insta
Last 15 lines from /Users/russt/Library/Logs/Homebrew/python@2/post_install.01.python:
copying pkg_resources/_vendor/packaging/_compat.py -> build/lib/pkg_resources/_vendor/packaging
copying pkg_resources/_vendor/packaging/specifiers.py -> build/lib/pkg_resources/_vendor/packaging
copying setuptools/script (dev).tmpl -> build/lib/setuptools
copying setuptools/script.tmpl -> build/lib/setuptools
copying setuptools/cli-64.exe -> build/lib/setuptools
copying setuptools/gui-64.exe -> build/lib/setuptools
copying setuptools/cli.exe -> build/lib/setuptools
copying setuptools/cli-32.exe -> build/lib/setuptools
copying setuptools/gui-32.exe -> build/lib/setuptools
copying setuptools/gui.exe -> build/lib/setuptools
copying setuptools/command/launcher manifest.xml -> build/lib/setuptools/command
running install_lib
copying build/lib/easy_install.py -> /usr/local/lib/python2.7/site-packages
copying build/lib/setuptools/ssl_support.py -> /usr/local/lib/python2.7/site-packages/setuptools
error: could not delete '/usr/local/lib/python2.7/site-packages/setuptools/ssl_support.py': Permission denied
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall python@2`
==> Caveats
Pip and setuptools have been installed. To update them
  pip install --upgrade pip setuptools

You can install Python packages with
  pip install <package>

They will install into the site-package directory
  /usr/local/lib/python2.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python@2/2.7.16: 3,343 files, 45.9MB
Removing: /usr/local/Cellar/python@2/2.7.14_3... (4,618 files, 82.3MB)
Removing: /usr/local/Cellar/python@2/2.7.15... (4,742 files, 84.1MB)
Removing: /usr/local/Cellar/python@2/2.7.15_1... (4,776 files, 84.6MB)
Russs-MacBook-Pro:build russt$ pip2
-bash: pip2: command not found
Russs-MacBook-Pro:build russt$ /usr/local/bin/pip2
-bash: /usr/local/bin/pip2: No such file or directory
RussTedrake commented 5 years ago

You version didn't work either: 

Russs-MacBook-Pro:build russt$ brew link --overwrite numpy && brew reinstall python && brew reinstall python@2
Linking /usr/local/Cellar/numpy/1.16.4... 885 symlinks created
==> Reinstalling python
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.3.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/25/25e0099852136c4ef1efd221247d0f67aa71f7b624211b98898f8b46c612f40d?__gda__=exp=1560451322~hmac=f6a2c6c89869c0da5c2994a
######################################################################## 100.0%
==> Pouring python-3.7.3.mojave.bottle.tar.gz
==> /usr/local/Cellar/python/3.7.3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.3/bin --install-lib=/
==> /usr/local/Cellar/python/3.7.3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.3/bin --install-lib=/
==> /usr/local/Cellar/python/3.7.3/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.3/bin --install-lib=/
==> Caveats
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin

If you need Homebrew's Python 2.7 run
  brew install python@2

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python/3.7.3: 3,863 files, 59.8MB
==> Reinstalling python@2
==> Downloading https://homebrew.bintray.com/bottles/python@2-2.7.16.mojave.bottle.1.tar.gz
Already downloaded: /Users/russt/Library/Caches/Homebrew/downloads/d7439d403969848d192da11358322f7eaba24694652c70cd2c4aeb4e16a3b8d6--python@2-2.7.16.mojave.bottle.1.tar.gz
==> Pouring python@2-2.7.16.mojave.bottle.1.tar.gz
==> /usr/local/Cellar/python@2/2.7.16/bin/python -s setup.py --no-user-cfg install --force --verbose --single-version-externally-managed --record=installed.txt --insta
Last 15 lines from /Users/russt/Library/Logs/Homebrew/python@2/post_install.01.python:
copying pkg_resources/_vendor/packaging/_compat.py -> build/lib/pkg_resources/_vendor/packaging
copying pkg_resources/_vendor/packaging/specifiers.py -> build/lib/pkg_resources/_vendor/packaging
copying setuptools/script (dev).tmpl -> build/lib/setuptools
copying setuptools/script.tmpl -> build/lib/setuptools
copying setuptools/cli-64.exe -> build/lib/setuptools
copying setuptools/gui-64.exe -> build/lib/setuptools
copying setuptools/cli.exe -> build/lib/setuptools
copying setuptools/cli-32.exe -> build/lib/setuptools
copying setuptools/gui-32.exe -> build/lib/setuptools
copying setuptools/gui.exe -> build/lib/setuptools
copying setuptools/command/launcher manifest.xml -> build/lib/setuptools/command
running install_lib
copying build/lib/easy_install.py -> /usr/local/lib/python2.7/site-packages
copying build/lib/setuptools/ssl_support.py -> /usr/local/lib/python2.7/site-packages/setuptools
error: could not delete '/usr/local/lib/python2.7/site-packages/setuptools/ssl_support.py': Permission denied
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall python@2`
==> Caveats
Pip and setuptools have been installed. To update them
  pip install --upgrade pip setuptools

You can install Python packages with
  pip install <package>

They will install into the site-package directory
  /usr/local/lib/python2.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python@2/2.7.16: 3,343 files, 45.9MB
Russs-MacBook-Pro:build russt$ /usr/local/bin/pip2
-bash: /usr/local/bin/pip2: No such file or directory
RussTedrake commented 5 years ago

looks like the relevant line is 

error: could not delete '/usr/local/lib/python2.7/site-packages/setuptools/ssl_support.py': Permission denied
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall python@2`
jamiesnape commented 5 years ago

Yes. Remove /usr/local/lib/python2.7/site-packages/setuptools and try again.

jamiesnape commented 5 years ago

Also remove /usr/local/lib/python2.7/site-packages/pip as well as /usr/local/lib/python2.7/site-packages/pip-*-py2.7.egg-info and /usr/local/lib/python2.7/site-packages/setuptools-*-py2.7.egg-info if they exist.

RussTedrake commented 5 years ago

i had to keep crawling up the tree. this finally worked:

Russs-MacBook-Pro:build russt$ sudo rm -rf /usr/local/lib/python2.7
Russs-MacBook-Pro:build russt$ brew reinstall python@2
==> Reinstalling python@2
==> Downloading https://homebrew.bintray.com/bottles/python@2-2.7.16.mojave.bottle.1.tar.gz
Already downloaded: /Users/russt/Library/Caches/Homebrew/downloads/d7439d403969848d192da11358322f7eaba24694652c70cd2c4aeb4e16a3b8d6--python@2-2.7.16.mojave.bottle.1.tar.gz
==> Pouring python@2-2.7.16.mojave.bottle.1.tar.gz
==> /usr/local/Cellar/python@2/2.7.16/bin/python -s setup.py --no-user-cfg install --force --verbose --single-version-externally-managed --record=installed.txt --insta
==> /usr/local/Cellar/python@2/2.7.16/bin/python -s setup.py --no-user-cfg install --force --verbose --single-version-externally-managed --record=installed.txt --insta
==> /usr/local/Cellar/python@2/2.7.16/bin/python -s setup.py --no-user-cfg install --force --verbose --single-version-externally-managed --record=installed.txt --insta
==> Caveats
Pip and setuptools have been installed. To update them
  pip install --upgrade pip setuptools

You can install Python packages with
  pip install <package>

They will install into the site-package directory
  /usr/local/lib/python2.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python@2/2.7.16: 3,706 files, 50.6MB
Russs-MacBook-Pro:build russt$ pip2

Usage:
  pip2 <command> [options]
...

Not sure how i got in that state. But I guess there is a record here now if others hit it.

jamiesnape commented 5 years ago

You may need brew reinstall numpy protobuf.