`./pants` does not work out of the box - no `requests-sessions`

[jsirois]

Looks like so on a fresh clone:

jsirois@gill ~/dev/3rdparty/fsqio (master) $ git log -1
commit 837a4965444334a849f539ed20f8b6a7f2d46af9
Author: John Gallagher <john@foursquare.com>
Date:   Fri Jan 8 14:41:08 2016 -0500

    Update fhttp readme for fsq.io

    (sapling split of 15944090750f351ba99b4bee763c84019a33affc)
jsirois@gill ~/dev/3rdparty/fsqio (master) $ git clean -fdx && ./pants pom-resolve
++++ which python2.7
+++ PYTHON=/home/jsirois/.pyenv/shims/python2.7
+++ PANTS_HOME=/home/jsirois/.cache/pants/setup
+++ PANTS_BOOTSTRAP=/home/jsirois/.cache/pants/setup/bootstrap
+++ VENV_VERSION=13.1.2
+++ VENV_PACKAGE=virtualenv-13.1.2
+++ VENV_TARBALL=virtualenv-13.1.2.tar.gz
+++ FOURSQUARE_REQUIREMENTS=3rdparty/python/requirements.txt
++++ bootstrap_pants
++++ pants_requirement=pantsbuild.pants
+++++ grep -E '^[[:space:]]*pants_version' pants.ini
+++++ sed -E 's|^[[:space:]]*pants_version[:=][[:space:]]*([^[:space:]]+)|\1|'
++++ pants_version=0.0.65
++++ [[ -n 0.0.65 ]]
++++ pants_requirement=pantsbuild.pants==0.0.65
++++ [[ ! -d /home/jsirois/.cache/pants/setup/bootstrap/0.0.65 ]]
++++ echo /home/jsirois/.cache/pants/setup/bootstrap/0.0.65
+++ pants_dir=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65
+++ export PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
+++ PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
+ '[' '' == --help ']'
+ '[' '' == force ']'
+ '[' '' '!=' '' ']'
+ exec ./scripts/upkeep/check.sh
++++ PYTHON=/home/jsirois/.pyenv/shims/python2.7
++++ PANTS_HOME=/home/jsirois/.cache/pants/setup
++++ PANTS_BOOTSTRAP=/home/jsirois/.cache/pants/setup/bootstrap
++++ VENV_VERSION=13.1.2
++++ VENV_PACKAGE=virtualenv-13.1.2
++++ VENV_TARBALL=virtualenv-13.1.2.tar.gz
++++ FOURSQUARE_REQUIREMENTS=3rdparty/python/requirements.txt
+++++ bootstrap_pants
+++++ pants_requirement=pantsbuild.pants
++++++ grep -E '^[[:space:]]*pants_version' pants.ini
++++++ sed -E 's|^[[:space:]]*pants_version[:=][[:space:]]*([^[:space:]]+)|\1|'
+++++ pants_version=0.0.65
+++++ [[ -n 0.0.65 ]]
+++++ pants_requirement=pantsbuild.pants==0.0.65
+++++ [[ ! -d /home/jsirois/.cache/pants/setup/bootstrap/0.0.65 ]]
+++++ echo /home/jsirois/.cache/pants/setup/bootstrap/0.0.65
++++ pants_dir=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65
++++ export PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
++++ PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
+ set -e
+ shopt -s nullglob
+ ran=
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
++++ which python2.7
+++ PYTHON=/home/jsirois/.pyenv/shims/python2.7
+++ PANTS_HOME=/home/jsirois/.cache/pants/setup
+++ PANTS_BOOTSTRAP=/home/jsirois/.cache/pants/setup/bootstrap
+++ VENV_VERSION=13.1.2
+++ VENV_PACKAGE=virtualenv-13.1.2
+++ VENV_TARBALL=virtualenv-13.1.2.tar.gz
+++ FOURSQUARE_REQUIREMENTS=3rdparty/python/requirements.txt
++++ bootstrap_pants
++++ pants_requirement=pantsbuild.pants
+++++ grep -E '^[[:space:]]*pants_version' pants.ini
+++++ sed -E 's|^[[:space:]]*pants_version[:=][[:space:]]*([^[:space:]]+)|\1|'
++++ pants_version=0.0.65
++++ [[ -n 0.0.65 ]]
++++ pants_requirement=pantsbuild.pants==0.0.65
++++ [[ ! -d /home/jsirois/.cache/pants/setup/bootstrap/0.0.65 ]]
++++ echo /home/jsirois/.cache/pants/setup/bootstrap/0.0.65
+++ pants_dir=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65
+++ export PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
+++ PANTSBINARY=/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants
+ '[' -n '' ']'
+ '[' -z /home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants ']'
+ export PYTHONPATH=src/python
+ PYTHONPATH=src/python
+ exec /home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants --print-exception-stacktrace pom-resolve
Traceback (most recent call last):
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/extension_loader.py", line 140, in load_backend
    module = importlib.import_module(backend_module)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/jsirois/dev/3rdparty/fsqio/src/python/fsqio/pants/register.py", line 15, in <module>
    from fsqio.pants.pom.pom_resolve import PomResolve
  File "/home/jsirois/dev/3rdparty/fsqio/src/python/fsqio/pants/pom/pom_resolve.py", line 39, in <module>
    from requests_futures.sessions import FuturesSession
ImportError: No module named requests_futures.sessions

Exception caught: (<class 'pants.base.exceptions.BackendConfigurationError'>)
  File "/home/jsirois/.cache/pants/setup/bootstrap/0.0.65/bin/pants", line 11, in <module>
    sys.exit(main())
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/pants_exe.py", line 26, in main
    LocalPantsRunner(exiter).run()
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/pants_runner.py", line 60, in run
    self._maybe_profiled(self._run)
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/pants_runner.py", line 57, in _maybe_profiled
    runner()
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/pants_runner.py", line 65, in _run
    options, build_config = OptionsInitializer(options_bootstrapper, exiter=self.exiter).setup()
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/goal_runner.py", line 128, in setup
    return self._setup_options(self._options_bootstrapper, self._working_set)
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/goal_runner.py", line 102, in _setup_options
    build_configuration = load_plugins_and_backends(plugins, working_set, backend_packages)
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/extension_loader.py", line 36, in load_plugins_and_backends
    load_build_configuration_from_source(build_configuration, additional_backends=backends or [])
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/extension_loader.py", line 126, in load_build_configuration_from_source
    load_backend(build_configuration, backend_package)
  File "/home/jsirois/.cache/pants/setup/bootstrap/pants.4sOId7/install/lib/python2.7/site-packages/pants/bin/extension_loader.py", line 144, in load_backend
    .format(backend=backend_module, error=e))

Exception message: Failed to load the fsqio.pants.register backend: No module named requests_futures.sessions

So the build script either needs to bootstrap a venv with 3rdparty/python/requirements.txt or else instructions need to be added to the README to do some out of band manual setup.

[mateor]

Hello, John, how's things : )

Sooo...I had this whole thing where I was asking you what OS you were running and all that....but I see the issue. We thank you in our bootstrap script because that script is basically identical to your pants setup script you made. Since you already have ran that script for 0065, you already have a 0065 virtualenv in your home directory so it never installs the FOURSQUARE_REQUIREMENTS. I will push an update to the bootstrap scripts soon but the fix is generally to just differentiate upstream pants virtualenv from the fsqio one.

A random contributor to Fsqio (patrick) feels pretty strongly we should keep the virtualenv under the repo root, I guess this is an argument in his favor : )

On the plus side, you inspired me to go out and test this on some Linux images. I will update the README but you will need python-dev, JDK8 and libpq-dev and perhaps build-essential to run on linux. I will ping/close this issue when I push the fix, but you can probably get around this by passing PANTS_HOME to a location under the buildroot.

[mateor]

I think we have the fix for this pushed...thanks for the report!

https://github.com/foursquare/fsqio/commit/d9396ca5c833e0e57d3a008d6bb8ca1e024f5bf7