HACK: Add python-virtualenv apt package dependency

[blag]

This PR is a nasty hack to get the packaging tests to pass - adding python-virtualenv as a dependency to ensure that the virtualenv binary is available to StackStorm when it goes to create a virtualenv when it's installing a non-system pack.

This is a nasty, nasty hack, because the version of virtualenv that it installs is 15.0.1 on Ubuntu Xenial and 15.1.0 [look closely at those version strings, they are different], but only 15.1.0 officially supports Python 3.6 virtualenvs. While this "works" and passes tests, there is no guarantee that it will work in production for ST2 v3.4 on Ubuntu Xenial, so I opened this in a separate PR so we can decide if we are okay with merging this hack.

The proper fix is to refactor StackStorm to use Python 3's built-in venv module to create pack virtualenvs. Once that is done, then the python-virtualenv package dependency should be removed. I have not investigated this yet, but we may need to install the python3-venv package to ensure that StackStorm has the venv module available. I'm not sure that that module is available on PyPI/installable via pip. More investigation is needed.

[amanda11]

@blag Either way, if this is going in independantly of the pip upgrade we need to remove the call to the legacyresolver as it's failing CircleCI. Did we want to make this PR off master branch rather than the one we're using for pip upgrade?

Also don't we end up with virtualenv being 16.6.0 as that's the version that is specifed in the st2 packages.

When I was testing on CentOS and updated the virtualenv version I found that the version of virtualenv in /opt/stackstorm/st2/bin/virtualenv was the version specified by the st2/fixed-reuquirements. This is different to the virtualenv version on the system. So I'm not sure if this is required.

Can we check if we end up with the right version in /opt/stacktorm/st2/bin/virtualenv. I think perhaps this is just the venv

[blag]

From what I saw in the packaging tests for Ubuntu Bionic, the virtualenv module just shells out to the system's virtualenv binary to create a virtualenv.

I believe that Python 3's venv module is different, and actually creates the virtualenv directly. I have not verified this though.

[amanda11]

From when I tested on a real system then I thought we ended up with the virtualenv module from st2 fixed-requirements get placed into /opt/stackstorm/st2/bin/virtualenv and then that was what was used to create the package's virtualenv environments. (Note you have to look as the tests are running as it seems to clean up afterwards...) @armab may know more - as I have only found these things from trial and error.

[amanda11]

So something weird is going on...

1. Master dockerfiles and master st2 code - run the docker-compose on st2-packages and all works. Verified that in the test container that the only virtualenv installed is the one in /opt/stackstorm/st2/bin. But with that setup the tests pass including the creation of hubot.
2. With the branches we have for the upgrade to virtualenv. Again the only virtualenv around is the /opt/stackstorm/st2/bin version, but in this case I can reproduce the complain by st2common about needing it in the system directories:

   st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Creating virtualenv for pack "hubot" in "/opt/stackstorm/virtualenvs/hubot"
   Traceback (most recent call last):
   File "/opt/stackstorm/st2/lib/python3.6/site-packages/python_runner/python_action_wrapper.py", line 334, in <module>
   obj.run()
   File "/opt/stackstorm/st2/lib/python3.6/site-packages/python_runner/python_action_wrapper.py", line 193, in run
   output = action.run(**self._parameters)
   File "/opt/stackstorm/packs/packs/actions/pack_mgmt/setup_virtualenv.py", line 89, in run
   no_download=no_download)
   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/virtualenvs.py", line 94, in setup_pack_virtualenv
   no_download=no_download)
   File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/virtualenvs.py", line 154, in create_virtualenv
   raise Exception('Virtualenv binary "%s" doesn\\'t exist.' % (virtualenv_binary))
   Exception: Virtualenv binary "/usr/bin/virtualenv" doesn't exist.

So can confirm there is a problem but it shouldn't need to be resolved this way I think...

[amanda11]

I think the problem on the PRs is that for some reason /opt/stackstorm/st2/bin/python is a link to /usr/bin/python3.6 - and that's why its trying to find /usr/bin/virtualenv. In the master branch the xenial /opt/stackstorm/st2/bin/python is a link to the python3.6 in the /opt/stackstorm/st2/bin directory.

st2common works out the path to virtualenv based on taking the path of sys.executable, and as it's linked to /usr/bin that's the problem.

So problem in the PRs we are working on to up the virtualenv, rather than on master...