search

StackStorm / st2-packages

StackStorm deb/rpm packages (automated docker build pipeline)
https://stackstorm.com/
27 stars 59 forks source link

Upgrade venv and pip #684

Closed amanda11 closed 3 years ago

amanda11 commented 3 years ago

Updates to use pip 20.3.4. Using its legacy-resolver option.

amanda11 commented 3 years ago

Packages are find on EL7 and EL8. Problems building on Xenial. The --no-site-packages option is being added to the virtualenv call, and this is not supported in virtualenv >=20.0. It should be controllable by setting DH_VIRTUALENV_ARGUMENTS to "", so that it doesn't set it to --no-site-packages which is the default. But https://github.com/spotify/dh-virtualenv/commit/795e1a94d895dc215b433b3f80b47ac26420ab13 shows that it was being hardcoded unless system-packages option was being set. This commit is not in our fork of dh-virtualenv.

amanda11 commented 3 years ago

The --no-site-packages was fixed by @blag create a new branch of dh-virtualenv that didn't set it.

This then led to problems whereby /opt/stackstorm/st2/bin/python was a sym link to /usr/bin/python. So the pack install hubot failed, as the path to the executables was found from the python sys.executalbe, which when sym linked meant it looked for files in /usr/bin instead of /opt/stackstorm/st2/bin.

Using the --copies virtualenv option fixes this. Verified on U18. Will push commit up once U16 test finished.

amanda11 commented 3 years ago

NB. The packages relies on the PR https://github.com/StackStorm/st2packaging-dockerfiles/pull/98, so anticipate failures until the relevant st2packaging-dockerfiles change is merged.

amanda11 commented 3 years ago

st2packaging-dockerfiles PR now merged. Just waiting for the new dockerfiles to be uploaded to https://hub.docker.com/r/stackstorm/packagingbuild/tags?page=1&ordering=last_updated, and then will reset off the circleCI.

amanda11 commented 3 years ago

This will fail until we get the https://github.com/StackStorm/st2/pull/5123 fix in, as we need the change to fixate-requirements.py. But equally if we just take that PR we then get problems as we get pip and virtualenv problems. So these two PRs need to go in together.