jenkinsci / pyenv-pipeline-plugin

Execute commands in Python virtualenvs in Jenkins Pipeline DSL
https://plugins.jenkins.io/pyenv-pipeline/
MIT License
33 stars 15 forks source link

Multibranch pipelines can cause maximum shebang length in pip script to be exceeded #8

Closed oyvindrobertsen closed 5 years ago

oyvindrobertsen commented 6 years ago

For multibranch pipelines, the workspace directory paths become something like ${JENKINS_HOME}/workspace/<ITEM_NAME>_<BRANCH>-LI2O2G3RWTKS56SOETS5U2H4JVOMYTL5THTRAD4MFE5TD4JUBFBQ, resulting in a really long path to the python executable when a virtualenv is created inside the workspace dir. This path is used as a shebang in the pip shell script installed in the virtualenv. From what I can gather from https://github.com/pypa/virtualenv/issues/596, the maximum length of a shebang is 127 characters, which will easily be exceeded for many multibranch pipelines (probably even some non-multibranch setups). The result for me was that I fell back on the system wide pip/python installation and got a permission error since jenkins is not allowed to install packages to the system wide site-packages dir. I bypassed this by executing pip via python -m pip .... This isn't really something that is easily fixed, as the shebang length limit is set in the kernel, and one could argue that this isn't really an issue with this plugin specifically, as this will happen in all cases where pip is pointing to a python installation in a really deep path. It is however likely that more people than me will stumble upon this issue, so maybe a note explaining the issue and giving the workaround in the documentation could help?

cstarner commented 6 years ago

I will update the documentation to reflect this issue. I appreciate you bringing it to my attention.

It is possible that I would be able to fix this problem through a different method, as part of the work I'm doing on #6. Since I am planning on capturing the environmental variables of a virtualenv, and applying them within a withPythonEnv block, I don't technically need the virtualenv to be located within the workspace.

However, I am concerned with discarding virtualenvs all over the place. I could provide a cleanup function, but ideally I would be able to hook into something like https://wiki.jenkins.io/display/JENKINS/Workspace+Cleanup+Plugin