Unable to migrate on windows to 2.0.0

[climblinne]

The plugin was updated to 2.0.0. So I tried to change pybat to bat and run it again. It seems, that the pip and python commands are not using the virtualenv.

Here my testscript:

pipeline {
  agent any
  stages {
    stage ('MyTest') {
      steps {
            timeout(1) {
                // build
                withPythonEnv('C:\\tools\\python3\\python3') {
                    // Creates the virtualenv before proceeding
                    bat 'pip list'
                    bat 'pip install vunit_hdl'
                    bat 'pip list'
                    bat 'pip -V'
                    bat 'python --version'
                }
            }
      }
    }
  }
}

It looks like, that python use my python2 installation and pip install to the default python3 site-packages and not to the virtualenv

[climblinne]

Here is the output from Jenkins:

[Pipeline] {
[Pipeline] withPythonEnv
[TestPybat] Running batch script
[TestPybat] Running batch script
[Pipeline] {
[Pipeline] bat
[TestPybat] Running batch script

C:\Jenkins\workspace\TestPybat>pip list 
Package           Version   
----------------- ----------
...
vunit-hdl         3.8.0     
...
[Pipeline] bat
[TestPybat] Running batch script

C:\Jenkins\workspace\TestPybat>pip install vunit_hdl 
Requirement already satisfied: vunit_hdl in c:\tools\python3\lib\site-packages (3.8.0)
Requirement already satisfied: colorama in c:\tools\python3\lib\site-packages (from vunit_hdl) (0.3.9)
[Pipeline] bat
[TestPybat] Running batch script

C:\Jenkins\workspace\TestPybat>pip list 
Package           Version   
----------------- ----------
...
vunit-hdl         3.8.0     
...   
[Pipeline] bat
[TestPybat] Running batch script

C:\Jenkins\workspace\TestPybat>pip -V 
pip 18.0 from c:\tools\python3\lib\site-packages\pip (python 3.7)

[Pipeline] bat
[TestPybat] Running batch script

C:\Jenkins\workspace\TestPybat>python --version 
Python 2.7.10
[Pipeline] }
[Pipeline] // withPythonEnv

I also tested with pyenv-pipeline-2.0.1-beta, same result.

[cstarner]

Can you show me the output from the Jenkins system logs? Also, what version of python is this, and what is the OS version?

[climblinne]

The problem arise on two Jenkins instances. One is on Windows 7 Enterprise / Jenkins 2.121.3 / Python 3.6.5 64bit My test system runs on Windows 10 Enterprise / Jenkins 2.121.3 / Python 3.7.0 64bit

[climblinne]

jenkins.err.log:

Sep 13, 2018 8:12:16 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
Sep 13, 2018 8:13:47 PM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: TestPybat #13 completed: SUCCESS

Is this the system logs you wanted?

[cstarner]

Yes, but there should be considerably more output then that. Try looking at Manage Jenkins -> System Log -> All Jenkins Logs. Scrub as necessary, obviously.

[climblinne]

Nothing more in the logs. Should I see some special environment variable? How should the $path should look like?

[cstarner]

The path variable should have the path to the virtualenv directory prepended to the rest of the path. Can you dump the environmental variables in each block, and show me that?

On Thu, Sep 13, 2018, 11:54 PM Jan notifications@github.com wrote:

Nothing more in the logs. Should I see some special environment variable? How should the $path should look like?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jenkinsci/pyenv-pipeline-plugin/issues/24#issuecomment-421239252, or mute the thread https://github.com/notifications/unsubscribe-auth/AKQ39FrN0DBGDs-GC4eRkQri5r-B2Eyiks5ua0SKgaJpZM4WnZDD .

[climblinne]

Now I got the problem. We install the default python and rename the python.exe to python3.exe. This worked with the old version of PyEnv. On our real Jenkins Slave there is only the python3.exe available, no other python installation.

[climblinne]

When I (named python3.exe) use the 'withPythonEnv('C:\tools\python3\python3')' the path starts as following: Path=C:\Program Files (x86)\Common... - The pyenv is missing!

When I (renamed back to python.exe) use the 'withPythonEnv('C:\tools\python3\python')' the path starts as following: Path=C:\Jenkins\workspace\TestPybat\.pyenv-tools-python3-python\Scripts;C:\Program File....

[climblinne]

When I use the withPythonEnv('C:\tools\python3\python3.exe') it works. With withPythonEnv('C:\tools\python3\python3') it doesn't!

[cstarner]

Thanks for the detective work! I'll see how this managed to regress, and put out another RC.

[climblinne]

Thanks, I will try the RC as soon as you publish it.

[cstarner]

I've been having trouble recreating the your issue on my end. I am using the following script to try and figure out the problem:

    stages {
        stage('Test') {
            steps {

                cleanWs()

                withPythonEnv('C:\\tools\\python3\\python3') {
                    bat "python --version"
                    bat "pip -V"
                }

                withPythonEnv('C:\\tools\\python3\\python3.exe') {
                    bat "python --version"
                    bat "pip -V"
                }
            }
        }
    }

So far, I haven't been able to reproduce the problem you encountered with withPythonEnv('C:\\tools\\python3\\python3'). When I run the build on my end, the output I get is:

[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] withPythonEnv
$ C:\tools\python3\python3.exe -m virtualenv --python=C:\tools\python3\python3.exe "C:\Program Files (x86)\Jenkins\workspace\windows-test\.pyenv-tools-python3-python3"
[windows-test] Running batch script
[windows-test] Running batch script
[Pipeline] {
[Pipeline] bat
[windows-test] Running batch script

(PYENV-~1) C:\Program Files (x86)\Jenkins\workspace\windows-test>
python --version 
Python 3.6.6
[Pipeline] bat
[windows-test] Running batch script

(PYENV-~1) C:\Program Files (x86)\Jenkins\workspace\windows-test>
pip -V 
pip 18.0 from c:\progra~2\jenkins\worksp~1\window~3\pyenv-~1\lib\site-packages\pip (python 3.6)

[Pipeline] }
[Pipeline] // withPythonEnv
[Pipeline] withPythonEnv
$ C:\tools\python3\python3.exe -m virtualenv --python=C:\tools\python3\python3.exe "C:\Program Files (x86)\Jenkins\workspace\windows-test\.pyenv-tools-python3-python3.exe"
[windows-test] Running batch script
[windows-test] Running batch script
[Pipeline] {
[Pipeline] bat
[windows-test] Running batch script

(PYENV-~1.EXE) C:\Program Files (x86)\Jenkins\workspace\windows-test>
python --version 
Python 3.6.6
[Pipeline] bat
[windows-test] Running batch script

(PYENV-~1.EXE) C:\Program Files (x86)\Jenkins\workspace\windows-test>
pip -V 
pip 18.0 from c:\progra~2\jenkins\worksp~1\window~3\pyenv-~1.exe\lib\site-packages\pip (python 3.6)

[Pipeline] }
[Pipeline] // withPythonEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }

Looking at the folder names from the pip -V commands, I can see that they are operating out of the correct folders. Looking back through the commits that led up to v2.0.0, I didn't make any changes to the code that resolves a python executable from a string, on either Windows or Linux.

Is there anyway you could rerun your tests, with a bat set command thrown in the blocks, so I can see exactly what is going on?

[climblinne]

I tested it today with your script and it all worked fine! I think, it must have been some of the old stuff, because I dign_t use cleanWs before. THanks for your support.

[cstarner]

Of course, glad we got it figured out