jfrog / jenkins-artifactory-plugin

Jenkins artifactory plugin
http://jenkins-ci.org/
115 stars 186 forks source link

rtPipInstall fails when envActivation provided #915

Open bewczardski opened 5 months ago

bewczardski commented 5 months ago

Describe the bug When using rtPipInstall to install dependencies in a virtual environment an error occurs:

Note the escaping backslashes in the /bin/sh command below "/bin/sh -c source\ venv-unittest/bin/activate\ &&"

Executing sh script inside container python of pod bbouchard-test-jobs-bou-python-pipeline-13-w9h29-kf1vp-wqckw
Executing command: "java" "-Djava.io.tmpdir=/home/jenkins/agent/workspace/bbouchard_test_jobs/bou-python-pipeline_tmp/artifactory/javatmpdir" "-cp" "/home/jenkins/agent/workspace/bbouchard_test_jobs/bou-python-pipeline_tmp/artifactory/cache/artifactory-plugin/4.0.3/*" "org.jfrog.build.extractor.pip.extractor.PipInstall" 
exit
Jan 19, 2024 2:37:14 PM org.jfrog.build.extractor.packageManager.PackageManagerLogger info
INFO: Executing command: /bin/sh -c source\ venv-unittest/bin/activate\ &&\ pip install --editable . -i https://<****>.jfrog.io/artifactory/api/pypi/pypi-dev/simple
Jan 19, 2024 2:37:14 PM org.jfrog.build.extractor.packageManager.PackageManagerLogger error
SEVERE: pip install failed: /bin/sh: 1: source venv-unittest/bin/activate : not found

java.io.IOException: pip install failed: /bin/sh: 1: source venv-unittest/bin/activate : not found

    at org.jfrog.build.extractor.pip.PipDriver.install(PipDriver.java:36)
    at org.jfrog.build.extractor.pip.extractor.PipInstall.execute(PipInstall.java:98)
    at org.jfrog.build.extractor.packageManager.PackageManagerExtractor.executeAndSaveBuildInfo(PackageManagerExtractor.java:33)
    at org.jfrog.build.extractor.pip.extractor.PipInstall.main(PipInstall.java:85)
Caused by: java.io.IOException: /bin/sh: 1: source venv-unittest/bin/activate : not found

    at org.jfrog.build.extractor.pip.PipDriver.runCommand(PipDriver.java:53)
    at org.jfrog.build.extractor.pip.PipDriver.install(PipDriver.java:34)
    ... 3 more

java.io.IOException: /bin/sh: 1: source venv-unittest/bin/activate : not found

To Reproduce call rtPipInstall with args and envActivation defined


    agent {
        kubernetes {
            inheritFrom 'k8s-agent'
            yamlMergeStrategy merge()
            defaultContainer 'python'
            yaml """\
                spec:
                  containers:
                    - name: python
                      image: python:3.12-slim-bookworm
                      imagePullPolicy: Always
                      command:
                        - cat
                      tty: true
                      resources:
                        requests:
                          cpu: .5
                          memory: 512Mi
                        limits:
                          memory: 512Mi
                """.stripIndent()
        }
    }

    stages {
        stage ('Clone') {
            steps {
                git <Repo Checkout>
                sh '''apt-get update && apt-get install default-jre-headless -y --no-install-recommends'''
                rtPipResolver (
                    id: "PYPI-DEV",
                    serverId: "jfrog",
                    repo: 'pypi-dev'
                )
                sh 'python -m venv venv-unittest'
                rtPipInstall (
                    resolverId: "PYPI-DEV",
                    args: "--editable .",
                    envActivation: "source venv-unittest/bin/activate"
                )
            }
        }
    }
}```

**Expected behavior**
pip install should complete successfully with all dependencies of the package installed in the specified virtual environment

**Screenshots**
N/A, console output was provided

**Versions**
- Jenkins Artifactory plugin version: 4.0.3
- Jenkins operating system: 2.414.3
- Artifactory Version:

**Additional context**
The rtPipInstall command works if `envActivation` is not provided. From the output it looks like the plugin is not preserving the spaces between the arguments. The plugin is escaping all spaces to make 1 single long argument string. 

This looks like a regression of this issue: https://github.com/jfrog/jenkins-artifactory-plugin/issues/415 which was resolved with https://github.com/jfrog/build-info/pull/452

`/bin/sh -c source\ venv-unittest/bin/activate\ &&\ pip install --editable . -i https://<****>.jfrog.io/artifactory/api/pypi/pypi-dev/simple`

should be 
`/bin/sh -c "source venv-unittest/bin/activate && pip install --editable . -i https://<****>.jfrog.io/artifactory/api/pypi/pypi-dev/simple`
eyalbe4 commented 5 months ago

@bewczardski, We recommend using the JFrog Jenkins Plugin to avoid this issue.

bewczardski commented 5 months ago

@eyalbe4 We have a Conan build process that isn't supported by the CLI Plugin, so the build is currently using the artifactory plugin and want to leverage the functionality for installing python libraries used in the build process. Containerization isn't an option as its a windows build and the build process cannot utilize windows containers. For linux builds the python module could be installed in the build container.

I just looked at the build-info plugin an I found this submission: https://github.com/jfrog/build-info/issues/719 which is the exact same issue. Same statement has been made there as well.