ckristo
commented
6 days ago I switched to use venv (which was a challenge, but I think it works now fine) and got a working version. However, I encountered that I can also break the venv version by making slight changes in the Setup stage.
Here the working version:
#!/usr/bin/env groovy
pipeline {
agent { label 'python311-preview' }
environment {
ARTIFACTORY_REPO_NAME = 'pypi-repo'
PYPI_REPO_URL = "https://jfrog-platform.internal/artifactory/api/pypi/${ARTIFACTORY_REPO_NAME}"
VENV_ENV_DIR = '.venv'
VIRTUAL_ENV = "${env.WORKSPACE}/${VENV_ENV_DIR}"
PATH = "${VIRTUAL_ENV}/bin:/opt/jenkins/.local/bin:${PATH}"
SOLUTION_CODE = 'app-id'
PROJECT_NAME = 'module_name'
PROJECT_KEY = "${SOLUTION_CODE}-${PROJECT_NAME}"
JFROG_CLI_BUILD_NAME = "${PROJECT_KEY}-${BRANCH_NAME}"
JFROG_CLI_BUILD_NUMBER = "${BUILD_NUMBER}"
JFROG_CLI_LOG_LEVEL = 'DEBUG'
}
stages {
stage('Setup') {
steps {
sh '''
python -m venv $VENV_ENV_DIR
source $VENV_ENV_DIR/bin/activate
pip config set global.index-url "${PYPI_REPO_URL}/simple"
pip install --upgrade pip
pip install build twine setuptools setuptools-scm
'''
script {
def stdout = sh script: 'python -m setuptools_scm', returnStdout: true
env.VERSION = stdout.trim()
}
jf 'c show'
jf "pipc --global --repo-resolve=${ARTIFACTORY_REPO_NAME}"
jf "pip install ."
}
}
stage('Test') {
steps {
sh "pip install '.[test]'"
sh 'pytest'
tar dir: 'htmlcov', compress: true, file: "${env.PROJECT_NAME}-${env.VERSION}-htmlcov.tgz", archive: true, overwrite: true
}
}
stage('Build package') {
steps {
sh 'python -m build --verbose'
archiveArtifacts artifacts: 'dist/*', fingerprint: true
}
}
// stage('SAST scan') {
stage('SCA scan') {
steps {
jf 'rt bag'
jf 'rt bce'
jf 'rt bp'
jf 'bs'
script {
try {
jf 'aud --pip --watches DA-All-build-watch'
} catch (err) {
unstable(message: "JFrog CLI SCA via `audit` command indicated build failure: ${err}")
// NOTE: do not fail on `jf audit` because this is additional info for devs only
// fail should be indicated by `jf bs`
}
}
}
}
stage('Deploy package') {
when {
anyOf {
allOf {
branch 'master'
expression {
def tag = sh script: 'git tag --points-at ${GIT_COMMIT}', returnStdout: true
return tag.startsWith('v')
}
}
tag 'v*'
}
}
steps {
jf 'twine upload "dist/*"'
/*
script {
def stdout = sh script: 'python -m twine upload --verbose --skip-existing --disable-progress-bar --non-interactive --repository-url $PYPI_REPO_URL dist/* -u$XRAY_USERNAME -p$XRAY_PASSWORD', returnStdout: true
stdout = stdout.trim()
println stdout
if (stdout.contains('WARNING')) {
unstable(message: "${STAGE_NAME} issued warning(s)")
}
}
*/
}
}
}
post {
always {
sh 'rm -rf build dist *.egg-info .pytest_cache .coverage htmlcov coverage.xml .scannerwork .jfrog $VENV_ENV_DIR'
}
}
}If I change the Setup stage as follows, I get the same issue again:
@@ -27,15 +27,15 @@
source $VENV_ENV_DIR/bin/activate
pip config set global.index-url "${PYPI_REPO_URL}/simple"
pip install --upgrade pip
- pip install build twine setuptools setuptools_scm
'''
+ jf 'c show'
+ jf "pipc --global --repo-resolve=${ARTIFACTORY_REPO_NAME}"
+ jf 'pip install build twine setuptools setuptools-scm'
+ jf "pip install ."
script {
def stdout = sh script: 'python -m setuptools_scm', returnStdout: true
env.VERSION = stdout.trim()
}
- jf 'c show'
- jf "pipc --global --repo-resolve=${ARTIFACTORY_REPO_NAME}"
- jf "pip install ."
}
}It looks like there is some issue when packages are already installed or cached somehow...
Describe the bug
Hey,
I'm trying to publish build infos for my python3.11 project to Artifactory, but get an error at the
jf rt bpstep:Current behavior
Enabling JFrog CLI debug mode and setting the dry-run option gives me the following build job output:
Looking at the
modules.dependenciesJSON structure, the first n elements indeed miss anidfield. What I do not understand yet is the JsonEOFException shown at the end of the dry run output.Reproduction steps
2.63.1and2.59.1- same issue on bothjf c showshows:Expected behavior
JFrog CLI should only generate valid build infos which can be published to Artifactory (in my case: all dependencies should have an
idfield).JFrog CLI version
2.59.1
Operating system type and version
Linux
JFrog Artifactory version
7.71.23
JFrog Xray version
No response