search

SAP-archive / cloud-s4-sdk-pipeline

The Cloud SDK pipeline uses the Cloud SDK continuous delivery server for building, checking, and deploying extension applications. Projects based on the SAP Cloud SDK archetype will automatically use this pipeline.
Apache License 2.0
65 stars 25 forks source link

Missing groupId during Nexus artifactDeployment with V32 #32

Closed goeblj closed 4 years ago

goeblj commented 4 years ago

Issue Description

The pipeline is failing in the stage: Artifact Deployment

In version V32 the nexus artifact deployment will not work anymore with this configuration:

  artifactDeployment:
    nexus:
      url: url nexus host
      repository: respository on nexus host
      credentialsId: credentials on jenkins

We tried to add the optional attribute groupId but then an other issue comes up that the artifactId must not be empty. Seems to be an issue to determine the groupId and artifactId from pom.xml

Thanks for your help Jürgen

Log Output

16:22:44  --- Begin library step of: nexusUpload ---
[Pipeline] echo
16:22:44  Unstash content: piper-bin
[Pipeline] unstash
[Pipeline] echo
16:22:45  Unstash content: pipelineConfigAndTests
[Pipeline] unstash
[Pipeline] fileExists
[Pipeline] writeFile
[Pipeline] fileExists
[Pipeline] writeFile
[Pipeline] libraryResource
[Pipeline] writeFile
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
16:22:45  + ./piper getConfig --contextConfig --stepMetadata .pipeline/tmp/metadata/nexusUpload.yaml --defaultConfig .pipeline/default_s4_pipeline_environment.yml
[Pipeline] readJSON
[Pipeline] echo
16:22:45  Config: [containerShell:, dockerEnvVars:[], dockerImage:maven:3.5.0-jdk-8, dockerOptions:, dockerPullImage:false, dockerWorkspace:, nexusCredentialsId:IRE-NEXUS]
[Pipeline] echo
16:22:45  --- Begin library step of: dockerExecute ---
[Pipeline] timeout
16:22:46  Timeout set to expire in 10 sec
[Pipeline] {
[Pipeline] httpRequest
[Pipeline] }
[Pipeline] // timeout
[Pipeline] sh
16:22:46  + docker ps -q
[Pipeline] isUnix
[Pipeline] sh
16:22:46  + docker pull maven:3.5.0-jdk-8
16:22:48  3.5.0-jdk-8: Pulling from library/maven
16:22:48  Digest: sha256:53b8e8f22464a07266ca34f90ac274c66f806a4edf71ebbf88e67e9499e08e05
16:22:48  Status: Image is up to date for maven:3.5.0-jdk-8
16:22:48  docker.io/library/maven:3.5.0-jdk-8
[Pipeline] isUnix
[Pipeline] sh
16:22:48  + docker inspect -f . maven:3.5.0-jdk-8
16:22:48  .
[Pipeline] withDockerContainer
16:22:48  Jenkins seems to be running inside container 495121830be451edd6c8651ca40a2922d2811898717e30f8bf47b249860296ba
16:22:48  $ docker run -t -d -u 1000:999 --env http_proxy --env https_proxy --env no_proxy --env HTTP_PROXY --env HTTPS_PROXY --env NO_PROXY -w /var/jenkins_home/workspace/-capacity-management_master-CBPFWSKFU4QLBEBUXW5SKHB6KYA4256WJHQADEA26T3IAY6NBSVQ@2 --volumes-from 495121830be451edd6c8651ca40a2922d2811898717e30f8bf47b249860296ba -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** maven:3.5.0-jdk-8 cat
16:22:49  $ docker top 3a18ca97d4b5328be79ca888efb51fe11b7ecad44d78238ee41a244ca7f24826 -eo pid,comm
[Pipeline] {
[Pipeline] withCredentials
16:22:49  Masking supported pattern matches of $PIPER_username or $PIPER_password
[Pipeline] {
[Pipeline] sh
16:22:49  + ./piper nexusUpload --defaultConfig .pipeline/default_s4_pipeline_environment.yml
16:22:49  time="2020-04-23T14:22:49Z" level=info msg="Added default config '.pipeline/default_s4_pipeline_environment.yml'" library=SAP/jenkins-library stepName=nexusUpload
16:22:49  time="2020-04-23T14:22:49Z" level=info msg="Skipping npm upload because either no package json was found or NpmRepository option is not provided." library=SAP/jenkins-library stepName=nexusUpload
16:22:49  time="2020-04-23T14:22:49Z" level=info msg="Maven project structure detected" library=SAP/jenkins-library stepName=nexusUpload
16:22:50  time="2020-04-23T14:22:50Z" level=info msg="running command: mvn --file application/pom.xml -Dexpression=project.packaging -DforceStdout -q --batch-mode org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate" library=SAP/jenkins-library stepName=nexusUpload
16:23:00  time="2020-04-23T14:22:59Z" level=info msg="running command: mvn --file application/pom.xml -Dexpression=project.groupId -DforceStdout -q --batch-mode org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate" library=SAP/jenkins-library stepName=nexusUpload
16:23:02  time="2020-04-23T14:23:02Z" level=info msg="running command: mvn --file application/pom.xml -Dexpression=project.artifactId -DforceStdout -q --batch-mode org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate" library=SAP/jenkins-library stepName=nexusUpload
16:23:05  time="2020-04-23T14:23:05Z" level=info msg="running command: mvn --file application/pom.xml -Dexpression=project.version -DforceStdout -q --batch-mode org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate" library=SAP/jenkins-library stepName=nexusUpload
16:23:08  time="2020-04-23T14:23:07Z" level=fatal msg="step execution failed" error="groupID must not be empty" library=SAP/jenkins-library stepName=nexusUpload
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
16:23:08  $ docker stop --time=1 3a18ca97d4b5328be79ca888efb51fe11b7ecad44d78238ee41a244ca7f24826
16:23:09  $ docker rm -f 3a18ca97d4b5328be79ca888efb51fe11b7ecad44d78238ee41a244ca7f24826
[Pipeline] // withDockerContainer
[Pipeline] libraryResource
[Pipeline] echo
16:23:09  ----------------------------------------------------------
16:23:09  --- An error occurred in the library step: dockerExecute
16:23:09  ----------------------------------------------------------
16:23:09  
16:23:09  The following parameters were available to the step:
16:23:09  ***
16:23:09  *** to show step parameters, set verbose:true in general pipeline configuration
16:23:09  *** WARNING: this may reveal sensitive information. ***
16:23:09  ***
16:23:09  
16:23:09  The error was:
16:23:09  ***
16:23:09  hudson.AbortException: script returned exit code 1
16:23:09  ***
16:23:09  
16:23:09  Further information:
16:23:09  * Documentation of library step dockerExecute: https://sap.github.io/jenkins-library/steps/dockerExecute/
16:23:09  * Source code of library step dockerExecute: https://github.com/SAP/jenkins-library/blob/master/vars/dockerExecute.groovy
16:23:09  * Library documentation: https://sap.github.io/jenkins-library/
16:23:09  * Library repository: https://github.com/SAP/jenkins-library/
16:23:09  
16:23:09  ----------------------------------------------------------
16:23:09  --- End library step of: dockerExecute ---
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] libraryResource
[Pipeline] echo
16:23:10  ----------------------------------------------------------
16:23:10  --- An error occurred in the library step: nexusUpload
16:23:10  ----------------------------------------------------------
16:23:10  
16:23:10  The following parameters were available to the step:
16:23:10  ***
16:23:10  *** to show step parameters, set verbose:true in general pipeline configuration
16:23:10  *** WARNING: this may reveal sensitive information. ***
16:23:10  ***
16:23:10  
16:23:10  The error was:
16:23:10  ***
16:23:10  hudson.AbortException: script returned exit code 1
16:23:10  ***
16:23:10  
16:23:10  Further information:
16:23:10  * Documentation of library step nexusUpload: https://sap.github.io/jenkins-library/steps/nexusUpload/
16:23:10  * Source code of library step nexusUpload: https://github.com/SAP/jenkins-library/blob/master/vars/nexusUpload.groovy
16:23:10  * Library documentation: https://sap.github.io/jenkins-library/
16:23:10  * Library repository: https://github.com/SAP/jenkins-library/
16:23:10  
16:23:10  ----------------------------------------------------------
16:23:10  --- End library step of: nexusUpload ---

Search for existing solution beforehand

Project Details

daniel-kurzynski commented 4 years ago

Hi Jürgen,

I could reproduce the issue with locally when using the same maven docker image (maven:3.5.0-jdk-8).

When using a new one (maven:3.6-jdk-8) it worked. I think it is related to the maven version.

Could you please try to configure the maven docker image for the step mavenExecute (or nexusUplaod) and let us know whether the issue still persists?

Thanks.

Daniel

goeblj commented 4 years ago

Hi Daniel,

after using the new docker image groupId and artifactId were correct determined. Now I run into a new error during nexus deployment.

Thanks, Jürgen

13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ------------------------------------------------------------------------" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] Reactor Summary for ire-obligatory-capacity-management - Root 1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4:" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] " library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ire-obligatory-capacity-management - Root .......... FAILURE [  0.263 s]" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ire-obligatory-capacity-management - Application ... SKIPPED" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ire-obligatory-capacity-management - Unit Tests .... SKIPPED" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ire-obligatory-capacity-management - Integration Tests SKIPPED" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ------------------------------------------------------------------------" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] BUILD FAILURE" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ------------------------------------------------------------------------" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] Total time:  1.851 s" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] Finished at: 2020-04-27T11:57:49Z" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[INFO] ------------------------------------------------------------------------" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy-file (default-cli) on project ire-obligatory-capacity-management: Failed to deploy artifacts: Could not transfer artifact group.msg.ire:ire-obligatory-capacity-management-application:pom:1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4 from/to remote-repository (I've changed this): Failed to transfer file I've changed this/group/msg/ire/ire-obligatory-capacity-management-application/1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4/ire-obligatory-capacity-management-application-1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4.pom with status code 401 -> [Help 1]" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] " library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch." library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] Re-run Maven using the -X switch to enable full debug logging." library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] " library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] For more information about the errors and possible solutions, please read the following articles:" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:49Z" level=info msg="[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=fatal msg="step execution failed" error="uploading artifacts for ID 'ire-obligatory-capacity-management-application' failed: failed to run executable, command: '[mvn -Dmaven.repo.local=s4hana_pipeline/maven_local_repo -Durl=http:''I've changed this'' -DgroupId=group.msg.ire -Dversion=1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4 -DartifactId=ire-obligatory-capacity-management-application -Dfile=application/pom.xml -Dpackaging=pom -Dfiles=application/target/ire-obligatory-capacity-management-application.jar -Dclassifiers= -Dtypes=jar --batch-mode org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy-file]', error: running command 'mvn' failed: cmd.Run() failed: exit status 1" library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=info msg=ire-obligatory-capacity-management-application library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=info msg=group.msg.ire library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=info msg=1.0-2020-04-27T115014UTC_c9e7ac2efde27cc238da975d358cace514dee5e4 library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=info msg=ire-obligatory-capacity-management-application library=SAP/jenkins-library stepName=nexusUpload
13:57:50  time="2020-04-27T11:57:50Z" level=info msg=jar library=SAP/jenkins-library stepName=nexusUpload
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
13:57:50  $ docker stop --time=1 06599283deacaefcabaeb0a786ad4d13cb8858fa41e16e01d666c9aebd12dc7f
13:57:51  $ docker rm -f 06599283deacaefcabaeb0a786ad4d13cb8858fa41e16e01d666c9aebd12dc7f
[Pipeline] // withDockerContainer
[Pipeline] libraryResource
[Pipeline] echo
13:57:51  ----------------------------------------------------------
13:57:51  --- An error occurred in the library step: dockerExecute
13:57:51  ----------------------------------------------------------
13:57:51  
13:57:51  The following parameters were available to the step:
13:57:51  ***
13:57:51  *** to show step parameters, set verbose:true in general pipeline configuration
13:57:51  *** WARNING: this may reveal sensitive information. ***
13:57:51  ***
13:57:51  
13:57:51  The error was:
13:57:51  ***
13:57:51  hudson.AbortException: script returned exit code 1
13:57:51  ***
13:57:51  
13:57:51  Further information:
13:57:51  * Documentation of library step dockerExecute: https://sap.github.io/jenkins-library/steps/dockerExecute/
13:57:51  * Source code of library step dockerExecute: https://github.com/SAP/jenkins-library/blob/master/vars/dockerExecute.groovy
13:57:51  * Library documentation: https://sap.github.io/jenkins-library/
13:57:51  * Library repository: https://github.com/SAP/jenkins-library/
13:57:51  
13:57:51  ----------------------------------------------------------
13:57:51  --- End library step of: dockerExecute ---
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] libraryResource
[Pipeline] echo
13:57:51  ----------------------------------------------------------
13:57:51  --- An error occurred in the library step: nexusUpload
13:57:51  ----------------------------------------------------------
13:57:51  
13:57:51  The following parameters were available to the step:
13:57:51  ***
13:57:51  *** to show step parameters, set verbose:true in general pipeline configuration
13:57:51  *** WARNING: this may reveal sensitive information. ***
13:57:51  ***
13:57:51  
13:57:51  The error was:
13:57:51  ***
13:57:51  hudson.AbortException: script returned exit code 1
13:57:51  ***
13:57:51  
13:57:51  Further information:
13:57:51  * Documentation of library step nexusUpload: https://sap.github.io/jenkins-library/steps/nexusUpload/
13:57:51  * Source code of library step nexusUpload: https://github.com/SAP/jenkins-library/blob/master/vars/nexusUpload.groovy
13:57:51  * Library documentation: https://sap.github.io/jenkins-library/
13:57:51  * Library repository: https://github.com/SAP/jenkins-library/
13:57:51  
13:57:51  ----------------------------------------------------------
13:57:51  --- End library step of: nexusUpload ---
daniel-kurzynski commented 4 years ago

Somehow hidden, but I see Could not transfer artifact...with status code 401. Just to be sure, have you checked the credentials?

MarcusSchoelzel commented 4 years ago

Hi Daniel,

yes, we have checked the credentials.

Regards Marcus

goeblj commented 4 years ago

Hi Daniel, with V30 the credentials work. In https://github.com/SAP/jenkins-library/blob/master/vars/nexusUpload.groovy nexusCredentialsId is used. Is there anything to change in our config? Regards Jürgen

daniel-kurzynski commented 4 years ago

nexusCredentialsId is used but there is an alias which will convert what you have convert to this key. This works. See also: Config: [...:, nexusCredentialsId:IRE-NEXUS]

It looks like the credentials are not passed to the nexusUpload step. We will investigate what the reason could be and come back to you.

daniel-kurzynski commented 4 years ago

I think I found the issue. Should be solved with: https://github.com/SAP/jenkins-library/pull/1480

goeblj commented 4 years ago

Hi Daniel,

thanks for solving the issue! Do you need to update the pipeline-lib to use v1.26.0 of piper? With the current version I can see in the logs that v1.24.0 of piper is loaded which doesn't contain the fix.

Best regards Jürgen

fwilhe commented 4 years ago

Hi @goeblj,

I've updated the library to v1.26.0, so in the master branch of the pipeline it should work now.

In general, we don't recommend to change the version of the "piper" library manually as there might be incompatible changes.

Looking forward to your feedback. Please let me know if we can close this issue.

Best, Florian

goeblj commented 4 years ago

Hi @fwilhe, everything works fine again. Thanks for your help and best regards, Jürgen