SAP-archive / cloud-s4-sdk-pipeline

The Cloud SDK pipeline uses the Cloud SDK continuous delivery server for building, checking, and deploying extension applications. Projects based on the SAP Cloud SDK archetype will automatically use this pipeline.
Apache License 2.0
65 stars 25 forks source link

OpenSAP Code doesn't pass Code check #41

Closed Oschm closed 4 years ago

Oschm commented 4 years ago

Issue Description (Mandatory)

I'm following the setup in OpenSAP Course Week 2 Create and Deliver Cloud-Native SAP S/4HANA Extensions

The pipeline is failing in the stage: Static Code Checks. Error Message: [2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] Failed to execute goal org.apache.maven.plugins:maven-pmd-plugin:3.13.0:check (default-cli) on project address-manager-application: You have 6 PMD violations. For more details see: /var/jenkins_home/workspace/Address-Manager_master@5/application/target/pmd.xml -> [Help 1]

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR]

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] Re-run Maven using the -X switch to enable full debug logging.

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR]

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] For more information about the errors and possible solutions, please read the following articles:

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR]

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] After correcting the problems, you can resume the build with the command

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] mvn -rf :address-manager-application

PLEASE PLACE YOUR DESCRIPTION HERE I'm using the SAP provided code for the course. The code I'm using can be found here: https://github.com/Oschm/s-4-cloud-hello-world

If I build it with the S/4 Build Pipeline it fails the Step for static code checks. The error message doesn't help me to find the error. What can I do that makes it work?

Log Output (Mandatory)


[2020-07-06T13:04:45.803Z] + ./piper mavenExecuteStaticCodeChecks --defaultConfig .pipeline/default_s4_pipeline_environment.yml --ignoreCustomDefaults

[2020-07-06T13:04:46.054Z] info  mavenExecuteStaticCodeChecks - Project config: '.pipeline/config.yml'

[2020-07-06T13:04:46.054Z] info  mavenExecuteStaticCodeChecks - Project defaults: '.pipeline/default_s4_pipeline_environment.yml'

[2020-07-06T13:04:46.054Z] info  mavenExecuteStaticCodeChecks - Ignoring custom defaults from pipeline config

[2020-07-06T13:04:46.054Z] info  mavenExecuteStaticCodeChecks - running command: mvn --global-settings .pipeline/global_settings.xml -Dmaven.repo.local=s4hana_pipeline/maven_local_repo -pl !unit-tests -pl !integration-tests -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn --batch-mode com.github.spotbugs:spotbugs-maven-plugin:3.1.12:check org.apache.maven.plugins:maven-pmd-plugin:3.13.0:check

[2020-07-06T13:04:52.663Z] info  mavenExecuteStaticCodeChecks - [INFO] Scanning for projects...

[2020-07-06T13:04:53.215Z] info  mavenExecuteStaticCodeChecks - [INFO] ------------------------------------------------------------------------

[2020-07-06T13:04:53.215Z] info  mavenExecuteStaticCodeChecks - [INFO] Reactor Build Order:

[2020-07-06T13:04:53.215Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:04:53.215Z] info  mavenExecuteStaticCodeChecks - [INFO] address-manager - Root                                             [pom]

[2020-07-06T13:04:53.215Z] info  mavenExecuteStaticCodeChecks - [INFO] address-manager - Application                                      [war]

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] -----------< com.sap.cloud.s4hana.examples:address-manager >------------

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] Building address-manager - Root 1.0-SNAPSHOT-20200706130146_b972175bbc718dd2881f9acbcd94f36c7080d192 [1/2]

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] --------------------------------[ pom ]---------------------------------

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] >>> spotbugs-maven-plugin:3.1.12:check (default-cli) > :spotbugs @ address-manager >>>

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:04:56.445Z] info  mavenExecuteStaticCodeChecks - [INFO] --- spotbugs-maven-plugin:3.1.12:spotbugs (spotbugs) @ address-manager ---

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] <<< spotbugs-maven-plugin:3.1.12:check (default-cli) < :spotbugs @ address-manager <<<

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] --- spotbugs-maven-plugin:3.1.12:check (default-cli) @ address-manager ---

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] >>> maven-pmd-plugin:3.13.0:check (default-cli) > :pmd @ address-manager >>>

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:14.406Z] info  mavenExecuteStaticCodeChecks - [INFO] --- maven-pmd-plugin:3.13.0:pmd (pmd) @ address-manager ---

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] <<< maven-pmd-plugin:3.13.0:check (default-cli) < :pmd @ address-manager <<<

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] --- maven-pmd-plugin:3.13.0:check (default-cli) @ address-manager ---

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] PMD version: 6.21.0

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] -----< com.sap.cloud.s4hana.examples:address-manager-application >------

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] Building address-manager - Application 1.0-SNAPSHOT-20200706130146_b972175bbc718dd2881f9acbcd94f36c7080d192 [2/2]

[2020-07-06T13:05:15.761Z] info  mavenExecuteStaticCodeChecks - [INFO] --------------------------------[ war ]---------------------------------

[2020-07-06T13:05:16.011Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:16.011Z] info  mavenExecuteStaticCodeChecks - [INFO] >>> spotbugs-maven-plugin:3.1.12:check (default-cli) > :spotbugs @ address-manager-application >>>

[2020-07-06T13:05:17.355Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:17.355Z] info  mavenExecuteStaticCodeChecks - [INFO] --- spotbugs-maven-plugin:3.1.12:spotbugs (spotbugs) @ address-manager-application ---

[2020-07-06T13:05:18.699Z] info  mavenExecuteStaticCodeChecks - [INFO] Fork Value is true

[2020-07-06T13:05:40.502Z] info  mavenExecuteStaticCodeChecks - [INFO] Done SpotBugs Analysis....

[2020-07-06T13:05:41.053Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:41.053Z] info  mavenExecuteStaticCodeChecks - [INFO] <<< spotbugs-maven-plugin:3.1.12:check (default-cli) < :spotbugs @ address-manager-application <<<

[2020-07-06T13:05:41.053Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:41.053Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:41.053Z] info  mavenExecuteStaticCodeChecks - [INFO] --- spotbugs-maven-plugin:3.1.12:check (default-cli) @ address-manager-application ---

[2020-07-06T13:05:41.604Z] info  mavenExecuteStaticCodeChecks - [INFO] BugInstance size is 0

[2020-07-06T13:05:41.604Z] info  mavenExecuteStaticCodeChecks - [INFO] Error size is 0

[2020-07-06T13:05:41.855Z] info  mavenExecuteStaticCodeChecks - [INFO] No errors/warnings found

[2020-07-06T13:05:41.855Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:41.855Z] info  mavenExecuteStaticCodeChecks - [INFO] >>> maven-pmd-plugin:3.13.0:check (default-cli) > :pmd @ address-manager-application >>>

[2020-07-06T13:05:41.855Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:41.855Z] info  mavenExecuteStaticCodeChecks - [INFO] --- maven-pmd-plugin:3.13.0:pmd (pmd) @ address-manager-application ---

[2020-07-06T13:05:42.106Z] warn  mavenExecuteStaticCodeChecks - [WARNING] Unable to locate Source XRef to link to - DISABLED

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] <<< maven-pmd-plugin:3.13.0:check (default-cli) < :pmd @ address-manager-application <<<

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] --- maven-pmd-plugin:3.13.0:check (default-cli) @ address-manager-application ---

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] PMD version: 6.21.0

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] ------------------------------------------------------------------------

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] Reactor Summary for address-manager - Root 1.0-SNAPSHOT-20200706130146_b972175bbc718dd2881f9acbcd94f36c7080d192:

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] 

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] address-manager - Root ............................. SUCCESS [ 19.857 s]

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] address-manager - Application ...................... FAILURE [ 35.900 s]

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] ------------------------------------------------------------------------

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] BUILD FAILURE

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] ------------------------------------------------------------------------

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] Total time:  59.724 s

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] Finished at: 2020-07-06T13:05:51Z

[2020-07-06T13:05:51.982Z] info  mavenExecuteStaticCodeChecks - [INFO] ------------------------------------------------------------------------

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] Failed to execute goal org.apache.maven.plugins:maven-pmd-plugin:3.13.0:check (default-cli) on project address-manager-application: You have 6 PMD violations. For more details see: /var/jenkins_home/workspace/Address-Manager_master@5/application/target/pmd.xml -> [Help 1]

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] 

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] Re-run Maven using the -X switch to enable full debug logging.

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] 

[2020-07-06T13:05:51.982Z] error mavenExecuteStaticCodeChecks - [ERROR] For more information about the errors and possible solutions, please read the following articles:

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] 

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR] After correcting the problems, you can resume the build with the command

[2020-07-06T13:05:51.983Z] error mavenExecuteStaticCodeChecks - [ERROR]   mvn <args> -rf :address-manager-application

[2020-07-06T13:05:51.983Z] fatal mavenExecuteStaticCodeChecks - step execution failed - failed to run executable, command: '[mvn --global-settings .pipeline/global_settings.xml -Dmaven.repo.local=s4hana_pipeline/maven_local_repo -pl !unit-tests -pl !integration-tests -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn --batch-mode com.github.spotbugs:spotbugs-maven-plugin:3.1.12:check org.apache.maven.plugins:maven-pmd-plugin:3.13.0:check]', error: running command 'mvn' failed: cmd.Run() failed: exit status 1

Link to Jenkins Job (if it can be shared)

PLEASE PLACE THE LINK HERE

Reproduce the Issue Locally (Mandatory)

In case a maven/npm command or similar failed:

Search for existing solution beforehand (Mandatory)

Please try the GitHub search, it works really well

Project Details (Mandatory)

fwilhe commented 4 years ago

Hi, thanks for reporting.

I've got some questions:

Link to GitHub repo: https://github.com/Oschm/s-4-cloud-hello-world

I get a 404. Is the repo public? Have you deleted it? What branch of which example project is this based on?

Oschm commented 4 years ago

Hello fwilhe, Thank you for the quick response. Sorry my repo was indeed private. I set it to public. The Code is based on the application in this repo https://github.com/SAP/cloud-s4-sdk-book/. My code basically is similar to branch course/2_4_multitenancy. Unfortantely the codebase still contains a pipeline_config.yml file which is now deprecated. Thats why I changed it in my code to a .pipeline/config.yml. I also changed the step

stages:
  staticCodeChecks:
    findbugsExcludesFile: '' #Path to findbugs exclude file
    pmdExcludes: '' # List or ANT style patterns

to

#Stage Specific Configurations
stages:
  mavenExecuteStaticCodeChecks:
    spotBugsExcludeFilterFile: '' #Path to findbugs exclude file

Since the code in the provided version didn't work with the new jenkins version. In my opinion this could cause a problem with the static code checks but i can't see how since i basically just updated the syntax.

greetings, Oliver

fwilhe commented 4 years ago

Hi Oliver,

as you found out not all branches of that project are well maintained. Lots of things have changed in the meantime.

For the bare essential config, you can get started with this. You can move it to the new location, but the old one will still work for now. We might change this at a later point in time.

For the pmd issue, I think it fails because of one rule that forbids shadowing the name of javax.servlet.http.HttpServlet, so renaming com.sap.cloud.s4hana.examples.addressmgr.util.HttpServlet to something different should fix that. Seems like this changed in the meantime.

I hope this helps.

Oschm commented 4 years ago

Hi fwilhe,

As you can see in my code I already changed the class name to MyHttpServlet, since I ran into the same shadowclass issue before opening the thread. I changed the file and all occurrences in three more files. The Project then builds locally without a problem.

I used the new config file you suggested. As you can see I still get the static code check error. image

The exact error message shows in the beginning that he uses the .pipeline/config.yml file. Am i correct in assuming that the other file is provided by sap during the build or inside the docker? image

Anyways the whole log doesn't provide any conceivable error till that occurs: image

Unfortunately I can't access the additional infos under /var/jenkins_home/workspace/Address-Manager_master@5/application/target/pmd.xml as this seems to be a location inside the docker container.

So even with the new config file I'm still stuck with the setup of the pipeline and i just can assume there is an issue with the s/4 configuration environment?

best regards, Oliver

fwilhe commented 4 years ago

Am i correct in assuming that the other file is provided by sap during the build or inside the docker?

Yes, the default_s4_pipeline_environment.yml file is part of the SAP Cloud SDK Pipeline. As the name implies, it provides default values that you can override in your .pipeline/config.yml.

Unfortunately I can't access the additional infos under /var/jenkins_home/workspace/Address-Manager_master@5/application/target/pmd.xml as this seems to be a location inside the docker container.

In jenkins you'll see what the issues are. In doubt, use the "old" Jenkins user interface and inspect your build, you will get a nice visual representation of what failed your build there, cf:

image

fwilhe commented 4 years ago

As a general remark, I would suggest starting with a new project created from a recent archetype as described here. I can't tell you the exact difference, but the example project might be outdated in multiple areas.

fwilhe commented 4 years ago

In addition to Jenkins, you could also run this command locally mvn -pl \!unit-tests -pl \!integration-tests -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn --batch-mode com.github.spotbugs:spotbugs-maven-plugin:3.1.12:check org.apache.maven.plugins:maven-pmd-plugin:3.13.0:check. This is (almost exactly) what the pipeline is doing (you can find it also in the log).

image

This will create xml file of pmd locally

image

I hope this helps.

Florian

Oschm commented 4 years ago

Hi Florian,

thanks for the answers. Especially the part where to find the PMD errors helped me a lot. Most of these errors were unused logging instantiations. I fixed all of them and now I pass the PMD step.

Now I'm stuck at another step. This time the code coverage threshold is set too low. image That makes sense since I am just setting up Jenkins and haven't written any tests yet.

But if I understand correctly I'm only in the setup step and this shouldn't happen. At least that's what happens in the OpenSAP Course. I checked the documentation and found the threshold can be influenced in the pom build file. See minimum property here https://www.jacoco.org/jacoco/trunk/doc/examples/build/pom.xml

As far as I understand this config is loaded by a groovy script in the original Jenkins file of my project and I have no influence on this, right? Is there any way to overwrite the code coverage threshold for the first build?

Best Regards, Oliver

fwilhe commented 4 years ago

Hi Oliver,

this is part of the Quality Checks stage, and is independent of your maven config. The idea is to enforce high quality standards, this is why the pipeline even for a new project fails with too low coverage.

If you need to disable specific quality checks, we have an option for that, here are the docs on it. I recommend to only do this if you need to, but for the start it might be fine.

Example config

stages:
  s4SdkQualityChecks:
    disabledChecks:
      - checkBackendCodeCoverage

This will disable the check for backend code coverage. Remember to re-active the check once you're up to speed with test coverage.

Best regards

Florian