search

gabrie-allaigre / sonar-gitlab-plugin

Add to each commit GitLab in a global commentary on the new anomalies added by this commit and add comment lines of modified files
GNU Lesser General Public License v3.0
713 stars 207 forks source link

Plugin fails to post comments and update the result to gitlab when integrated via Jenkins pipeline. #243

Open padmiv opened 5 years ago

padmiv commented 5 years ago

Workflow:

Whenever developer opens merge request in gitlab, a jenkins pipeline gets triggered with sonarqube and quality gate integrated to it.

Requirement with the use of this plugin:

  1. Need to get the sonar violation comments on the code-base of the MR.
  2. SonarQube analysis should get commented as part of MR

Setup: Sonarqube ce 7.2.1 Sonar-gitlab-plugin 4.1.0 SNAPSHOT GitLab Community Edition 10.6.4

Sonar scanner script as part of pipeline:

 ${scannerHome}/bin/sonar-scanner -X -Dsonar.login='xxxxxxxxxxxxxxxxxxxx' -Dproject.settings=sonar-project.properties -Dsonar.projectVersion=$tag -Dsonar.gitlab.ignore_certificate=true -Dsonar.branch=${branch} -Dsonar.gitlab.project_id=${project_id} -Dsonar.gitlab.commit_sha=${commit_sha} -Dsonar.gitlab.ref_name=${branch} -Dsonar.gitlab.url='${url}' -Dsonar.gitlab.disable_proxy=true -Dsonar.issuesReport.console.enable=true -Dsonar.gitlab.failure_notification_mode=exit-code -Dsonar.analysis.mode=publish

Comments are only seen in the console logs of Jenkins and issues are posted in logs as well. Nothing is getting updated in gitlab Merge Request when integrated via Jenkins pipeline.

Logs:

12:14:31.463 DEBUG: Post-jobs : GitLab Commit Issue Publisher (wrapped) 12:14:31.463 INFO: Executing post-job GitLab Commit Issue Publisher (wrapped) 12:14:35.590 DEBUG: getPatchPositionsToLineMapping {642c250301977259ece317bcc8e3f5ec28ed5fe5={wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_update.go=[Line{number=-1, content='++ b/wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_update.go'}, Line{number=101, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: status, TenantUuid: tenantUUID}, nil'}, Line{number=161, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: statusVal, TenantUuid: tenantUUID}, nil'}], wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_delete.go=[Line{number=-1, content='++ b/wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_delete.go'}, Line{number=90, content=' return &pbTenant.DeleteTenantResponse{Status: status, TenantUuid: tenantUUID}, nil'}]}} 12:14:44.336 DEBUG: Requesting quality gate status for analysisId AWrUhWGB1qFHo7sfUN2M 12:14:44.357 INFO: Quality gate status: OK 12:14:44.359 INFO: Security Rating on New Code : 1 12:14:44.359 INFO: Reliability Rating on New Code : 1 12:14:44.359 INFO: Maintainability Rating on New Code : 1 12:14:44.360 INFO: Coverage : 81.6 12:14:44.360 INFO: Duplicated Lines (%) : 2.8 12:14:44.699 DEBUG: getRevisionForLine common_conf.go Line{number=61, content=' Verb string json:"Verb"'} 12:14:44.699 DEBUG: getRevisionForLine {tenant_update.go=[Line{number=-1, content='++ tenant_update.go'}, Line{number=101, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: status, TenantUuid: tenantUUID}, nil'}, Line{number=161, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: statusVal, TenantUuid: tenantUUID}, nil'}], tenant_delete.go=[Line{number=-1, content='++ tenant_delete.go'}, Line{number=90, content='return &pbTenant.DeleteTenantResponse{Status: status, TenantUuid: tenantUUID}, nil'}]} 2:14:44.891 DEBUG: getRevisionForLine notfound 12:14:44.891 DEBUG: Revision for issue com.talanlabs.sonar.plugins.gitlab.models.Issue@1fb77d8 null false 12:14:44.902 INFO: ------------------------------------------------------------------------ 12:14:44.902 INFO: EXECUTION FAILURE 12:14:44.902 INFO: ------------------------------------------------------------------------ 12:14:44.903 INFO: Total time: 18.828s 12:14:45.035 INFO: Final Memory: 72M/2091M 12:14:45.035 INFO: ------------------------------------------------------------------------ 12:14:45.035 ERROR: Error during SonarQube Scanner execution Report status=failed, desc=SonarQube reported QualityGate is ok, with 5 ok, 129 issues, with 16 critical (fail) and 112 major and 1 info 12:14:45.036 DEBUG: Execution getVersion 12:14:45.037 DEBUG: Execution stop script returned exit code 1

Needed something like this with sonar-scanner integrated with jenkins pipeline script. image

@gabrie-allaigre Please help in solving this issue.

pedroIsai21 commented 5 years ago

Workflow:

Whenever developer opens merge request in gitlab, a jenkins pipeline gets triggered with sonarqube and quality gate integrated to it.

Requirement with the use of this plugin:

  1. Need to get the sonar violation comments on the code-base of the MR.
  2. SonarQube analysis should get commented as part of MR

Setup: Sonarqube ce 7.2.1 Sonar-gitlab-plugin 4.1.0 SNAPSHOT GitLab Community Edition 10.6.4

Sonar scanner script as part of pipeline:

 ${scannerHome}/bin/sonar-scanner -X -Dsonar.login='xxxxxxxxxxxxxxxxxxxx' -Dproject.settings=sonar-project.properties -Dsonar.projectVersion=$tag -Dsonar.gitlab.ignore_certificate=true -Dsonar.branch=${branch} -Dsonar.gitlab.project_id=${project_id} -Dsonar.gitlab.commit_sha=${commit_sha} -Dsonar.gitlab.ref_name=${branch} -Dsonar.gitlab.url='${url}' -Dsonar.gitlab.disable_proxy=true -Dsonar.issuesReport.console.enable=true -Dsonar.gitlab.failure_notification_mode=exit-code -Dsonar.analysis.mode=publish

Comments are only seen in the console logs of Jenkins and issues are posted in logs as well. Nothing is getting updated in gitlab Merge Request when integrated via Jenkins pipeline.

Logs:

12:14:31.463 DEBUG: Post-jobs : GitLab Commit Issue Publisher (wrapped) 12:14:31.463 INFO: Executing post-job GitLab Commit Issue Publisher (wrapped) 12:14:35.590 DEBUG: getPatchPositionsToLineMapping {642c250301977259ece317bcc8e3f5ec28ed5fe5={wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_update.go=[Line{number=-1, content='++ b/wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_update.go'}, Line{number=101, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: status, TenantUuid: tenantUUID}, nil'}, Line{number=161, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: statusVal, TenantUuid: tenantUUID}, nil'}], wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_delete.go=[Line{number=-1, content='++ b/wifi_management/src/wifi_mgmt/features/tenant/tenant_rpc/tenant_delete.go'}, Line{number=90, content=' return &pbTenant.DeleteTenantResponse{Status: status, TenantUuid: tenantUUID}, nil'}]}} 12:14:44.336 DEBUG: Requesting quality gate status for analysisId AWrUhWGB1qFHo7sfUN2M 12:14:44.357 INFO: Quality gate status: OK 12:14:44.359 INFO: Security Rating on New Code : 1 12:14:44.359 INFO: Reliability Rating on New Code : 1 12:14:44.359 INFO: Maintainability Rating on New Code : 1 12:14:44.360 INFO: Coverage : 81.6 12:14:44.360 INFO: Duplicated Lines (%) : 2.8 12:14:44.699 DEBUG: getRevisionForLine common_conf.go Line{number=61, content=' Verb string json:"Verb"'} 12:14:44.699 DEBUG: getRevisionForLine {tenant_update.go=[Line{number=-1, content='++ tenant_update.go'}, Line{number=101, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: status, TenantUuid: tenantUUID}, nil'}, Line{number=161, content=' return &pbtenant.UpdateTenantDetailsResponse{Status: statusVal, TenantUuid: tenantUUID}, nil'}], tenant_delete.go=[Line{number=-1, content='++ tenant_delete.go'}, Line{number=90, content='return &pbTenant.DeleteTenantResponse{Status: status, TenantUuid: tenantUUID}, nil'}]} 2:14:44.891 DEBUG: getRevisionForLine notfound 12:14:44.891 DEBUG: Revision for issue com.talanlabs.sonar.plugins.gitlab.models.Issue@1fb77d8 null false 12:14:44.902 INFO: ------------------------------------------------------------------------ 12:14:44.902 INFO: EXECUTION FAILURE 12:14:44.902 INFO: ------------------------------------------------------------------------ 12:14:44.903 INFO: Total time: 18.828s 12:14:45.035 INFO: Final Memory: 72M/2091M 12:14:45.035 INFO: ------------------------------------------------------------------------ 12:14:45.035 ERROR: Error during SonarQube Scanner execution Report status=failed, desc=SonarQube reported QualityGate is ok, with 5 ok, 129 issues, with 16 critical (fail) and 112 major and 1 info 12:14:45.036 DEBUG: Execution getVersion 12:14:45.037 DEBUG: Execution stop script returned exit code 1

Needed something like this with sonar-scanner integrated with jenkins pipeline script. image

@gabrie-allaigre Please help in solving this issue.

Have you fixed the problem?

heidricha commented 5 years ago

Similar case here. Only for javascript project. When all other projects finish, the javascipt scanner starts working on the dist/*.js files, and produces a huge load of errors, fails at the end. Even if its outside the sonar.sources directory.

11:08:01.283 DEBUG: getPatchPositionsToLineMapping {ace32d214ed9cc64f5112e7be3a092d6b4f4bc45={.gitlab-ci.yml=[Line{number=73, content='    sonar.exclusions=dist/**/*'}]}}
11:08:01.480 INFO: Waiting quality gate to complete...
11:08:02.494 INFO: Waiting quality gate to complete...
11:08:03.510 INFO: Waiting quality gate to complete...
11:08:04.532 DEBUG: Requesting quality gate status for analysisId AWu8q3TYo7Hkt6fyyP66
11:08:04.549 INFO: Quality gate status: OK
11:08:04.551 INFO: Open Issues : 0
11:08:04.551 INFO: Reopened Issues : 0
11:08:11.333 DEBUG: getRevisionForLine dist/dc/main.ff5551ab13eccf83691d.js Line{number=110831, content='null'}
11:08:11.334 DEBUG: getRevisionForLine {.gitlab-ci.yml=[Line{number=73, content='    sonar.exclusions=dist/**/*'}]}
11:08:11.334 DEBUG: getRevisionForLine notfound
11:08:11.334 DEBUG: Revision for issue com.talanlabs.sonar.plugins.gitlab.models.Issue@50628080 null false
11:08:11.335 DEBUG: file /src/dist/dc/main.ff5551ab13eccf83691d.js 110831
11:08:11.365 DEBUG: getRevisionForLine dist/dc/main.ff5551ab13eccf83691d.js Line{number=111100, content='null'}
11:08:11.365 DEBUG: getRevisionForLine {.gitlab-ci.yml=[Line{number=73, content='    sonar.exclusions=dist/**/*'}]}
11:08:11.365 DEBUG: getRevisionForLine notfound
11:08:11.365 DEBUG: Revision for issue com.talanlabs.sonar.plugins.gitlab.models.Issue@23cb8fcb null false
11:08:11.365 DEBUG: file /src/dist/dc/main.ff5551ab13eccf83691d.js 111100
11:08:11.366 DEBUG: getRevisionForLine dist/dc/main.ff5551ab13eccf83691d.js Line{number=129924, content='null'}
11:08:11.366 DEBUG: getRevisionForLine {.gitlab-ci.yml=[Line{number=73, content='    sonar.exclusions=dist/**/*'}]}
11:08:11.366 DEBUG: getRevisionForLine notfound

Looking at it again... why on earth do sonar "analyzing" my gitlab-ci-yml? scanner runs in a container, not even supported to see that file.

TimeDelta commented 5 years ago

I have a very similar situation but only when on master branch (not using the branch plugin for sonar, just specifying master as ref_name) and specifying the commit sha

./gradlew sonarqube --full-stacktrace -Dsonar.gitlab.project_id=developers/user-management -Dsonar.gitlab.commit_sha=6b0c88bb8ef8fa44ffddb8b44479aa13c06b06c0 -Dsonar.gitlab.ref_name=master -Dsonar.gitlab.json_mode=CODECLIMATE

Produces:

> Task :sonarqube FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sonarqube'.
> Report status=failed, desc=SonarQube reported QualityGate is ok, with 1 ok, 21 issues, with 2 critical (fail) and 14 major and 1 minor and 4 info

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':sonarqube'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.accept(ExecuteActionsTaskExecuter.java:151)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.accept(ExecuteActionsTaskExecuter.java:148)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:191)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:141)
        at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:108)
        at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
        at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:73)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:49)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:49)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
        at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: Report status=failed, desc=SonarQube reported QualityGate is ok, with 1 ok, 21 issues, with 2 critical (fail) and 14 major and 1 minor and 4 info

Removing the commit_sha fixes the issue but doesn't produce the codeclimate.json file. Specifying a different ref_name (whether it exists or not) fixes the issue and produces the codeclimate.json file as well but then it won't work for the master branch.

I just noticed that specifying a different profile that doesn't show any issues makes it pass as well so it seems to be enforcing that the quality profile has no issues on master but I don't want that behavior because the things it is pointing out as issues aren't actual issues (for example, it says possible SQL Injection but I have explicitly guarded against that and it is impossible - the analyzer just isn't smart enough to understand that my function only allows a string from a static, hard-coded whitelist of column names to be added into the SQL query and only after the SELECT).

TimeDelta commented 5 years ago

@padmiv does your issue happen on every branch or just master like mine?

angelusGJ commented 4 years ago

I have found the problem. The sonar-gitlab-plugin-4.1.0-SNAPSHOT.jar artifact is badly generated. I don't know how the artifact was generated but I have compiled the artifact from the source code from 4.1.0-SNAPSHOT tag and the plugin works.

The artifact should be generated again to fix de problem.

I have created the issue to fix this #264