search

gabrie-allaigre / sonar-gitlab-plugin

Add to each commit GitLab in a global commentary on the new anomalies added by this commit and add comment lines of modified files
GNU Lesser General Public License v3.0
713 stars 207 forks source link

ERROR: Caused by: Duplicate key #224

Open Kovshar-OS opened 5 years ago

Kovshar-OS commented 5 years ago

Hello, community!

When running sonar-scanner on

SonarQube 7.0. sonar-gitlab-plugin-4.0.0.jar GitLab 10.5.4-ee

getting error like

INFO: ANALYSIS SUCCESSFUL INFO: Executing post-job GitLab Commit Issue Publisher INFO: ------------------------------------------------------------------------ INFO: EXECUTION FAILURE INFO: ------------------------------------------------------------------------ INFO: Total time: 16.706s INFO: Final Memory: 61M/454M INFO: ------------------------------------------------------------------------ ERROR: Error during SonarQube Scanner execution ERROR: SonarQube failed to complete the review of this commit: Duplicate key [Line{number=1, content='/home/acd_developer/projects/acd/c/acdutils/src/UriCodec.cpp'}, Line{number=-1, content='++ b/c/nph-acdCgiBadFilesManipulator.cgi/src/utils/UriCodec.cpp'}] ERROR: Caused by: Duplicate key [Line{number=1, content='/home/acd_developer/projects/acd/c/acdutils/src/UriCodec.cpp'}, Line{number=-1, content='++ b/c/nph-acdCgiBadFilesManipulator.cgi/src/utils/UriCodec.cpp'}]

Help please!

Kovshar-OS commented 5 years ago

Some details: File acd/c/nph-acdCgiBadFilesManipulator.cgi/src/utils/UriCodec.cpp contain only string /home/acd_developer/projects/acd/c/acdutils/src/UriCodec.cpp that reffer to real file in same path.

gabrie-allaigre commented 5 years ago

Hi, Run sonar analyse with debug mode and get logs please.

jkroepke commented 5 years ago

Hi!

first of all we getting this error, too. Just on specific git commits not at all. We are running sonarqube via maven. If we are excluding all gitlab related options everything is fine.

We are running sonarqube via maven.

[INFO] Executing post-job 'GitLab Commit Issue Publisher'
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:28 min
[INFO] Finished at: 2019-04-24T16:14:10+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project sb-backend: SonarQube failed to complete the review of this commit: Duplicate key [] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project sb-backend: SonarQube failed to complete the review of this commit: Duplicate key []
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: SonarQube failed to complete the review of this commit: Duplicate key []
    at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute (ScannerBootstrapper.java:67)
    at org.sonarsource.scanner.maven.SonarQubeMojo.execute (SonarQubeMojo.java:104)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.sonar.api.utils.MessageException: SonarQube failed to complete the review of this commit: Duplicate key []
Caused by: java.lang.IllegalStateException: Duplicate key []
    at java.util.stream.Collectors.lambda$throwingMerger$0 (Collectors.java:133)
    at java.util.HashMap.merge (HashMap.java:1254)
    at java.util.stream.Collectors.lambda$toMap$58 (Collectors.java:1320)
    at java.util.stream.ReduceOps$3ReducingSink.accept (ReduceOps.java:169)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining (ArrayList.java:1382)
    at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential (ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect (ReferencePipeline.java:499)
    at com.talanlabs.sonar.plugins.gitlab.GitLabApiV4Wrapper.getPatchPositionsToLineMapping (GitLabApiV4Wrapper.java:162)
    at com.talanlabs.sonar.plugins.gitlab.GitLabApiV4Wrapper.init (GitLabApiV4Wrapper.java:66)
    at com.talanlabs.sonar.plugins.gitlab.CommitFacade.init (CommitFacade.java:87)
    at com.talanlabs.sonar.plugins.gitlab.CommitPublishPostJob.execute (CommitPublishPostJob.java:85)
    at org.sonar.scanner.postjob.PostJobWrapper.execute (PostJobWrapper.java:46)
    at org.sonar.scanner.phases.PostJobsExecutor.execute (PostJobsExecutor.java:51)
    at org.sonar.scanner.phases.PostJobsExecutor.execute (PostJobsExecutor.java:42)
    at org.sonar.scanner.phases.AbstractPhaseExecutor.execute (AbstractPhaseExecutor.java:80)
    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart (ModuleScanContainer.java:164)
    at org.sonar.core.platform.ComponentContainer.startComponents (ComponentContainer.java:136)
    at org.sonar.core.platform.ComponentContainer.execute (ComponentContainer.java:122)
    at org.sonar.scanner.scan.ProjectScanContainer.scan (ProjectScanContainer.java:319)
    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively (ProjectScanContainer.java:314)
    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart (ProjectScanContainer.java:288)
    at org.sonar.core.platform.ComponentContainer.startComponents (ComponentContainer.java:136)
    at org.sonar.core.platform.ComponentContainer.execute (ComponentContainer.java:122)
    at org.sonar.scanner.task.ScanTask.execute (ScanTask.java:48)
    at org.sonar.scanner.task.TaskContainer.doAfterStart (TaskContainer.java:82)
    at org.sonar.core.platform.ComponentContainer.startComponents (ComponentContainer.java:136)
    at org.sonar.core.platform.ComponentContainer.execute (ComponentContainer.java:122)
    at org.sonar.scanner.bootstrap.GlobalContainer.executeTask (GlobalContainer.java:131)
    at org.sonar.batch.bootstrapper.Batch.doExecuteTask (Batch.java:116)
    at org.sonar.batch.bootstrapper.Batch.execute (Batch.java:71)
    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute (BatchIsolatedLauncher.java:46)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke (IsolatedLauncherProxy.java:60)
    at com.sun.proxy.$Proxy44.execute (Unknown Source)
    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute (EmbeddedScanner.java:185)
    at org.sonarsource.scanner.api.EmbeddedScanner.execute (EmbeddedScanner.java:137)
    at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute (ScannerBootstrapper.java:65)
    at org.sonarsource.scanner.maven.SonarQubeMojo.execute (SonarQubeMojo.java:104)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

If I'm looking at the trace I see this line here:

    at com.talanlabs.sonar.plugins.gitlab.GitLabApiV4Wrapper.getPatchPositionsToLineMapping (GitLabApiV4Wrapper.java:162)

https://github.com/gabrie-allaigre/sonar-gitlab-plugin/blob/0535ffedd7c1d4c7c40dd9754911d0f64b6c04e8/src/main/java/com/talanlabs/sonar/plugins/gitlab/GitLabApiV4Wrapper.java#L162

The code tries to build an HashMap depends on git diff provided via gitlab api.

If the commit have 2 diff for the same file the plugin would crash here. This happens for example if you replace a symlink with a real file.

schnapster commented 4 years ago

Seeing the same problem here, replaced a real file with a symlink. Tried excluding the file in the Gradle sonarqube config but no luck so far. This should be easy to fix in the code, once we determine what the correct behaviour is aka which of the duplicates to keep?