search

mc1arke / sonarqube-community-branch-plugin

A plugin that allows branch analysis and pull request decoration in the Community version of Sonarqube
GNU Lesser General Public License v3.0
2.22k stars 521 forks source link

Azure Devops not available as Pull requester provider #228

Closed Malone5923 closed 4 years ago

Malone5923 commented 4 years ago

I have SonarQube 8.0.0.29455 installed and branch community plugin installed 1.3.0 and even Pull Request Decorator Plugin 0.0.1. I cant find azure devops in the list of pull request providers on SonarQube Configuration Dashboard. I can successfully run PR but can see it decorate vulnerabilities. Please assist.

jenciso commented 4 years ago

Hi @Malone5923, Is there a PR for sonar 8.2, probably it will solve your demand https://github.com/mc1arke/sonarqube-community-branch-plugin/pull/218

Malone5923 commented 4 years ago

I'm not sure there is. I really dont want to have to upgrade to 8.2 only to find out PR decorator doesnt work for Azure Devops. That will be a real bummer cause downgrading is not advisable.

jcuzzi commented 4 years ago

@Malone5923 #218 adds support for Azure DevOps. It only works with SQ 8.2. If you need a compiled version of the plugin with this PR, you can find it here: https://github.com/jcuzzi/sonarqube-community-branch-plugin/releases/download/v1.4.1-SNAPSHOT/sonarqube-community-branch-plugin-1.4.1-SNAPSHOT.jar

Otherwise, you can wait until the PR is merged and released

Malone5923 commented 4 years ago

Ok thanks @jcuzzi . I will upgrade to SQ 8.2 and try out the plugin. Hopefully it works without no ish.

alekssako commented 4 years ago

Ok thanks @jcuzzi . I will upgrade to SQ 8.2 and try out the plugin. Hopefully it works without no ish.

@Malone5923 It is working and tested on SQ 8.2/8.3/8.4, currently active in prod SQ 8.4 sonarqube-community-branch-plugin-1.4.1-SNAPSHOT.jar.

Malone5923 commented 4 years ago

So I upgraded to SonarQube version 8.2 and I have installed the community branch plugin 1.4.1 but I still cant get pull request decoration. I have added the branch status check as well as created token for pr decoration on SonarQube. Also published to the target branch first before running a pull request. Here are details of my prepare ananlysis options on Azure Build pipeline. Some direction would be great as to what I am missing. Thanks.

Additional properties that will be passed to the scanner,

Put one key=value per line, example:

sonar.exclusions=*/.bin

sonar.verbose=true sonar.branch.name=development sonar.pullrequest.key=$(System.PullRequest.PullRequestId) sonar.pullrequest.base=$(System.PullRequest.SourceBranch) sonar.pullrequest.branch=development sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/coverage.opencover.xml

jcuzzi commented 4 years ago

@Malone5923 You need to set the following scanner properties in the "Prepare analysis on SonarQube" task in the pipeline:

sonar.pullrequest.vsts.instanceUrl=https://{whatever your tenant name is}.visualstudio.com/ sonar.pullrequest.vsts.project={whatever your Azure Project name is} sonar.pullrequest.vsts.repository={whatever your Azure Repo name is}

You also need to set up branch policies to trigger a build when a pull request is created.

If its still not working, take a look at the Compute Engine log in SonarQube to get an idea of whats happening.

Malone5923 commented 4 years ago

Thanks for your swift response @jcuzzi so that we are on the same page. This is what the prepare analysis options should be?

Additional properties that will be passed to the scanner,

Put one key=value per line, example:

sonar.exclusions=*/.bin

sonar.verbose=true sonar.branch.name=development sonar.pullrequest.key=$(System.PullRequest.PullRequestId) sonar.pullrequest.base=$(System.PullRequest.SourceBranch) sonar.pullrequest.branch=development sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/coverage.opencover.xml sonar.pullrequest.vsts.instanceUrl=https://dev.azure.com/companyname/ sonar.pullrequest.vsts.project=AzureProjectName sonar.pullrequest.vsts.repository=Project Repository name

jcuzzi commented 4 years ago

@Malone5923 I don't think you need to set the below properties manually. They should be automatically set with appropriate values:

sonar.branch.name sonar.pullrequest.key sonar.pullrequest.base sonar.pullrequest.branch

The only properties I set are these:

sonar.pullrequest.vsts.instanceUrl sonar.pullrequest.vsts.project sonar.pullrequest.vsts.repository

Malone5923 commented 4 years ago

"The only properties I set are these:

sonar.pullrequest.vsts.instanceUrl sonar.pullrequest.vsts.project sonar.pullrequest.vsts.repository"

So my previous build this is already been set automatically. That is why I wanted to be sure if you want me to add it to prepare analysis option

Malone5923 commented 4 years ago

computeenginesonar.txt

Here is my coumpute engine log from sonarqube. I have gone through it and cant see what is wrong. Maybe you guys can take a look as you have more experience. Thanks

Malone5923 commented 4 years ago

So I realized sonar.branch.name was part of the properties when running a pull request. After removing that I was able to get an error of branch name returning null. Im trying to do a merge to a branch called development. Yes, this branch has been previously analysed by SonarQube and I cant see it from list of branches Here is the illegat stat exception been thrown. java.lang.IllegalStateException: Could not find target branch 'null' in project at com.github.mc1arke.sonarqube.plugin.ce.CommunityBranchLoaderDelegate.createPullRequest(CommunityBranchLoaderDelegate.java:98) at com.github.mc1arke.sonarqube.plugin.ce.CommunityBranchLoaderDelegate.load(CommunityBranchLoaderDelegate.java:78) at com.github.mc1arke.sonarqube.plugin.ce.CommunityBranchLoaderDelegate.load(CommunityBranchLoaderDelegate.java:50) at org.sonar.ce.task.projectanalysis.component.BranchLoader.load(BranchLoader.java:44) at org.sonar.ce.task.projectanalysis.step.LoadReportAnalysisMetadataHolderStep.execute(LoadReportAnalysisMetadataHolderStep.java:90) at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81) at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72) at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59) at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81) at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:209) at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:191) at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:158) at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:133) at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:85) at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Also here is my sonar.properties on Azure build pipeline for the pull request `# Additional properties that will be passed to the scanner,

Put one key=value per line, example:

sonar.exclusions=*/.bin

sonar.verbose=true sonar.pullrequest.key=$(System.PullRequest.PullRequestId) sonar.pullrequest.base=$(System.PullRequest.SourceBranch) sonar.pullrequest.branch=development sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/coverage.opencover.xml`

Please help, I really need PR decoration to work

jcuzzi commented 4 years ago

You need to do the following:

  1. In SonarQube, go to Administration > Configuration > General > ALM Integration and enable Azure DevOps with your PAT. The PAT needs Code: Read & Write permission to Azure DevOps.

  2. In SonarQube, go to Projects > {Your Project} >Project Settings > General Settings > Pull Request Decoration and choose "azure" for that project.

  3. In the Azure Pipeline, make sure you have the following scanner properties configured: sonar.pullrequest.vsts.instanceUrl sonar.pullrequest.vsts.project sonar.pullrequest.vsts.repository

  4. In Azure, configure a branch policy and enable Build Validation to run a pipeline build when a PR is issued.

  5. Issue a PR targeting that branch with the policy.

  6. Watch as the pipeline triggers, SonarQube analyzes, and PR decoration is triggered on the SQ Server (you should see this in the CE log)

  7. Profit.

Malone5923 commented 4 years ago

@jcuzzi . Thanks so much. Got it working. Apparently the whole sonar.pullrequest.key=$(System.PullRequest.PullRequestId) sonar.pullrequest.base=$(System.PullRequest.SourceBranch) sonar.pullrequest.branch=development and only caused more harm than good. I think the readme.md else most people we spend hours trying to figure out what went wrong. Issue closed. Thanks once again

rezarms commented 4 years ago

Hi @jcuzzi I'm using SQ 8.3 and Azure Devops Server 2019 and followed your instructions but when I want to enable PR decoration on project setting it says 'this feature must be enable first in global settings' which I did and in global setting I chose AzureDevops Server and put my PAT.I also get another error on project setting that says 'This request can not be processed. Try again later'. I have set below properties and I can see them in the log: sonar.pullrequest.vsts.instanceUrl sonar.pullrequest.vsts.project sonar.pullrequest.vsts.repository But still there is no luck with it. I don't see any error in logs. sonar

jcuzzi commented 4 years ago

@rezarms After you enable it globally, you also need to enable decoration on each project. Look at the general settings section at the project level - you should see a Pull Request Decoration tab: image

rezarms commented 4 years ago

@jcuzzi I'm using the same version. It seems the issue is from project setting because for me it's not enabled even though I enabled it globally. I'm not sure what I'm missing.

sonar sonar2

jcuzzi commented 4 years ago

@rezarms Could be a compatibility issue with SQ 8.3. I only ever tested this on SQ 8.2 because my branch is based off of @mc1arke 's master branch which only supports SQ 8.2

rezarms commented 4 years ago

@jcuzzi Found the issue. It was a stupid mistake. I had your version in extension folder and the original version in common folder. I was able to select azure in project PR decorate tab after triggering a PR another issue occurred. Just to add that my Sonar and Azure server both are using SSL. I guess I need to add my certificate to JVM.

I can see below error in ce.log

java.lang.IllegalStateException: Could not decorate Pull Request on AzureDevOps Server at com.github.mc1arke.sonarqube.plugin.ce.pullrequest.azuredevops.AzureDevOpsServerPullRequestDecorator.decorateQualityGateStatus(AzureDevOpsServerPullRequestDecorator.java:141) at com.github.mc1arke.sonarqube.plugin.ce.pullrequest.PullRequestPostAnalysisTask.finished(PullRequestPostAnalysisTask.java:160) at org.sonar.ce.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.executeTask(PostProjectAnalysisTasksExecutor.java:118) at org.sonar.ce.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.finished(PostProjectAnalysisTasksExecutor.java:109) at org.sonar.ce.task.step.ComputationStepExecutor.executeListener(ComputationStepExecutor.java:91) at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:63) at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81) at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:209) at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:191) at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:158) at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:133) at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:85) at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:327) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:270) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:265) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:646) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:465) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:361) at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:451) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:428) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:184) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1062) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at com.github.mc1arke.sonarqube.plugin.ce.pullrequest.azuredevops.AzureDevOpsServerPullRequestDecorator.sendPost(AzureDevOpsServerPullRequestDecorator.java:312) at com.github.mc1arke.sonarqube.plugin.ce.pullrequest.azuredevops.AzureDevOpsServerPullRequestDecorator.decorateQualityGateStatus(AzureDevOpsServerPullRequestDecorator.java:112) ... 21 common frames omitted Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) at java.base/sun.security.validator.Validator.validate(Validator.java:264) at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313) at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222) at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:630) ... 45 common frames omitted Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)

jcuzzi commented 4 years ago

@rezarms Yes, looks like you're on the right track. Java doesn't like your Azure DevOps Server certificate. Make sure the cert chain is trusted in the java keystore that SQ uses.

rezarms commented 4 years ago

@jcuzzi I was able to get rid of the error but unfortunately it doesn't support Azure Devops server 2019. Can the rest api be called with lower version or it's supported only on Azure Cloud?

sendPost: {"$id":"1","innerException":null,"message":"The requested REST API version of 6.0 is out of range for this server. The latest REST API version this server supports is 5.1.","typeName":"Microsoft.VisualStudio.Services.WebApi.VssVersionOutOfRangeException, Microsoft.VisualStudio.Services.WebApi","typeKey":"VssVersionOutOfRangeException","errorCode":0,"eventId":3000}

jcuzzi commented 4 years ago

@rezarms You can use the following scanner property to set the api version of Azure DevOps.

sonar.pullrequest.vsts.apiVersion

rezarms commented 4 years ago

@jcuzzi good job you already thought of that. I made it work. Thanks for your help.

bbarman4u commented 3 years ago

Ok thanks @jcuzzi . I will upgrade to SQ 8.2 and try out the plugin. Hopefully it works without no ish.

@Malone5923 It is working and tested on SQ 8.2/8.3/8.4, currently active in prod SQ 8.4 sonarqube-community-branch-plugin-1.4.1-SNAPSHOT.jar.

@alekssako I am trying this plugin on SQ 8.4 and I am not seeing the options to even enable Azure DevOps, I only see github and gitlab. Is this confirmed to work on SQ 8.4 ?

jcuzzi commented 3 years ago

@bbarman4u Are you using the version from this repository or my fork? Azure DevOps support is not yet included in the official releases.

bbarman4u commented 3 years ago

@bbarman4u Are you using the version from this repository or my fork? Azure DevOps support is not yet included in the official releases.

@jcuzzi Yes Sir ! I have your snapshot in the plugins folder. I also went back and tested the same on 8.3 and 8.2 versions and I am not seeing the ALM page show Azure DevOps. There must be something I am missing.

bbarman4u commented 3 years ago

@bbarman4u Are you using the version from this repository or my fork? Azure DevOps support is not yet included in the official releases.

@jcuzzi Yes Sir ! I have your snapshot in the plugins folder. I also went back and tested the same on 8.3 and 8.2 versions and I am not seeing the ALM page show Azure DevOps. There must be something I am missing.

Ok I may have misinterpreted an earlier comment in this thread, as per the official readme, I need to put the extension in both extensions and common folder. I should have read the readme better, my apologies for that.

bbarman4u commented 3 years ago

@bbarman4u Are you using the version from this repository or my fork? Azure DevOps support is not yet included in the official releases.

@jcuzzi Yes Sir ! I have your snapshot in the plugins folder. I also went back and tested the same on 8.3 and 8.2 versions and I am not seeing the ALM page show Azure DevOps. There must be something I am missing.

Ok I may have misinterpreted an earlier comment in this thread, as per the official readme, I need to put the extension in both extensions and common folder. I should have read the readme better, my apologies for that.

@jcuzzi So it did work out great on SonarQube 8.4, Great Job Done on this new feature addition, Kudos !