search

uartois / sonar-golang

Sonarqube plugin for the golang language.
GNU Lesser General Public License v3.0
245 stars 32 forks source link

Capture coverage failing when using docker #35

Open kenjones-cisco opened 6 years ago

kenjones-cisco commented 6 years ago

Description

When attempting to publish coverage.xml, report.xml, and test.xml to SonarQube after generating the individual xml files, I'm seeing the following happen.

  1. The coverage is not honoring the sonar.exclusions entry, as all the files in my .glide/** and vendor/** are being processed.
  2. unable to create InputFile object: gitscm.cisco.com/ccdev/cicdtest/version/info.go
  3. java.lang.StringIndexOutOfBoundsException: String index out of range: -1

Steps to Reproduce

  1. Build Dockerfile for running SonarQube locally 
    
FROM sonarqube:6.5-alpine

RUN wget -O "$SONARQUBE_HOME/extensions/plugins/sonar-golang-plugin-1.2.11-rc3.jar" \ https://github.com/uartois/sonar-golang/releases/download/v1.2.11-rc3/sonar-golang-plugin-1.2.11-rc3.jar


```bash
docker build -t cicdtest_sonar -f Dockerfile.cd .
  1. Start SonarQube 
    docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 cicdtest_sonar
  2. Run sonar-scanner 
    sonar-scanner -X \
-Dsonar.host.url=http://localhost:9000 \
-Dsonar.login=534221ba8db9121b9afe15a322a5fa1f17d3c9df

Content of your sonar-project.properties

https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1#file-sonar-project-properties

It will help us to check how you configured the plugin.

Log of sonar-scanner related to the plugin

https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1#file-scanner-log

The plugin logs major events, including which report files have been found and analyzed.

Versions

SonarQube Server

            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin",
                "LANG=C.UTF-8",
                "JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk",
                "JAVA_VERSION=8u131",
                "JAVA_ALPINE_VERSION=8.131.11-r2",
                "SONAR_VERSION=6.5",
                "SONARQUBE_HOME=/opt/sonarqube",
                "SONARQUBE_JDBC_USERNAME=sonar",
                "SONARQUBE_JDBC_PASSWORD=sonar",
                "SONARQUBE_JDBC_URL="
            ],

sonar-scanner

INFO: Scanner configuration file: C:\Users\kenjones\.local\bin\sonar-scanner-3.0.3.778\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: C:\Users\kenjones\projects\examples\cicdtest\sonar-project.properties
INFO: SonarQube Scanner 3.0.3.778
INFO: Java 1.8.0_131 Oracle Corporation (64-bit)
INFO: Windows 10 10.0 amd64

go version go version go1.9 linux/amd64

gometalinter c039b9f1fbff9a38595e19e9d97474ed57270cb8 Last commit as of yesterday

OS

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

All go, gometalinter, etc. are run inside of Docker containers

Versions of your sonar installation (server, sonar-scanner), gometalinter, go and OS.

Additional Information

All *.xml are in the same gist as the other files: https://gist.github.com/kenjones-cisco/8b5f55f4f5197d3ddfba66142aeefbc1

Seems similar to #14

Sample reports may be necessary to reproduce the issue. Please attach if possible reports files (report.xml, coverage.xml, test.xml) corresponding to the feature which is not working as expected.

kenjones-cisco commented 6 years ago

To be completely clear. SonarQube is running inside of a container. All development tasks are executed inside of a container. The only command running locally is sonar-scanner.

Once SonarCube is upgraded to within my CI/CD platform. SonarQube is run on a VM. Jenkins provides the sonar-scanner to integrate with SonarQube, but all the go related tasks will happen in the exact same container that I'm using locally (provides the abstract layer that avoids the "it works on my laptop", as Docker runs on linux, MacOS, and Windows).

kenjones-cisco commented 6 years ago

Just in case it is helpful or relevant, the development container is defined as follows:

FROM golang:1.9

RUN apt-get update && apt-get install -y --no-install-recommends \
        gettext-base \
        zip \
    && rm -rf /var/lib/apt/lists/*

ENV YAML_BIN_VERSION 1.13.1
ENV JQ_BIN_VERSION 1.5

RUN wget -O /usr/local/bin/jq "https://github.com/stedolan/jq/releases/download/jq-${JQ_BIN_VERSION}/jq-linux64" \
    && chmod 755 /usr/local/bin/jq \
    && wget -O /usr/local/bin/yaml "https://github.com/mikefarah/yaml/releases/download/${YAML_BIN_VERSION}/yaml_linux_amd64" \
    && chmod 755 /usr/local/bin/yaml

ARG DEVELOPER

RUN git config --global user.name "${DEVELOPER}" \
    && git config --global user.email "${DEVELOPER}@cisco.com" \
    && git config --list

ENV GLIDE_VERSION v0.12.3

RUN go get github.com/alecthomas/gometalinter \
    && gometalinter --install --vendor \
    && go get golang.org/x/tools/cmd/goimports \
    && go get github.com/mitchellh/gox \
    && go get github.com/davecheney/godoc2md \
    && go get github.com/axw/gocov/gocov \
    && go get github.com/t-yuki/gocov-xml \
    && go get github.com/tebeka/go2xunit \
    && go get github.com/go-swagger/go-swagger/cmd/swagger \
    && curl -Lo /tmp/glide.tgz https://github.com/Masterminds/glide/releases/download/$GLIDE_VERSION/glide-$GLIDE_VERSION-linux-amd64.tar.gz \
    && tar -C /usr/bin -xzf /tmp/glide.tgz --strip=1 linux-amd64/glide \
    && rm /tmp/glide.tgz

ENV CGO_ENABLED 0
ENV GOPATH /go:/cicdtest
danielleberre commented 6 years ago
  1. happens because we manually look for XML cover files in the project subdirectory in the plugin.

@thibaultfalque we need to retrieve the sonar.exclusions property and prevent our search to enter those directories.

danielleberre commented 6 years ago

We just added the code to prevent files in the exclusion path to be analyzed. The next RC will allow to check that it fixes your problem.

thibaultfalque commented 6 years ago

Please check release 1.2.11-RC8.

thibaultfalque commented 6 years ago

@kenjones-cisco could you check with the last release and close the issue if the problem is resolved ?

kenjones-cisco commented 6 years ago

Issues 1 and 2 still exist

Even though I have the following defined: sonar.exclusions=vendor/**,.git/**,.glide/**,**/*.xml It still keeps scanning the vendor/, .git/, and the .glide/ directories.

And when the coverage file is generated the filename inside the coverage.xml shows up as gitscm.cisco.com/ccdev/cicdtest/version/info.go since it includes the package. But given the file is actually in the local directory as the top level package is "virtual" due to the use of docker, the error unable to create InputFile object: gitscm.cisco.com/ccdev/cicdtest/version/info.go is still happening.

danielleberre commented 6 years ago

@thibaultfalque we need to check if sonar.exclusions is properly honored in each sensor.

Regarding the second issue, I have really no idea how to fix it since I am not a regular docker user. Any idea would be welcome.

kenjones-cisco commented 6 years ago

@danielleberre I wondering if there is an assumption about what file paths look like for a golang code base.

Most have their source control within github which results in a base import package of github.com/<username|orgname>/<project_name>. When cloning a project from github the code is by default in a directory named <project_name>.

As such if your code base has multiple packages you will have to provide the full import path. Therefore when you build, test, etc. The code needs to appear as if it is in src/<import_path> (ex. src/github.com/uartois/sonar-golang).

Therefore when mounting the code base into a docker container, it will be mounted on a path that looks like /opt/src/<import_path>. As such when the coverage file is generated the full package name is captured under package name="" and then the file name looks like <package_name>/<file>.go

As such I have tried running sonar-scanner inside a container with my source code mounted in a similar way with the sonar-project.properties file at the root of my project, but even then it seems to be unable to match the filename path to the actual files.

It seems as if sonar-scanner expects to be run from src/ and then walks through down to the code located inside of <import_path>. That of course resulted in a huge problem because sonar-scanner wanted to walk through every directory within src/ even if I specified in the sonar-project.properties that the source is located at <import_path> or tried to exclude every directory within src/ except for the one I wanted scanned. As a result the lack of exclusion resulted in 6 or 7 projects with their own vendor/ directories being scanned and uploading 28MB of results but because the data set for a single project was now so large it was difficult to determine if the file was picked properly or not.

danielleberre commented 6 years ago

You are right, we crawl the top level directory to find the resources.

Maybe we do not do it properly because we have to take into account sonar.exclusion manually. This is our first SonarQube plugin, we looked around how the other plugin proceed, and did not notice any specific way to do that crawling.

Let us sleep on it, we will try to find a way to solve your problem. Docker is getting so popular that it is probably an issue for other users too.

introllo commented 6 years ago

Hi @kenjones-cisco. Try putting "." on the "Sources location" in build step (we use TeamCity). On the sonar-scanner side, I put this in my sonar-project.properties:

sonar.sources=.

This works for me (from a Docker container). Updating the plugin to try again via TC runner.

kenjones-cisco commented 6 years ago

Thanks @introllo but my configuration is below and that is what I use as well.

sonar.projectKey=cicdtest:0.2.0
sonar.projectName=cicdtest
sonar.projectVersion=0.2.0
sonar.golint.reportPath=report.xml
sonar.coverage.dtdVerification=false
sonar.coverage.reportPath=coverage.xml
sonar.test.reportPath=test.xml
sonar.sources=.
sonar.exclusions=vendor/**,.git/**,.glide/**,**/*.xml
introllo commented 6 years ago

Hmmm. Pretty much the same as mine:

sonar.projectKey=[SET_ME]
sonar.projectName=[SET_ME]
sonar.projectVersion=[SET_ME]
sonar.golint.reportPath=report.xml
sonar.coverage.reportPath=coverage.xml
sonar.coverage.dtdVerification=false
sonar.test.reportPath=test.xml
sonar.sources=.
sonar.coverage.exclusions=vendor/**
sonar.exclusions=vendor/**,testdata/**

Let me dbl-check my setup.

introllo commented 6 years ago

Apologies @kenjones-cisco . I checked my notes (its been awhile) and see that we use vagrant provisioned VMs for this as we can't run our test code inside of a Docker container. So, in my case, I'm running sonar-scanner in an Ubuntu VM. Thx.

danielleberre commented 6 years ago

Ok, I asked for advices to take into account sonar.exclusion properly: https://groups.google.com/forum/#!topic/sonarqube/vPrza6ovvSU

thibaultfalque commented 6 years ago

I have fix the problem of sonar.exclusion in this commit d5c2d0224bbc20267177e43ff9ddf1fa9def1905

thibaultfalque commented 6 years ago

Please test https://github.com/uartois/sonar-golang/releases/tag/v1.2.11-rc9

thibaultfalque commented 6 years ago

@kenjones-cisco could you test with the last release ?

denouche commented 6 years ago

Hello,

I also have this issue with the latest release (v1.2.11-rc13).

This is what I have:

INFO: Analyse for /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/coverage.xml

WARN: unable to create InputFile object: /go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/handler.go

And then no coverage in Sonarqube.

And this is the beginning of the generated coverage.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-03.dtd">
<coverage line-rate="0" branch-rate="0" version="" timestamp="1518793888035">
    <packages>
        <package name="github.enterprise.com/enterprise/AuthenticationService-Go/handlers" line-rate="0" branch-rate="0" complexity="0">
            <classes>
                <class name="AuthenticationHandler" filename="/go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/token_handler.go" line-rate="0" branch-rate="0" complexity="0">
                    <methods>
                        <method name="PostToken" signature="" line-rate="0" branch-rate="0">
                            <lines>
                                <line number="129" hits="1"></line>
                                <line number="130" hits="0"></line>
                                <line number="131" hits="0"></line>
...
thibaultfalque commented 6 years ago

Could you post your coverage report?

denouche commented 6 years ago

Yes of course

coverage.zip

Thank you!

thibaultfalque commented 6 years ago

/go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/handler.go exists ? Do you use docker ? What is your hierarchy of files ? What is the path when sonar-scanner is run ?

denouche commented 6 years ago

This is the Dockerfile of the image where I create the sonar reports files:

FROM golang:1.9-alpine3.7

RUN apk --no-cache add -U make git musl-dev gcc

RUN go get -u github.com/golang/dep/cmd/dep
RUN go get -u github.com/go-swagger/go-swagger/cmd/swagger

WORKDIR $GOPATH/src/github.enterprise.com/enterprise/AuthenticationService-Go
ADD . .

# Install dependencies
RUN dep ensure
RUN go get -u github.com/alecthomas/gometalinter
RUN gometalinter --install

RUN go get github.com/axw/gocov/...
RUN go get github.com/AlekSi/gocov-xml

Then I use the following commands to get the reports: docker run --rm image gometalinter --checkstyle > report.xml docker run --rm image sh -c 'gocov test ./... | gocov-xml' > coverage.xml

And then I execute directly on the jenkins agent: /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube-scanner/bin/sonar-scanner from the project path /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA Which give me this stacktrace:

INFO: Scanner configuration file: /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/sonar-project.properties
INFO: SonarQube Scanner 3.0.1.733
INFO: Java 1.8.0_151 Oracle Corporation (64-bit)
INFO: Linux 4.4.0-112-generic amd64
INFO: User cache: /var/lib/jenkins/.sonar/cache
INFO: Load global settings
INFO: Load global settings (done) | time=127ms
INFO: User cache: /var/lib/jenkins/.sonar/cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=6ms
INFO: SonarQube server 6.3.1
INFO: Default locale: "en_US", source code encoding: "US-ASCII" (analysis is platform dependent)
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=75ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=1ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=35ms
INFO: Load active rules
INFO: Load active rules (done) | time=696ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=77ms
INFO: Publish mode
INFO: Project key: AuthenticationService-Go
INFO: -------------  Scan AuthenticationService-Go
INFO: Load server rules
INFO: Load server rules (done) | time=126ms
INFO: Language is forced to go
INFO: Initializer GenericCoverageSensor
INFO: Initializer GenericCoverageSensor (done) | time=0ms
INFO: Base dir: /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA
INFO: Working dir: /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.scannerwork
INFO: Source paths: .
INFO: Source encoding: US-ASCII, default locale: en_US
INFO: Index files
INFO: Excluded sources: 
INFO:   "vendor"
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/report.xml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/values.yaml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/Chart.yaml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/NOTES.txt' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/ingress.yaml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/_helpers.tpl' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/service.yaml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/deployment.yaml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Makefile' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/sonar-project.properties' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Gopkg.toml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/keycloak-init.json' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Gopkg.lock' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Dockerfile' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Dockerfile.sonar' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/integration-test/features/token_handler.feature' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/integration-test/features/userinfo_handler.feature' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/integration-test/features/introspect_handler.feature' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/docker-compose.yml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/README.md' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/coverage.xml' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Jenkinsfile' is ignored because it doens't belong to the forced langauge 'go'
WARN: File '/data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/Dockerfile.integration-test' is ignored because it doens't belong to the forced langauge 'go'
INFO: 34 files indexed
INFO: 0 files ignored because of inclusion/exclusion patterns
INFO: Quality profile for go: Golint Rules
INFO: Sensor GoMetaLinter issues loader sensor [golang]
INFO: Parsing the file report.xml
INFO: Parsing 'GoMetaLinter' Analysis Results
INFO: Load /key.properties
INFO: loaded 52
INFO: Sensor GoMetaLinter issues loader sensor [golang] (done) | time=37ms
INFO: Sensor Go Coverage [golang]
INFO: /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA
INFO: Analyse for /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/coverage.xml
WARN: unable to create InputFile object: /go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/handler.go
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/deployment/authenticationservice-go/templates/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/constant/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/info/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/branches/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/logs/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/logs/refs/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/logs/refs/remotes/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/logs/refs/remotes/origin/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/logs/refs/remotes/origin/chore/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/hooks/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/objects/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/objects/pack/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/objects/info/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/heads/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/remotes/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/remotes/origin/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/remotes/origin/chore/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/.git/refs/tags/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/services/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/utils/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/handlers/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/model/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/integration-test/coverage.xml
INFO: no coverage file in package /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/integration-test/features/coverage.xml
INFO: Sensor Go Coverage [golang] (done) | time=92ms
INFO: Sensor Go test JUnit loader sensor [golang]
INFO: no junit report
INFO: Sensor Go test JUnit loader sensor [golang] (done) | time=0ms
INFO: Sensor Go Highlighter Sensor [golang]
WARN: Invalid character encountered in file /data/jenkins/workspace/ationService-Go_chore_sonar-BUGXILDQDK6PPNYIOVBIGNWPEOEDYM7IDLNDBWO5MBCRQOTNCQSA/doc.go at line 168 for encoding US-ASCII. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
INFO: Sensor Go Highlighter Sensor [golang] (done) | time=76ms
INFO: Sensor Go Metrics Sensor [golang]
INFO: Sensor Go Metrics Sensor [golang] (done) | time=11ms
INFO: Sensor C# Unit Test Results Import [csharp]
INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=0ms
INFO: Sensor SonarJavaXmlFileSensor [java]
INFO: Sensor SonarJavaXmlFileSensor [java] (done) | time=0ms
INFO: Sensor Analyzer for "php.ini" files [php]
INFO: Sensor Analyzer for "php.ini" files [php] (done) | time=2ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=14ms
INFO: Sensor Code Colorizer Sensor
INFO: Sensor Code Colorizer Sensor (done) | time=1ms
INFO: Sensor CPD Block Indexer
INFO: org.sonar.scanner.cpd.deprecated.DefaultCpdBlockIndexer@5fdd97c1 is used for go
INFO: Sensor CPD Block Indexer (done) | time=0ms
INFO: Calculating CPD for 0 files
INFO: CPD calculation finished
INFO: Analysis report generated in 100ms, dir size=195 KB
INFO: Analysis reports compressed in 44ms, zip size=87 KB
INFO: Analysis report uploaded in 67ms
INFO: ANALYSIS SUCCESSFUL, you can browse https://sonarqube.enterprise.com/dashboard/index/AuthenticationService-Go
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at https://sonarqube.enterprise.com/api/ce/task?id=AWGfho-qhf3uZUkgo6ux
INFO: Task total time: 2.938 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 4.037s
INFO: Final Memory: 53M/364M
INFO: ------------------------------------------------------------------------

Containing the WARN: unable to create InputFile object: /go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/handler.go warning which stop the coverage report process I guess.

So in Sonarqube I have the lint reports but not the coverage informations.

Thank you for your help!!

thibaultfalque commented 6 years ago

I am not a daily user of docker. Your docker container have access on /go/... ?

danielleberre commented 6 years ago

AFAI understand what's going on, the test and coverage reports are run on docker while the sonar-scanner is run directly. As such, the paths in the coverage report do not match the paths in the current filesystem.

If we can rebuild the correct file path knowing the prefix of the path in the docker image, we could probably add a specific option for docker users to fix this issue.

But we need some help from regular users of docker to do that properly.

denouche commented 6 years ago

Yes that's also what I suspected.

The project path is generated since I'm building a Jenkins Pipeline. So I thought to replace the path in the coverage.xml path to match the jenkins workspace path, but it's not the cleaner way.

What did you mean by "add a specific option"? Add a parameter when executing sonar-scanner ? A parameter giving the base path of the project files in the docker image, so it would be possible the rebuild the path using the current path of the project in the jenkins workspace and this new argument ?

denouche commented 6 years ago

There is an opened pull request on github.com/AlekSi/gocov-xml (here github.com/AlekSi/gocov-xml/pull/4 adding some interesting options.

When using github.com/Centny/gocov-xml and using the -pwd argument, the generated coverage.xml is now:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-03.dtd">
<coverage line-rate="0" branch-rate="0" version="" timestamp="1518885747608">
    <packages>
        <package name="github.enterprise.com/enterprise/AuthenticationService-Go/handlers" line-rate="0" branch-rate="0" complexity="0">
            <classes>
                <class name="AuthenticationHandler" filename="handlers/token_handler.go" line-rate="0" branch-rate="0" complexity="0">
                    <methods>

instead of:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-03.dtd">
<coverage line-rate="0" branch-rate="0" version="" timestamp="1518885504243">
    <packages>
        <package name="github.enterprise.com/enterprise/AuthenticationService-Go/handlers" line-rate="0" branch-rate="0" complexity="0">
            <classes>
                <class name="AuthenticationHandler" filename="/go/src/github.enterprise.com/enterprise/AuthenticationService-Go/handlers/health_handler.go" line-rate="0" branch-rate="0" complexity="0">
                    <methods>

So the filename is now relative, so maybe I would solve the problem.

I will investigate this on monday and I will give you some feedback.

denouche commented 6 years ago

I could not wait until monday, I just tested it and it works, I now have the coverage informations in sonarqube.

But I guess I will have to use the github.com/Centny/gocov-xml fork since the PR is opened since 2013...

Thanks for your help!

danielleberre commented 6 years ago

That's a good news!

Maybe we should comment in gocov's PR that it would be helpful in our case?

danielleberre commented 6 years ago

Dear all,

I updated the README.md page to mention the workaround for docker users.

@kenjones-cisco could you check that the workaround solves your problem as well?

Thanks again @denouche for providing us that workaround.

introllo commented 6 years ago

Have only recently been able to return to this issue. Have deployed updated version and now see my Go coverage metrics in my Sonar server. I tested this combination:

TeamCity 10.0.4 & 2017.1.2 running as Docker containers (both build server and build agent) SonarQube 6.4 running as Docker containers (Postgres and "eval" databases) Sonar-Golang 1.2.11-rc13

Thanks for all the hard work!