Find an alternative distribution channel to JPM

[yaravind]

I am trying to use codacy to enable code quality checks for my https://github.com/yaravind/kafka-connect-jenkins

It is getting very hard to install jpm on snap ci infrastructure. I was wondering if there is alternate way. Or if you can provide some details on where to start, I can contribute a maven plug-in to report the coverage (just like jococo maven plug-in)

[machadoit]

Hi @yaravind,

That sounds amazing, we would really appreciate your contribution! You can check the sbt-plugin https://github.com/codacy/sbt-codacy-coverage which is similar, to help to get you started.

Let us know if you need anything.

[pedrorijo91]

@yaravind do you have any updates on this? can we help in something?

[halkeye]

~~~I can't find the definition of CoverageParserFactory~~~

https://github.com/halkeye/codacy-maven-plugin has been started.
Very basic port right now,  Turned out easier than expected.
Just gotta figure out how commit id is figured out, and get my own tokens to test thigns

[pedrorijo91]

hey @halkeye !

really happy to hear that !

To figure out commit id, I would recommend to fetch from a env variable like we did on our sbt plugin

Alternatively, you can assume you are in a git folder and exec a git command - through a command (example) or using a git library.

[halkeye]

Can you upgrade to netty-http-pipeline 1.1.4 so it can be pulled from maven central? https://github.com/typesafehub/netty-http-pipelining/issues/13

[halkeye]

Yea, trying to do excludes and stuff, i get:

[ERROR] Failed to execute goal com.gavinmogan:codacy-maven-plugin:1.0-SNAPSHOT:coverage (default-cli) on project ci-sauce: Execution default-cli of goal com.gavinmogan:codacy-maven-plugin:1.0-SNAPSHOT:coverage failed: An API incompatibility was encountered while executing com.gavinmogan:codacy-maven-plugin:1.0-SNAPSHOT:coverage: java.lang.NoSuchMethodError: org.jboss.netty.handler.codec.http.HttpRequest.setHeader(Ljava/lang/String;Ljava/lang/Object;)V

So i don't think i can upgrade it just on my side.

[pedrorijo91]

just did a pr that should fix your problem @halkeye

Should fix the error regarding netty-http-pipelining dependency

Also, removed codacy-api scala sdk from dependencies, since you should not need it.

Probably you can use the coverage-parser to parse the coverage report, and then you just need to use codacy-api-java to send the coverage report parsed to our format.

You should not need codacy-coverage-reporter. This plugin works as an alternative for codacy-coverage-reporter, so you shouldn't need it as a dependency

Note: I did not had the time to look into the code, so you may have valid reasons to use codacy-coverage-reporter. In that case, ignore my last comment and keep the good work 👍

[halkeye]

I can't imagine this is the best place for these conversations, but I'm kinda out of ideas.

It gets into the sendReport function, and just hangs.

Adding debugging gets me:

[DEBUG]
Non cached request
DefaultHttpRequest(chunked: false)
POST /2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
Host: api.codacy.com
api_token: <snipApiKey>
project_token: <snipProjectKey>
Content-Type: application/json
Connection: keep-alive
Accept: */*
User-Agent: NING/1.0
Content-Length: 4804

using Channel
[id: 0x2a859b27]

[DEBUG]

Request DefaultHttpRequest(chunked: false)
POST /2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
Host: api.codacy.com
api_token: <snipApiKey>
project_token: <snipProjectKey>
Content-Type: application/json
Connection: keep-alive
Accept: */*
User-Agent: NING/1.0
Content-Length: 4804

Response DefaultHttpResponse(chunked: false)
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Tue, 30 Aug 2016 02:13:24 GMT
Request-Time: 14
Server: nginx/1.6.2
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Permitted-Cross-Domain-Policies: master-only
X-XSS-Protection: 1; mode=block
Content-Length: 45
Connection: keep-alive

[DEBUG] Adding uri: https://api.codacy.com:443 for channel [id: 0x2a859b27, /172.16.10.2:56612 => api.codacy.com/54.72.117.61:443]
[DEBUG] AsyncHttpClient.close() hasn't been invoked, which may produce file descriptor leaks

I know its blocked in CodacyClient.scala

      val jpromise = client.url(s"$remoteUrl/${request.endpoint}")
        .withHeaders(headers: _*)
        .withFollowRedirects(follow = true)
        .post(value)

doesn't even get to val result = Await.result(jpromise, Duration(10, SECONDS))

[halkeye]

MORE INFO!

I switched to a java http client, just so i could better control what was going on

I'm getting {"error":"could not get json body."} and not really sure what I could be doing wrong. I'm sending a json body, and json headers.

[INFO] Executing request POST https://api.codacy.com/2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
[DEBUG] CookieSpec selected: default
[DEBUG] Auth cache not set in the context
[DEBUG] Connection request: [route: {s}->https://api.codacy.com:443][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[DEBUG] Connection leased: [id: 0][route: {s}->https://api.codacy.com:443][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] Opening connection {s}->https://api.codacy.com:443
[DEBUG] Connecting to api.codacy.com/54.171.250.4:443
[DEBUG] Connecting socket to api.codacy.com/54.171.250.4:443 with timeout 0
[DEBUG] Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
[DEBUG] Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
[DEBUG] Starting handshake
[DEBUG] Secure session established
[DEBUG]  negotiated protocol: TLSv1.2
[DEBUG]  negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[DEBUG]  peer principal: CN=*.codacy.com, OU=Domain Control Validated - RapidSSL(R), OU=See www.rapidssl.com/resources/cps (c)15, OU=GT93619575
[DEBUG]  peer alternative names: [*.codacy.com, codacy.com]
[DEBUG]  issuer principal: CN=RapidSSL SHA256 CA - G3, O=GeoTrust Inc., C=US
[DEBUG] Connection established 172.16.10.2:40428<->54.171.250.4:443
[DEBUG] Executing request POST /2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
[DEBUG] Target auth state: UNCHALLENGED
[DEBUG] Proxy auth state: UNCHALLENGED
[DEBUG] http-outgoing-0 >> POST /2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
[DEBUG] http-outgoing-0 >> api_token: <apiToken>
[DEBUG] http-outgoing-0 >> project_token: <projectToken>
[DEBUG] http-outgoing-0 >> Content-Type: application/json
[DEBUG] http-outgoing-0 >> Content-Length: 4806
[DEBUG] http-outgoing-0 >> Host: api.codacy.com
[DEBUG] http-outgoing-0 >> Connection: Keep-Alive
[DEBUG] http-outgoing-0 >> User-Agent: Apache-HttpClient/4.4 (Java 1.5 minimum; Java/1.8.0_31)
[DEBUG] http-outgoing-0 >> Accept-Encoding: gzip,deflate
[DEBUG] http-outgoing-0 >> "POST /2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1[\r][\n]"
[DEBUG] http-outgoing-0 >> "api_token: <apiToken>[\r][\n]"
[DEBUG] http-outgoing-0 >> "project_token: <projectToken>[\r][\n]"
[DEBUG] http-outgoing-0 >> "Content-Type: application/json[\r][\n]"
[DEBUG] http-outgoing-0 >> "Content-Length: 4806[\r][\n]"
[DEBUG] http-outgoing-0 >> "Host: api.codacy.com[\r][\n]"
[DEBUG] http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.4 (Java 1.5 minimum; Java/1.8.0_31)[\r][\n]"
[DEBUG] http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] http-outgoing-0 >> "[\r][\n]"
[DEBUG] http-outgoing-0 >> "[{"language":"Java","total":51,"fileReports":[{"filename":"com/saucelabs/ci/OperatingSystemDescription.java","total":100,"coverage":{"12":1,"15":1,"11":1,"9":1,"33":1,"22":1,"26":1,"50":1,"37":1,"13":1,"16":1,"10":1,"48":1,"32":1,"17":1,"25":1,"14":1,"47":1,"31":1,"18":1,"30":1,"52":1,"28":1}},{"filename":"com/saucelabs/ci/Browser.java","total":46,"coverage":{"113":1,"34":1,"147":1,"73":1,"62":1,"33":1,"163":1,"50":1,"68":1,"61":1,"46":1,"155":1,"35":1,"152":1,"159":1,"59":1,"144":1,"54":1,"65":1,"32":1,"31":1,"151":1,"42":1,"27":1,"70":1,"38":1,"167":1,"160":1,"171":1,"30":1,"143":1,"29":1,"28":1}},{"filename":"com/saucelabs/ci/JobInformation.java","total":81,"coverage":{"205":1,"169":1,"153":1,"51":1,"124":1,"276":1,"170":1,"265":1,"88":1,"40":1,"269":1,"236":1,"33":1,"273":1,"280":1,"250":1,"123":1,"61":1,"206":1,"296":1,"79":1,"279":1,"152":1,"268":1,"274":1,"87":1,"196":1,"281":1,"270":1,"188":1,"43":1,"285":1,"297":1,"241":1,"289":1,"60":1,"133":1,"96":1,"69":1,"278":1,"245":1,"95":1,"264":1,"42":1,"246":1,"189":1,"266":1,"238":1,"282":1,"277":1,"187":1,"235":1,"125":1,"247":1,"154":1,"143":1,"97":1,"179":1,"114":1,"272":1,"41":1,"105":1,"204":1,"305":1,"172":1,"161":1}},{"filename":"com/saucelabs/ci/CacheTimeUtil.java","total":75,"coverage":{"34":1,"37":1,"24":1,"25":1,"31":1,"20":1}},{"filename":"com/saucelabs/ci/SeleniumVersion.java","total":0,"coverage":{}},{"filename":"com/saucelabs/ci/BrowserFactory.java","total":65,"coverage":{"205":1,"45":1,"34":1,"67":1,"169":1,"120":1,"93":1,"158":1,"142":1,"147":1,"216":1,"164":1,"170":1,"175":1,"104":1,"201":1,"33":1,"197":1,"186":1,"212":1,"163":1,"134":1,"123":1,"166":1,"107":1,"46":1,"206":1,"177":1,"209":1,"94":1,"83":1,"35":1,"79":1,"152":1,"103":1,"112":1,"199":1,"159":1,"59":1,"87":1,"76":1,"138":1,"71":1,"202":1,"185":1,"106":1,"137":1,"49":1,"36":1,"140":1,"180":1,"221":1,"60":1,"208":1,"111":1,"102":1,"96":1,"69":1,"162":1,"191":1,"151":1,"95":1,"184":1,"64":1,"53":1,"203":1,"115":1,"195":1,"156":1,"149":1,"178":1,"27":1,"210":1,"81":1,"187":1,"171":1,"92":1,"101":1,"154":1,"220":1,"143":1,"97":1,"179":1,"218":1,"222":1,"105":1,"211":1,"150":1,"91":1,"52":1,"172":1,"200":1,"119":1,"136":1}},{"filename":"com/saucelabs/ci/SODSeleniumConfiguration.java","total":0,"coverage":{}},{"filename":"com/saucelabs/ci/SeleniumBuilderManager.java","total":0,"coverage":{}},{"filename":"com/saucelabs/ci/sauceconnect/TunnelInformation.java","total":87,"coverage":{"45":1,"12":1,"33":1,"50":1,"37":1,"13":1,"46":1,"16":1,"21":1,"17":1,"38":1,"18":1,"30":1,"29":1}},{"filename":"com/saucelabs/ci/sauceconnect/SauceConnectFourManager.java","total":67,"coverage":{"45":1,"193":1,"302":1,"335":1,"223":1,"89":1,"51":1,"73":1,"164":1,"78":1,"324":1,"362":1,"40":1,"320":1,"44":1,"197":1,"280":1,"287":1,"291":1,"262":1,"186":1,"56":1,"329":1,"55":1,"303":1,"244":1,"37":1,"242":1,"107":1,"99":1,"155":1,"314":1,"79":1,"350":1,"103":1,"72":1,"330":1,"87":1,"263":1,"340":1,"196":1,"48":1,"281":1,"76":1,"347":1,"259":1,"202":1,"248":1,"185":1,"49":1,"241":1,"36":1,"300":1,"337":1,"39":1,"289":1,"267":1,"290":1,"25":1,"111":1,"344":1,"191":1,"245":1,"190":1,"322":1,"315":1,"53":1,"203":1,"75":1,"115":1,"156":1,"293":1,"357":1,"260":1,"27":1,"70":1,"210":1,"38":1,"81":1,"271":1,"304":1,"187":1,"165":1,"92":1,"261":1,"220":1,"30":1,"331":1,"222":1,"207":1,"288":1,"29":1,"41":1,"240":1,"211":1,"204":1,"342":1,"74":1,"243":1,"305":1,"131":1,"28":1,"301":1,"119":1,"239":1}},{"filename":"com/saucelabs/ci/sauceconnect/AbstractSauceTunnelManager.java","total":65,"coverage":{"619":1,"113":1,"381":1,"169":1,"444":1,"120":1,"335":1,"182":1,"147":1,"589":1,"608":1,"124":1,"411":1,"501":1,"519":1,"276":1,"164":1,"324":1,"332":1,"362":1,"552":1,"321":1,"580":1,"121":1,"415":1,"40":1,"593":1,"318":1,"378":1,"110":1,"284":1,"549":1,"385":1,"578":1,"273":1,"497":1,"280":1,"454":1,"585":1,"186":1,"227":1,"329":1,"55":1,"212":1,"512":1,"163":1,"123":1,"174":1,"410":1,"382":1,"166":1,"336":1,"328":1,"509":1,"46":1,"325":1,"437":1,"441":1,"209":1,"609":1,"379":1,"426":1,"103":1,"112":1,"387":1,"500":1,"330":1,"159":1,"631":1,"59":1,"550":1,"196":1,"375":1,"281":1,"572":1,"116":1,"498":1,"54":1,"511":1,"188":1,"148":1,"508":1,"213":1,"285":1,"185":1,"434":1,"360":1,"106":1,"431":1,"413":1,"386":1,"289":1,"582":1,"394":1,"290":1,"208":1,"122":1,"111":1,"102":1,"442":1,"191":1,"151":1,"322":1,"58":1,"439":1,"225":1,"53":1,"42":1,"528":1,"499":1,"293":1,"109":1,"189":1,"326":1,"210":1,"575":1,"564":1,"167":1,"277":1,"160":1,"187":1,"395":1,"165":1,"443":1,"573":1,"412":1,"520":1,"438":1,"319":1,"551":1,"194":1,"215":1,"30":1,"535":1,"331":1,"618":1,"432":1,"359":1,"607":1,"485":1,"384":1,"211":1,"150":1,"323":1,"226":1,"453":1,"172":1,"406":1,"380":1,"566":1,"440":1,"377":1,"334":1,"229":1,"119":1,"576":1,"632":1}}]}]"
[DEBUG] http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
[DEBUG] http-outgoing-0 << "Content-Type: application/json; charset=utf-8[\r][\n]"
[DEBUG] http-outgoing-0 << "Date: Tue, 30 Aug 2016 16:13:14 GMT[\r][\n]"
[DEBUG] http-outgoing-0 << "Request-Time: 3[\r][\n]"
[DEBUG] http-outgoing-0 << "Server: nginx/1.6.2[\r][\n]"
[DEBUG] http-outgoing-0 << "Strict-Transport-Security: max-age=31536000[\r][\n]"
[DEBUG] http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
[DEBUG] http-outgoing-0 << "X-Frame-Options: DENY[\r][\n]"
[DEBUG] http-outgoing-0 << "X-Permitted-Cross-Domain-Policies: master-only[\r][\n]"
[DEBUG] http-outgoing-0 << "X-XSS-Protection: 1; mode=block[\r][\n]"
[DEBUG] http-outgoing-0 << "Content-Length: 36[\r][\n]"
[DEBUG] http-outgoing-0 << "Connection: keep-alive[\r][\n]"
[DEBUG] http-outgoing-0 << "[\r][\n]"
[DEBUG] http-outgoing-0 << "{"error":"could not get json body."}"
[DEBUG] http-outgoing-0 << HTTP/1.1 200 OK
[DEBUG] http-outgoing-0 << Content-Type: application/json; charset=utf-8
[DEBUG] http-outgoing-0 << Date: Tue, 30 Aug 2016 16:13:14 GMT
[DEBUG] http-outgoing-0 << Request-Time: 3
[DEBUG] http-outgoing-0 << Server: nginx/1.6.2
[DEBUG] http-outgoing-0 << Strict-Transport-Security: max-age=31536000
[DEBUG] http-outgoing-0 << X-Content-Type-Options: nosniff
[DEBUG] http-outgoing-0 << X-Frame-Options: DENY
[DEBUG] http-outgoing-0 << X-Permitted-Cross-Domain-Policies: master-only
[DEBUG] http-outgoing-0 << X-XSS-Protection: 1; mode=block
[DEBUG] http-outgoing-0 << Content-Length: 36
[DEBUG] http-outgoing-0 << Connection: keep-alive
[DEBUG] Connection can be kept alive indefinitely
[DEBUG] Connection [id: 0][route: {s}->https://api.codacy.com:443] can be kept alive indefinitely
[DEBUG] Connection released: [id: 0][route: {s}->https://api.codacy.com:443][total kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
[INFO] ----------------------------------------
[INFO] {"error":"could not get json body."}
[INFO] Coverage data uploaded. Reason: {"error":"could not get json body."}
[DEBUG] Connection manager is shutting down
[DEBUG] http-outgoing-0: Close connection
[DEBUG] http-outgoing-0: Close connection
[DEBUG] Connection manager shut down

[mrfyda]

Hey @halkeye

The only thing I can think of is that you're wrapping the coverage report object in an array.

[{"language":"Java","total":51,"fileReports":[...]}] instead of {"language":"Java","total":51,"fileReports":[...]}

[halkeye]

Sweeeet. Thanks, it was a slight bug in the hack I did.

I'm going to have to refactor a bunch. I would love if one of the codacy api's could be updated to return the json object, 'cause java and scala doesn't want to play nice in that regard. But it is uploading.

 $ mvn com.gavinmogan:codacy-maven-plugin:1.0-SNAPSHOT:coverage -Dprefix="" -DcoverageReport=target/site/jacoco/jacoco.xml -Dcommit=a66d87c1031c375336fa3cb414245b02c2476da6
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 1 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 1 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ci-sauce 1.121-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- codacy-maven-plugin:1.0-SNAPSHOT:coverage (default-cli) @ ci-sauce ---
[INFO] Parsing coverage data... /home/halkeye/git/sauce/ci-sauce/target/site/jacoco/jacoco.xml
[INFO] Uploading coverage data...
[INFO] Executing request POST https://api.codacy.com/2.0/coverage/a66d87c1031c375336fa3cb414245b02c2476da6/java HTTP/1.1
[INFO] ----------------------------------------
[INFO] {"success":"Coverage received successfully."}
[INFO] Coverage data uploaded. Reason: {"success":"Coverage received successfully."}
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.132 s
[INFO] Finished at: 2016-08-30T10:26:24-07:00
[INFO] Final Memory: 23M/428M
[INFO] ------------------------------------------------------------------------

Any idea how long it should take to show something @ https://www.codacy.com/app/halkeye/ci-sauce/dashboard?bid=3528956

[mrfyda]

Great!

It shouldn't take more than a couple seconds as long as the commit has already been detected by Codacy.

Just make sure you are using the right project token.

[halkeye]

oh sweet! it showed up.

Does codacy have any interest in owning this long term? or should I try to eventually release this myself?

[pedrorijo91]

at this moment is hard for us to own/maintain another project (we already have so many open source projects, and our team is not very big...), but we are available to help you maintain the project. If you need help to release just let us know and we will help you

[halkeye]

https://halkeye.github.io/codacy-maven-plugin/project-summary.html https://oss.sonatype.org/content/groups/public/com/gavinmogan/codacy-maven-plugin/1.0.0/

Just waiting for maven central to update, then i'll update readmes and stuff

[halkeye]

mvn com.gavinmogan:codacy-maven-plugin:coverage -DcoverageReportFile=target/site/jacoco/jacoco.xml -DprojectToken=blah -DapiToken=blah

yep yep yep