search

uber / uberalls

Track code coverage metrics with Jenkins and Phabricator
MIT License
186 stars 14 forks source link

Proper setup guide? #21

Open connorwang opened 8 years ago

connorwang commented 8 years ago

So I've been trying to set up this, my steps as follow

go get github.com/uber/uberalls
cd $GOPATH/src/github.com/uber/uberalls
go get github.com/tools/godep
godep restore
go build && ./uberalls

The app starts successfully and listens to 14740 and use sqlite in this case. Then I set the location of the uberalls instance to a local ip address, let's say 192.168.1.100:14740 in Jenkins, and did a baseline run building on the trunk (master in this case). Up til this setp everyting works fine and Jenkins sends non-diff to phabricator (e.g. [phabricator:non-differential] Sending coverage result).

But when I started another differential build, I got the following errors:

java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at java.net.Socket.connect(Socket.java:538)
    at java.net.Socket.<init>(Socket.java:434)
    at java.net.Socket.<init>(Socket.java:211)
    at hudson.util.NoClientBindProtocolSocketFactory.createSocket(NoClientBindProtocolSocketFactory.java:105)
    at hudson.util.NoClientBindProtocolSocketFactory.createSocket(NoClientBindProtocolSocketFactory.java:90)
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
    at com.uber.jenkins.phabricator.uberalls.UberallsClient.getCoverage(UberallsClient.java:150)
    at com.uber.jenkins.phabricator.uberalls.UberallsClient.getParentCoverage(UberallsClient.java:75)
    at com.uber.jenkins.phabricator.BuildResultProcessor.processParentCoverage(BuildResultProcessor.java:85)
    at com.uber.jenkins.phabricator.PhabricatorNotifier.perform(PhabricatorNotifier.java:159)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
    at hudson.model.Build$BuildExecution.post2(Build.java:185)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
    at hudson.model.Run.execute(Run.java:1763)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:98)
    at hudson.model.Executor.run(Executor.java:410)
[phabricator:uberalls] unable to find coverage for parent commit
[phabricator:process-build-result] No unit results available.

I've also tried different protocals for the location instance, e.g. with and without http://, with or without port number, but so far no luck.

Could you shed some light on this?

ascandella commented 8 years ago

That is very strange. What version of Jenkins and the JRE are you running, and what version of the plugin are you on?

connorwang commented 8 years ago

I'm on openjdk-8 and Jenkins 1.643. The plugin is on 1.8.3. Firstly I thought it was due to no service listening to that port (14740), then I double checked:

coverage@coverage-server:~/.gvm/pkgsets/go1.4/global/src/github.com/uber/uberalls$ ./uberalls
2016/01/06 17:04:07 Configuring...
2016/01/06 17:04:07 Loading configuration from 'config/default.json'
2016/01/06 17:04:07 Listening on 127.0.0.1:14740...

Just to be clear, what exactly should I set for the instance location? e.g.http://192.168.1.110:14740?

ascandella commented 8 years ago

Try changing the listenAddress option in config/default.json from 127.0.0.1 to 0.0.0.0 (or the 192. address)

connorwang commented 8 years ago

So if I don't put http:// in front of the ip address for instance location, I get the following error 

java.net.URISyntaxException: Illegal character in scheme name at index 0: 192.168.1.122:14740

If I do, I get 

[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

I also tried useing it with mysql so the listening address was 0.0.0.0, so I don't think that's the cause.

ascandella commented 8 years ago

Based on the output you've pasted above, it's only listening on the loop back address meaning connection refused is to be expected. Can you switch your uberalls bind port and restart the process?

And yes, you do need http:// On Wed, Jan 6, 2016 at 5:38 PM Connor Wang notifications@github.com wrote:

So if I don't put http:// in front of the ip address for instance location, I get the following error

java.net.URISyntaxException: Illegal character in scheme name at index 0: 192.168.1.122:14740

If I do, I get

[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

I also tried useing it with mysql so the listening address was 0.0.0.0, so I don't think that's the cause.

— Reply to this email directly or view it on GitHub https://github.com/uber/uberalls/issues/21#issuecomment-169519196.

connorwang commented 8 years ago

I've just tried your suggestions and change the bind port to another, but still getting 

[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

This might be a silly question - do i need set up something else (e.g. a webserver) along with this?

ascandella commented 8 years ago

Ah, the 404 is expected, it means there is no coverage data available. You don't need to set anything else up. Once you have coverage data populated (via SCM polling) you will see 200 responses. On Wed, Jan 6, 2016 at 6:10 PM Connor Wang notifications@github.com wrote:

I've just tried your suggestions and change the bind port to another, but still getting

[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

This might be a silly question - do i need set up something else (e.g. a webserver) along with this?

— Reply to this email directly or view it on GitHub https://github.com/uber/uberalls/issues/21#issuecomment-169523876.

connorwang commented 8 years ago

Alright I'll give it a try and get back to you. Thanks!

connorwang commented 8 years ago

Also there is a duplicated key error when using either sqlite or mysql.

(Error 1061: Duplicate key name 'idx_metrics_repository_sha_timestamp')
(Error 1061: Duplicate key name 'idx_metrics_repository_branch_timestamp')

It also keeps getting 404 errors no matter how many polling builds or direct non-diff builds on master - so looks like connection is not the issue as it's getting responses back, but I don't think it's posting anything to sqlite/mysql.

[phabricator:non-differential] Sending coverage result for 0ab413350d124799b0ee89c28905f5d77e07e8da as package coverage = 100.0, files coverage = 100.0, classes coverage = 100.0, method coverage = 42.592594, line coverage = 50.8, conditional coverage = 33.333332
[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

Query mysql tables after several builds (both direct and polling) gives 

+--------------------+
| Tables_in_coverage |
+--------------------+
| metrics            |
+--------------------+
1 row in set (0.00 sec)
ascandella commented 8 years ago

Interesting, a 404 should not happen on sending coverage data unless the server isn't listening properly. You're running it directly, correct (no nginx/apache/varnish doing reverse proxying)?

connorwang commented 8 years ago

Yea that's what I mentioned above (service isn't listening), but I double checked and made sure the uberalls server is up and listening to 8080.

One thing to mention is that the our phabricator, jenkins and the coverage server all run in within the same local network but phabricator and jenkins have their own public https url. Could that be causing the problem?

connorwang commented 8 years ago

I just ran netstat -tulpn on the coverage server and turned out uberalls is listening on 8080 via tcp6. Could that be the problem?

ascandella commented 8 years ago

Potentially, yea, although I'm a bit mystified how that would be the case if you specified an IPv4 listen address.

connorwang commented 8 years ago

Well looks like it's not because of the tcp6 lol. I've done some additional dubugging with following steps.

First I check with the connection bewteen uberalls and mysql, which seems working properly

37 Connect   coverage@localhost on coverage
                   37 Query     SELECT @@max_allowed_packet
                   37 Query     SET NAMES utf8
                   37 Prepare   SELECT count(*) FROM INFORMATION_SCHEMA.tables where table_name = ? AND table_schema = ?
                   37 Execute   SELECT count(*) FROM INFORMATION_SCHEMA.tables where table_name = 'metrics' AND table_schema = 'coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'method_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'line_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'conditional_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'sha'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'package_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'branch'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'files_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'classes_coverage'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'timestamp'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'i_d'
                   37 Close stmt
                   37 Prepare   SELECT count(*) FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name = ?
                   37 Execute   SELECT count(*) FROM information_schema.columns WHERE table_schema = 'coverage' AND table_name = 'metrics' AND column_name = 'repository'
                   37 Close stmt
                   37 Query     CREATE INDEX idx_metrics_repository_sha_timestamp ON `metrics`(`repository`, `sha`, `timestamp`)
                   37 Query     CREATE INDEX idx_metrics_repository_branch_timestamp ON `metrics`(`repository`, `branch`, `timestamp`)

Then I run tcptrack on uberalls server (192.168.1.122) monitoring port 8080, I can see jenkin's instance's incoming traffic from 192.168.1.138 capture

That leaves me thinking either uberalls is not processing the request correctly or Jenkin's phabricator plugin sends malformed request to uberalls.

Also, is this normal?

(Error 1061: Duplicate key name 'idx_metrics_repository_sha_timestamp')
[2016-01-08 23:15:38]
2016/01/08 23:15:38 Listening on 0.0.0.0:8080...

(Error 1061: Duplicate key name 'idx_metrics_repository_branch_timestamp')
[2016-01-08 23:15:38]

Any thoughts?

aadis commented 7 years ago

I see the same in the log

2017/01/21 21:04:29 Listening on 0.0.0.0:8090...

(Error 1061: Duplicate key name 'idx_metrics_repository_branch_timestamp')
[2017-01-21 21:04:29]

Still setting up jenkins jobs, will update how it goes.

rphilipsen commented 7 years ago

I have exactly the same problem. It also shows the same message in the log and Jenkins keep giving `

[Cobertura] Publishing Cobertura coverage report...
Publishing Cobertura coverage results...
Cobertura coverage report found.
Recording test results
[phabricator:non-differential] Sending coverage result for 601e6de8b75098547eecf2417653c276f097b8a8 as package coverage = 100.0, files coverage = 100.0, classes coverage = 100.0, method coverage = 0.0, line coverage = 79.35551, conditional coverage = 44.31818
[phabricator:uberalls-client] Call failed: HTTP/1.1 404 Not Found

I did run the build first via SCM polling, and have configured the uberalls url in Jenkins to 192.168.0.108:14741

I can go to 192.168.0.108:14741/health, giving me ;-). Is there any solution on this matter?

maroux commented 6 years ago

I'm also seeing Connection refused in the DIFF builds. Oddly enough, CI builds work fine with result

[phabricator:non-differential] Sending coverage result for b6951531c277488711dbf0a669434e54835c2396 as package coverage = 88.888885, files coverage = 93.98496, classes coverage = 93.98496, method coverage = 0.0, line coverage = 94.08822, conditional coverage = 81.212845
maroux commented 6 years ago

Resolved my problem -

  1. Uberalls is supposed to be run on the master node in master-slave setup
  2. Phab jenkins plugin was suppressing errors in CI builds so it wasn't showing connection refused (fix uber/phabricator-jenkins-plugin#260)
maroux commented 6 years ago

@rphilipsen in case you haven't resolved your problem yet, I believe you need to set Uberalls url to http://localhost:<port>/metrics, not just / - which again goes back to bad setup guide :)

maroux commented 6 years ago

IMO uber/phabricator-jenkins-plugin#256 needs to be fixed as well for this to be considered stable. Running the job on a cron is useless and shouldn't be required.