flosell / lambdacd

a library to define a continuous delivery pipeline in code
https://www.lambda.cd/
Apache License 2.0
676 stars 59 forks source link

GUI interface crashes while running on a RaspberryPi 2 #62

Closed andoneve closed 8 years ago

andoneve commented 8 years ago

We are running LambdaCD on a RaspberryPi 2 with an Archlinux setup.

We run some simple shell/bash commands (jshint, scss-lint) and then our deploy script (git clone, database dump).

It crashes rather often (every build or 2) when we keep the GUI interface open. Here is the stacktrace and screenshot:

[qtp2796335-87] WARN  o.e.j.server.AbstractHttpConnection - /
java.lang.ArrayIndexOutOfBoundsException:
at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:119) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:112) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at java.io.PrintWriter.write(PrintWriter.java:456) ~[na:1.7.0_85]
at java.io.PrintWriter.write(PrintWriter.java:473) ~[na:1.7.0_85]
at java.io.PrintWriter.print(PrintWriter.java:617) ~[na:1.7.0_85]
at ring.util.servlet$set_body.invoke(servlet.clj:87) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at ring.util.servlet$update_servlet_response.invoke(servlet.clj:112) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at ring.adapter.jetty$proxy_handler$fn__8638.invoke(jetty.clj:20) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.Server.handle(Server.java:363) ~[ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [ci-0.1.0-SNAPSHOT-standalone.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
screen shot 2015-10-13 at 12 19 23
flosell commented 8 years ago

Hi,

I haven't yet tested LambdaCD on a device like a RaspberryPi yet, maybe LambdaCD is just a bit wasteful at the moment. Which requests fail? The Ajax-Requests the UI is sending or access to the UI page itself?

flosell commented 8 years ago

I just tried digging into this a bit, unfortunately, I couldn't reproduce your issue yet. I've tried playing around with resources but it looks good as far as I can tell right now, even with very little RAM for example.

There are some things you could try to help me debug this:

Try the most recent ring and jetty versions

in project.clj, replace

[ring-server "0.3.1"]

with

[ring-server "0.4.0"]
[ring "1.4.0"]

You might need to run lein clean after you change the dependencies.

Get rid of Jetty As the stacktrace points in the direction of the Jetty webserver, could you try what happens if you swap the webserver with something like http-kit?

You could also try one of the other ring-compatible implementations, http-kit just looked easiest to integrate.

Check your charset I also looked into the Jetty source at the locations indicated by your stacktrace, there is some logic dependent on character encoding. What charset are you using?

Get me an exact list of your dependencies Easiest way is to call lein deps :tree and paste the output into a Gist

andoneve commented 8 years ago

These are awesome tips!

For now we upgraded our jdk from 1.7 to 1.8 on the pi and it seems to be stable so far... we will let you know if we have anymore problems. If it's stable for more than a few days we'll close the issue.

flosell commented 8 years ago

ok, thanks for the feedback!

andoneve commented 8 years ago

It's been stable after upgrading to jdk 1.8. Closing the issue.