search

jepsen-io / jepsen

A framework for distributed systems verification, with fault injection
6.82k stars 717 forks source link

gnuplot: argument list too long #153

Open johanatan opened 8 years ago

johanatan commented 8 years ago

Hi,

When a long running test finishes, I receive the following crash from gnuplot:

Caused by: java.io.IOException: error=7, Argument list too long
 at java.lang.UNIXProcess.forkAndExec (UNIXProcess.java:-2)
    java.lang.UNIXProcess.<init> (UNIXProcess.java:248)
    java.lang.ProcessImpl.start (ProcessImpl.java:134)
    java.lang.ProcessBuilder.start (ProcessBuilder.java:1029)
    java.lang.Runtime.exec (Runtime.java:620)
    clojure.java.shell$sh.invokeStatic (shell.clj:113)
    clojure.java.shell$sh.doInvoke (shell.clj:79)
    clojure.lang.RestFn.invoke (RestFn.java:619)
    gnuplot.core$run_BANG_.invokeStatic (core.clj:66)
    gnuplot.core$run_BANG_.invoke (core.clj:56)
    gnuplot.core$raw_plot_BANG_.invokeStatic (core.clj:82)
    gnuplot.core$raw_plot_BANG_.invoke (core.clj:78)
    jepsen.checker.perf$point_graph_BANG_.invokeStatic (perf.clj:229)
    jepsen.checker.perf$point_graph_BANG_.invoke (perf.clj:220)
    jepsen.checker$latency_graph$reify__8015.check (checker.clj:293)
    jepsen.checker$compose$reify__8008$fn__8010.invoke (checker.clj:284)
    clojure.core$pmap$fn__6970$fn__6971.invoke (core.clj:6736)
    clojure.core$binding_conveyor_fn$fn__4676.invoke (core.clj:1938)
    clojure.lang.AFn.call (AFn.java:18)
    java.util.concurrent.FutureTask.run (FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    java.lang.Thread.run (Thread.java:745)

Steps to reproduce: 1-. Take an existing jepsen test and make its nemesis innocuous so that the analysis phase will complete relatively quickly once we increase the running time. 2-. Increase running time of the test to 10,000 seconds. 3-. Notice the above crash after the run & analysis is complete.

aphyr commented 8 years ago

Iirc this is because nemesis events are drawn on the graph by a draw-rect call--might have to figure out how to render those from a stream like points, or omit them for long tests.