jepsen-io / jepsen

A framework for distributed systems verification, with fault injection
6.69k stars 710 forks source link

java.util.concurrent.ExecutionException: java.io.IOException: Cannot run program "dot": error=2, No such file or directory #469

Open winddd opened 4 years ago

winddd commented 4 years ago

I am trying to run the stolon test and it can complete all the read and append transactions. But during analyzing period, one problem occurs. Here is the stack trace:

2020-07-08 16:02:33,588{GMT}    INFO    [jepsen test runner] jepsen.core: Run complete, writing
2020-07-08 16:02:34,670{GMT}    INFO    [jepsen test runner] jepsen.core: Analyzing...
2020-07-08 16:02:35,500{GMT}    WARN    [clojure-agent-send-off-pool-47] jepsen.checker: Error while checking history:
java.util.concurrent.ExecutionException: java.io.IOException: Cannot run program "dot": error=2, No such file or directory
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at clojure.core$deref_future.invokeStatic(core.clj:2300)
    at clojure.core$future_call$reify__8439.deref(core.clj:6974)
    at clojure.core$deref.invokeStatic(core.clj:2320)
    at clojure.core$deref.invoke(core.clj:2306)
    at clojure.core$map$fn__5851.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:58)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$dorun.invokeStatic(core.clj:3133)
    at clojure.core$dorun.invoke(core.clj:3133)
    at elle.viz$plot_analysis_BANG_.invokeStatic(viz.clj:203)
    at elle.viz$plot_analysis_BANG_.invoke(viz.clj:188)
    at elle.txn$cycles_BANG_.invokeStatic(txn.clj:514)
    at elle.txn$cycles_BANG_.invoke(txn.clj:495)
    at elle.list_append$check.invokeStatic(list_append.clj:777)
    at elle.list_append$check.invoke(list_append.clj:739)
    at jepsen.tests.cycle.append$checker$reify__1759.check(append.clj:19)
    at jepsen.checker$check_safe.invokeStatic(checker.clj:78)
    at jepsen.checker$check_safe.invoke(checker.clj:71)
    at jepsen.checker$compose$reify__8566$fn__8568.invoke(checker.clj:94)
    at clojure.core$pmap$fn__8447$fn__8448.invoke(core.clj:7022)
    at clojure.core$binding_conveyor_fn$fn__5739.invoke(core.clj:2030)
    at clojure.lang.AFn.call(AFn.java:18)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Cannot run program "dot": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at java.lang.Runtime.exec(Runtime.java:620)
    at clojure.java.shell$sh.invokeStatic(shell.clj:113)
    at clojure.java.shell$sh.doInvoke(shell.clj:79)
    at clojure.lang.RestFn.invoke(RestFn.java:457)
    at rhizome.viz$dot__GT_svg.invokeStatic(viz.clj:130)
    at rhizome.viz$dot__GT_svg.invoke(viz.clj:126)
    at elle.viz$save_dot_BANG_.invokeStatic(viz.clj:186)
    at elle.viz$save_dot_BANG_.invoke(viz.clj:176)
    at elle.viz$plot_analysis_BANG_$fn__1293.invoke(viz.clj:209)
    ... 7 common frames omitted
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 16 common frames omitted
2020-07-08 16:02:35,507{GMT}    INFO    [jepsen test runner] jepsen.core: Analysis complete

OS of control node: Debian 9.
OS of db node: Debian 10. gnuplot version: gnuplot 5.0 patchlevel 5.

aphyr commented 4 years ago

Like the error message says, you don't have dot installed--you'll need graphviz for Jepsen to render the anomalies.

winddd commented 4 years ago

wow, I didn't realize that dot is a library. Thank you!

aphyr commented 4 years ago

In your defense, the documentation for this is basically nonexistent! That's on me, sorry!

winddd commented 4 years ago

In your defense, the documentation for this is basically nonexistent! That's on me, sorry!

I have successfully reproduced the phenomenon of G2-item after several days' try. I am so excited! Thanks a lot for your help!

winddd commented 4 years ago

In your defense, the documentation for this is basically nonexistent! That's on me, sorry!

ah, I learnt the Jepsen from the etcd tutorial and it is based on the old version(may be before 0.18.0). It seems that Jepsen underwent a big refactoring/update after that. By the way, wish to see the updated tutorial for the redesigned Jepsen if possible.

aphyr commented 4 years ago

Yeah, that would be great. I'm... still trying to summon up the wherewithal to write any kind of code this week; when I get a free 10 hours or so I'll go through the tutorial again.