ConcurrentGenerator: IllegalArgumentException: Key must be integer

[kbr-]

Jepsen 0.2.0

reproduce with the following test:

(ns jepsen-bug.run
  (:require [jepsen [cli :as cli] [client :as client] [independent :as independent] [tests :as tests]]))

(defrecord Client []
  client/Client
  (open! [this _ _] this)
  (setup! [_ _])
  (invoke! [_ _ op] (assoc op :type :info))
  (teardown! [_ _])
  (close! [_ _]))

(defn test [opts]
  (merge tests/noop-test opts
         {:pure-generators true
          :client (Client.)
          :generator (->> (independent/concurrent-generator 1 (range) (fn [_] {:type :invoke})))}))

(defn -main [& args]
  (cli/run! (cli/single-test-cmd {:test-fn test}) args))

I debugged it, here's what happens:

1. client completes operation with :type :info, let's say the process was X
2. interpreter associates new process to the thread which invoked that op:

               ; Workers that crash (other than the nemesis) should be assigned
               ; new thread identifiers.
               ctx     (if (or (= :nemesis thread) (not= :info (:type op')))
                         ctx
                         (update ctx :workers assoc thread
                                 (gen/next-process ctx thread)))

3. ConcurrentGenerator's update takes the completion event, it has process X, tries to obtain the thread of process X, but there is no such thread anymore:

   
   Context:                                                                                                                                     

{... :workers {0 1, :nemesis :nemesis}}
Event:

{:type :info, :time 52925744, :process 0, :value [0 nil]}

therefore `update` obtains `thread == nil`:

(update [this test ctx event] (let [process (:process event) thread (gen/process->thread ctx process) group (thread->group thread)] (ConcurrentGenerator. n fgen group->threads thread->group keys (update gens group gen/update test ctx event)))))

4. since `thread` is `nil` above, `group` is also `nil`, so we try to modify the `gens` vector under index `nil`
5.

java.lang.IllegalArgumentException: Key must be integer
at clojure.lang.APersistentVector.assoc(APersistentVector.java:347)
at clojure.lang.APersistentVector.assoc(APersistentVector.java:18)
at clojure.lang.RT.assoc(RT.java:827)
at clojure.core$assoc__5416.invokeStatic(core.clj:191)
at clojure.core$update.invokeStatic(core.clj:6202)
at clojure.core$update.invoke(core.clj:6188)
at jepsen.independent.ConcurrentGenerator.update(independent.clj:209)

[aphyr]

I believe this is https://github.com/jepsen-io/jepsen/issues/485, which is fixed in 0.2.1-SNAPSHOT. :-)

[kbr-]

Oh, nice, thanks, google didn't catch it. Guess we can close.

[aphyr]

Sorry bout that! I did a lot of testing work prior to 0.2.0, but this one snuck through. There's also an issue in stagger which is fixed in 0.2.1.