oakes / play-clj

A Clojure game library
The Unlicense
942 stars 76 forks source link

Testing strategy #30

Closed Misophistful closed 10 years ago

Misophistful commented 10 years ago

I'm trying to figure out the best way to write tests for main game functionality, i.e. for functions that manipulate entities, rather than isolated helper functions.

I create a simplified list of entities that I pass into the function I'm testing, and I want to check that the expected game state is returned. But I get the following exception on running the test:

java.lang.NullPointerException: null
        ShaderProgram.java:196 com.badlogic.gdx.graphics.glutils.ShaderProgram.loadShader
        ShaderProgram.java:178 com.badlogic.gdx.graphics.glutils.ShaderProgram.compileShaders
        ShaderProgram.java:161 com.badlogic.gdx.graphics.glutils.ShaderProgram.<init>
ImmediateModeRenderer20.java:219 com.badlogic.gdx.graphics.glutils.ImmediateModeRenderer20.createDefaultShader
ImmediateModeRenderer20.java:55 com.badlogic.gdx.graphics.glutils.ImmediateModeRenderer20.<init>
        ShapeRenderer.java:114 com.badlogic.gdx.graphics.glutils.ShapeRenderer.<init>
        ShapeRenderer.java:110 com.badlogic.gdx.graphics.glutils.ShapeRenderer.<init>
          core_graphics.clj:34 play-clj.core/shape*
/Users/jamtru/Projects/clojure/elemental/desktop/src-common/elemental/hexagon.clj:22 elemental.hexagon/create-hexagon
/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:15 elemental.movement-test/fn
                 core.clj:2557 clojure.core/map[fn]
               LazySeq.java:40 clojure.lang.LazySeq.sval
               LazySeq.java:49 clojure.lang.LazySeq.seq
                   RT.java:484 clojure.lang.RT.seq
                  core.clj:133 clojure.core/seq
             core_print.clj:46 clojure.core/print-sequential
            core_print.clj:147 clojure.core/fn
              MultiFn.java:231 clojure.lang.MultiFn.invoke
                 core.clj:3392 clojure.core/pr-on
             core_print.clj:53 clojure.core/print-sequential
            core_print.clj:198 clojure.core/fn
              MultiFn.java:231 clojure.lang.MultiFn.invoke
                 core.clj:3392 clojure.core/pr-on
                 core.clj:3404 clojure.core/pr
                  AFn.java:154 clojure.lang.AFn.applyToHelper
               RestFn.java:132 clojure.lang.RestFn.applyTo
                  core.clj:624 clojure.core/apply
                 core.clj:4373 clojure.core/pr-str
               RestFn.java:408 clojure.lang.RestFn.invoke
                   eval.clj:69 lighttable.nrepl.eval/clean-serialize
               RestFn.java:423 clojure.lang.RestFn.invoke
                   eval.clj:82 lighttable.nrepl.eval/->result
                  AFn.java:156 clojure.lang.AFn.applyToHelper
                  AFn.java:144 clojure.lang.AFn.applyTo
                  core.clj:626 clojure.core/apply
                 core.clj:2468 clojure.core/partial[fn]
               RestFn.java:408 clojure.lang.RestFn.invoke
                 core.clj:2559 clojure.core/map[fn]
               LazySeq.java:40 clojure.lang.LazySeq.sval
               LazySeq.java:49 clojure.lang.LazySeq.seq
                   RT.java:484 clojure.lang.RT.seq
                  core.clj:133 clojure.core/seq
                 core.clj:2595 clojure.core/filter[fn]
               LazySeq.java:40 clojure.lang.LazySeq.sval
               LazySeq.java:49 clojure.lang.LazySeq.seq
                  Cons.java:39 clojure.lang.Cons.next
                   RT.java:598 clojure.lang.RT.next
                   core.clj:64 clojure.core/next
                 core.clj:2856 clojure.core/dorun
                 core.clj:2871 clojure.core/doall
                  eval.clj:126 lighttable.nrepl.eval/eval-clj
               RestFn.java:442 clojure.lang.RestFn.invoke
                  eval.clj:192 lighttable.nrepl.eval/eval2659[fn]
                  AFn.java:152 clojure.lang.AFn.applyToHelper
                  AFn.java:144 clojure.lang.AFn.applyTo
                  core.clj:624 clojure.core/apply
                 core.clj:1862 clojure.core/with-bindings*
               RestFn.java:425 clojure.lang.RestFn.invoke
                  eval.clj:177 lighttable.nrepl.eval/eval2659[fn]
                  eval.clj:176 lighttable.nrepl.eval/eval2659[fn]
              MultiFn.java:227 clojure.lang.MultiFn.invoke
                   core.clj:98 lighttable.nrepl.core/queued[fn]
                 core.clj:2402 clojure.core/comp[fn]
    interruptible_eval.clj:138 clojure.tools.nrepl.middleware.interruptible-eval/run-next[fn]
                   AFn.java:22 clojure.lang.AFn.run
   ThreadPoolExecutor.java:895 java.util.concurrent.ThreadPoolExecutor$Worker.runTask
   ThreadPoolExecutor.java:918 java.util.concurrent.ThreadPoolExecutor$Worker.run
               Thread.java:695 java.lang.Thread.run

I'm guessing it's because the test is outside of the openGL context of the game?

Is there a strategy I can use to get around this? Perhaps defscreening a test-screen? Some other approach?

oakes commented 10 years ago

Not having the full game set up could be problematic, but the error here doesn't necessarily indicate that. How are you initializing the shape? Normally if it wanted to be in an OpenGL context, it would explicitly say that in the error.

Misophistful commented 10 years ago

Interestingly, I just noticed that when I evaluate the test without having the game launched I get the exception I posted above, and when I evaluate the test with the game launched I get the following No OpenGL context found exception.

clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: No OpenGL context found in the current thread., compiling:(/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:17:35)
               Compiler.java:3558 clojure.lang.Compiler$InvokeExpr.eval
               Compiler.java:3552 clojure.lang.Compiler$InvokeExpr.eval
                Compiler.java:417 clojure.lang.Compiler$DefExpr.eval
               Compiler.java:6708 clojure.lang.Compiler.eval
               Compiler.java:6666 clojure.lang.Compiler.eval
                    core.clj:2927 clojure.core/eval
                      eval.clj:77 lighttable.nrepl.eval/->result
                     AFn.java:156 clojure.lang.AFn.applyToHelper
                     AFn.java:144 clojure.lang.AFn.applyTo
                     core.clj:626 clojure.core/apply
                    core.clj:2468 clojure.core/partial[fn]
                  RestFn.java:408 clojure.lang.RestFn.invoke
                    core.clj:2559 clojure.core/map[fn]
                  LazySeq.java:40 clojure.lang.LazySeq.sval
                  LazySeq.java:49 clojure.lang.LazySeq.seq
                      RT.java:484 clojure.lang.RT.seq
                     core.clj:133 clojure.core/seq
                    core.clj:2595 clojure.core/filter[fn]
                  LazySeq.java:40 clojure.lang.LazySeq.sval
                  LazySeq.java:49 clojure.lang.LazySeq.seq
                     Cons.java:39 clojure.lang.Cons.next
                      RT.java:598 clojure.lang.RT.next
                      core.clj:64 clojure.core/next
                    core.clj:2856 clojure.core/dorun
                    core.clj:2871 clojure.core/doall
                     eval.clj:126 lighttable.nrepl.eval/eval-clj
                  RestFn.java:442 clojure.lang.RestFn.invoke
                     eval.clj:192 lighttable.nrepl.eval/eval2800[fn]
                     AFn.java:152 clojure.lang.AFn.applyToHelper
                     AFn.java:144 clojure.lang.AFn.applyTo
                     core.clj:624 clojure.core/apply
                    core.clj:1862 clojure.core/with-bindings*
                  RestFn.java:425 clojure.lang.RestFn.invoke
                     eval.clj:177 lighttable.nrepl.eval/eval2800[fn]
                     eval.clj:176 lighttable.nrepl.eval/eval2800[fn]
                 MultiFn.java:227 clojure.lang.MultiFn.invoke
                      core.clj:98 lighttable.nrepl.core/queued[fn]
                    core.clj:2402 clojure.core/comp[fn]
       interruptible_eval.clj:138 clojure.tools.nrepl.middleware.interruptible-eval/run-next[fn]
                      AFn.java:22 clojure.lang.AFn.run
      ThreadPoolExecutor.java:895 java.util.concurrent.ThreadPoolExecutor$Worker.runTask
      ThreadPoolExecutor.java:918 java.util.concurrent.ThreadPoolExecutor$Worker.run
                  Thread.java:695 java.lang.Thread.run
Caused by: java.lang.RuntimeException: No OpenGL context found in the current thread.
               GLContext.java:124 org.lwjgl.opengl.GLContext.getCapabilities
                    GL20.java:219 org.lwjgl.opengl.GL20.glCreateShader
               LwjglGL20.java:176 com.badlogic.gdx.backends.lwjgl.LwjglGL20.glCreateShader
           ShaderProgram.java:199 com.badlogic.gdx.graphics.glutils.ShaderProgram.loadShader
           ShaderProgram.java:178 com.badlogic.gdx.graphics.glutils.ShaderProgram.compileShaders
           ShaderProgram.java:161 com.badlogic.gdx.graphics.glutils.ShaderProgram.<init>
 ImmediateModeRenderer20.java:219 com.badlogic.gdx.graphics.glutils.ImmediateModeRenderer20.createDefaultShader
  ImmediateModeRenderer20.java:55 com.badlogic.gdx.graphics.glutils.ImmediateModeRenderer20.<init>
           ShapeRenderer.java:114 com.badlogic.gdx.graphics.glutils.ShapeRenderer.<init>
           ShapeRenderer.java:110 com.badlogic.gdx.graphics.glutils.ShapeRenderer.<init>
             core_graphics.clj:34 play-clj.core/shape*
                  elements.clj:14 elemental.elements/create-element-shape
                  elements.clj:25 elemental.elements/create-element
                     AFn.java:160 clojure.lang.AFn.applyToHelper
                     AFn.java:144 clojure.lang.AFn.applyTo
               Compiler.java:3553 clojure.lang.Compiler$InvokeExpr.eval

If it helps, here's my test code, which is calling my game's standard initialisation functions h/create-hexagon and e/create-element, both of which create shapes:

(ns elemental.movement-test
  (:require [play-clj.core :refer :all]
            [elemental.movement :refer :all]
            [elemental.elements :as e]
            [elemental.hexagon :as h]
            [expectations :refer [expect]]))

(def game-width 300)
(def game-height 300)

(def simple-board-config [   [-1 1] [1 0]
                          [-2 1] [0 0] [2 -1]
                             [-1 0] [1 -1]   ])
(def element-config {:player 1, :element :air, :coords [0 0]})

(def simple-board (map #(h/create-hexagon % game-width game-height) simple-board-config))
(def element-with-no-moves (assoc (e/create-element element-config game-width game-height)
                             :selected? true
                             :moves-remaining 0))
(def entities [simple-board element-with-no-moves])

;;; An element with 0 moves remaining shouldn't be moved
(expect entities (move-selected-element (first simple-board) entities))
oakes commented 10 years ago

Can you try wrapping your def statements (starting with simple-board) with the on-gl macro? It won't necessarily be pretty, but that should get them to run on the GL thread. The last line calling expect will also need to be in it.

Misophistful commented 10 years ago

Wrapping the GL dependent def statements with on-gl allows the test to evaluate and run, but only while the game is launched. If the game isn't launched I get the following exception:

java.lang.NullPointerException: null
/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:40 elemental.movement-test/eval7703
       Compiler.java:6703 clojure.lang.Compiler.eval
       Compiler.java:6666 clojure.lang.Compiler.eval
            core.clj:2927 clojure.core/eval
              eval.clj:77 lighttable.nrepl.eval/->result
             AFn.java:156 clojure.lang.AFn.applyToHelper
             AFn.java:144 clojure.lang.AFn.applyTo
             core.clj:626 clojure.core/apply
            core.clj:2468 clojure.core/partial[fn]
          RestFn.java:408 clojure.lang.RestFn.invoke
            core.clj:2559 clojure.core/map[fn]
          LazySeq.java:40 clojure.lang.LazySeq.sval
          LazySeq.java:49 clojure.lang.LazySeq.seq
              RT.java:484 clojure.lang.RT.seq
             core.clj:133 clojure.core/seq
            core.clj:2595 clojure.core/filter[fn]
          LazySeq.java:40 clojure.lang.LazySeq.sval
          LazySeq.java:49 clojure.lang.LazySeq.seq
             Cons.java:39 clojure.lang.Cons.next
              RT.java:598 clojure.lang.RT.next
              core.clj:64 clojure.core/next
            core.clj:2856 clojure.core/dorun
            core.clj:2871 clojure.core/doall
             eval.clj:126 lighttable.nrepl.eval/eval-clj
          RestFn.java:442 clojure.lang.RestFn.invoke
             eval.clj:192 lighttable.nrepl.eval/eval2800[fn]
             AFn.java:152 clojure.lang.AFn.applyToHelper
             AFn.java:144 clojure.lang.AFn.applyTo
             core.clj:624 clojure.core/apply
            core.clj:1862 clojure.core/with-bindings*
          RestFn.java:425 clojure.lang.RestFn.invoke
             eval.clj:177 lighttable.nrepl.eval/eval2800[fn]
             eval.clj:176 lighttable.nrepl.eval/eval2800[fn]
         MultiFn.java:227 clojure.lang.MultiFn.invoke
              core.clj:98 lighttable.nrepl.core/queued[fn]
            core.clj:2402 clojure.core/comp[fn]
interruptible_eval.clj:138 clojure.tools.nrepl.middleware.interruptible-eval/run-next[fn]
              AFn.java:22 clojure.lang.AFn.run
ThreadPoolExecutor.java:895 java.util.concurrent.ThreadPoolExecutor$Worker.runTask
ThreadPoolExecutor.java:918 java.util.concurrent.ThreadPoolExecutor$Worker.run
          Thread.java:695 java.lang.Thread.run

Unfortunately, even when the test runs I can't see the results using Light Table's Expectations plugin (I'm guessing this might be due to the on-gl wrapping confusing the plugin, but I haven't looked into it yet). I also can't run my tests from the command line using lein autoexpect as I get the same NullPointerException as when running the test without the game launched.

Is there a way to get the test compiling without the game running? If not, I fear that this current approach might not be a good match for the type of testing that I'd like to do. Perhaps in that case I should switch to creating test-specific versions of the board and element, which don't have shapes or labels in them? There are a few downsides to doing that: firstly, I'll have to maintain two creation functions for each game entity; one real and one test, secondly I'd be testing the game functions with faked entities, and finally I won't be able to test the real creation code at all. All of which seems like a high price to pay.

oakes commented 10 years ago

You could probably monkey patch play-clj directly rather than making test-specific versions of your entities. For example, if you put the following in the beginning of your test, it should cause anything using shape to use a plain hash map instead:

(intern 'play-clj.core 'shape* (fn [& args] {}))

oakes commented 10 years ago

The previous idea will interfere with your game if you're running your tests in the same process, though. To keep the monkey patching local, you might be able to do it with binding instead:

(binding [play-clj.core/shape* (fn [& args] {})]
  ; test code here
  )
Misophistful commented 10 years ago

binding the shapes to hash-maps sounds like it might be a decent compromise, but it gives me the following exception:

java.lang.IllegalStateException: Can't dynamically bind non-dynamic var: play-clj.core/shape*
             Var.java:320 clojure.lang.Var.pushThreadBindings
            core.clj:1809 clojure.core/push-thread-bindings
/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:16 elemental.movement-test/eval7894
       Compiler.java:6703 clojure.lang.Compiler.eval
       Compiler.java:6666 clojure.lang.Compiler.eval
            core.clj:2927 clojure.core/eval
              eval.clj:77 lighttable.nrepl.eval/->result
             AFn.java:156 clojure.lang.AFn.applyToHelper
             AFn.java:144 clojure.lang.AFn.applyTo
             core.clj:626 clojure.core/apply
            core.clj:2468 clojure.core/partial[fn]
           <snip>
oakes commented 10 years ago

Yeah I forgot about that limitation of binding. Can you try with-redefs? I think it works similarly:

(with-redefs [play-clj.core/shape* (fn [& args] {})]
  ; test code here
  )
Misophistful commented 10 years ago

with-redefs works better, but still falls over at the point my code is calling the shape macro:

java.lang.Exception: The keyword :object is not found.
             utils.clj:19 play-clj.utils/throw-key-not-found
             utils.clj:25 play-clj.utils/get-obj
          elements.clj:14 elemental.elements/create-element-shape
          elements.clj:25 elemental.elements/create-element
/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:20 elemental.movement-test/eval8099[fn]
            core.clj:6861 clojure.core/with-redefs-fn
/Users/jamtru/Projects/clojure/elemental/desktop/test/elemental/movement_test.clj:18 elemental.movement-test/eval8099
       Compiler.java:6703 clojure.lang.Compiler.eval
       Compiler.java:6666 clojure.lang.Compiler.eval
            core.clj:2927 clojure.core/eval
              eval.clj:77 lighttable.nrepl.eval/->result
             AFn.java:156 clojure.lang.AFn.applyToHelper
             AFn.java:144 clojure.lang.AFn.applyTo
             core.clj:626 clojure.core/apply
<snip>

Here's the code that failing:

(defn create-element-shape [radius colour angle]
  (let [[tri-x1 tri-y1] [0 (+ radius (/ radius 4))]
        [tri-x2 tri-y2] (cir/calculate-point-on-circle radius 125)
        [tri-x3 tri-y3] (cir/calculate-point-on-circle radius 55)]
    (shape :filled
           :set-color colour
           :circle 0 0 radius
           :triangle tri-x1 tri-y1 tri-x2 tri-y2 tri-x3 tri-y3)))
oakes commented 10 years ago

Looks like we just need to get more creative with the mock function. Instead of returning {}, try returning something like {:object (Object.)}. That should at least get shape working, though calls to shape! may fail.

Misophistful commented 10 years ago

That works, though it looks like I also need to redef label* too. I'll play with this some more later when I have some time. Thank you so much for your help so far.

oakes commented 10 years ago

Sure! I hope it works.

Misophistful commented 10 years ago

When I try:

(with-redefs [play-clj.core/shape* (fn [& args] {:object (Object.)})
              play-clj.ui/label* (fn [& args] {:object (Object.)})]

(def simple-board (map #(h/create-hexagon % game-width game-height) simple-board-config))
<snip>

I get:

Failed trying to require elemental.movement-test with: java.lang.ClassCastException: java.lang.Object cannot be cast to com.badlogic.gdx.scenes.scene2d.ui.Label
             elements.clj:27 elemental.elements/create-element
        movement_test.clj:22 elemental.movement-test/eval7673[fn]
               core.clj:6861 clojure.core/with-redefs-fn
        movement_test.clj:18 elemental.movement-test/eval7673
          Compiler.java:6703 clojure.lang.Compiler.eval
          Compiler.java:7130 clojure.lang.Compiler.load
<snip>

Do I need to construct an actual Label. and assign it to :object? If that's the case, why does just Object. work for shape*?

oakes commented 10 years ago

Darn, I guess it doesn't like it because it's trying to run Label methods on the object. I suppose shape works because the method calls don't actually run immediately; they're deferred until it's time to render. I think it's the only entity that works that way. I'll have to keep thinking about this.

oakes commented 10 years ago

Do you think it might be possible to start a separate game instance just for your game code? Maybe your test code could import your desktoplauncher.clj and manually run (-main). It will throw an exception if the game is already running, but if you catch errors it shouldn't be a problem: `(try (-main) (catch Exception ))`. Just a thought.

Misophistful commented 10 years ago

For anyone who is following along with this issue, I thought I'd post the work-around that Zach came up with:

(def finished? (promise))

(defn tests []
  ;; <Put your tests here>

  (deliver finished? true)))

(defame your-game-name
  :on-create
  (fn [this]
    (tests)))

(try
  (let [config (LwjglApplicationConfiguration.)]
    (set! (. config width) 1)
    (set! (. config height) 1)
    (set! (. config x) 0)
    (set! (. config y) 0)
    (LwjglApplication. your-game-name config))
  (catch Exception _ (on-gl (tests))))

@finished?

This approach works for both lein autoexpect and the Light Table Expectations plugin.