cognitect-labs / test-runner

A test runner for clojure.test
Eclipse Public License 2.0
286 stars 31 forks source link

Tests do not finish if an agent has been started. #43

Open farbeiza opened 1 year ago

farbeiza commented 1 year ago

If an agent is started, tests do not finish, waiting for the agent threads to finish:

These files should reproduce the issue: deps.edn:

{:paths ["src"]
 :deps {}
 :aliases {:test {:extra-paths ["test"]
                  :extra-deps {io.github.cognitect-labs/test-runner {:git/sha "a522ab2851a2aa5bf9c22a942b45287a3a019310"}}
                  :main-opts ["-m" "cognitect.test-runner"]
                  :exec-fn cognitect.test-runner.api/test}}}

_coretest.clj:

(ns core-test
  (:require [clojure.test :refer :all]))

(deftest duration-test
  (let [a (agent 0)]
    (send a inc)
    (await a)
    (is (= 1 @a))))

I've seen a PR which could be very useful for this: Add before and after options since it will allow to run (shutdown-agents) after running the tests.

puredanger commented 1 year ago

We used to have such logic (see https://github.com/cognitect-labs/test-runner/commit/23771f4bee77d4e938b9bfa89031d2822b4e2622) but that was removed in favor of having the execution context handle it.

How are you running test-runner?

puredanger commented 1 year ago

And if you are running with the Clojure CLI, which version (exit behavior has changed there several times)?

farbeiza commented 1 year ago

I'm running it with clj -X

$ clj -X:test

Running tests in #{"test"}

Testing core-test

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.

Version is:

$ clj -version
Clojure CLI version 1.11.1.1165
seancorfield commented 3 weeks ago

I think it's okay for -M contexts to call shutdown-agents at the end but it is not okay for -X contexts to do this since it undermines programmatic invocation of the test runner.

raystubbs commented 1 week ago

Seems like this happens if a Timer, or any other user thread remains alive too... seems like it'd be good to have something (on an opt-in basis) that just calls System/exit after all the test runner stuff finishes.