Shinmera / parachute

An extensible and cross-compatible testing framework.
https://shinmera.github.io/parachute
zlib License
94 stars 9 forks source link

running tests on a package unexpectedly ignores hierarchy #26

Closed 3b closed 1 year ago

3b commented 3 years ago

with

(defpackage #:package
  (:use :cl :parachute))
(in-package #:package)

(define-test fail
  (fail t))

(define-test foo
  :depends-on (fail))

(define-test (foo bar)
  (true :ok))

running (test 'foo) runs test FAIL, then skips FOO and its children.

running (test 'package) runs test FAIL then skips FOO but then runs BAR separately even though its parent was skipped.

sabracrolleton commented 3 years ago

I think this is working as documented.

FOO depends-on FAIL, so when FAIL "fails", the test FOO is skipped as documented.

BAR does not depend on anything (parents fail if children fail, but children do not depend on their parent - it is bottom up, not top down), so when you are running the entire package, BAR is run.

Suppose FOO has three children: BAR1, BAR2 and BAR3.

FOO will fail if any of the children fail but we do not want to skip testing BAR2 or BAR3 if BAR1 fails.

So same idea with FOO depending on FAIL. We still want to know the results of test BAR.

I think I understand that you are asking for the ability of children to inherit parental dependencies. Am I getting that right?

Shinmera commented 3 years ago

Yes, that is what he's asking for, and I think it's a reasonable request.