Shinmera / parachute

An extensible and cross-compatible testing framework.
https://shinmera.github.io/parachute
zlib License
97 stars 9 forks source link

Test is not skipped #36

Closed shukryzablah closed 1 year ago

shukryzablah commented 2 years ago

When defining:

(in-package #:dataframe-test)

(define-test manipulate
  :skip (select))

(define-test select
  :parent manipulate
  (is #'dataframe:dataframe=
      (dataframe:make-dataframe :a '(1 2 3))
      (manipulate:select (dataframe:make-dataframe
                          :a '(1 2 3)
                          :b '("a" "b" "c"))
        '(:a))))

I get the following among the test output:

WARNING: Unhandled error when evaluating DATAFRAME-TEST::MANIPULATE:
  There is no applicable method for the generic function
    #<STANDARD-GENERIC-FUNCTION (COMMON-LISP:SETF PARACHUTE:STATUS) (1)>
  when called with arguments
    (:SKIPPED #<PARACHUTE:TEST DATAFRAME-TEST::SELECT>).
See also:
  The ANSI Standard, Section 7.6.6

  0.000 ✘ DATAFRAME-TEST::MANIPULATE
        ? DATAFRAME-TEST::SELECT
  0.000 ✘   (is #'dataframe= (make-dataframe :a '(1 2 3)) (select (make-dataframe :a '(1 2 3) :b '("a" "b" "c")) '(:a)))
  0.000 ✘ DATAFRAME-TEST::SELECT

;; Summary:
Passed:    10
Failed:     1
Skipped:    0

;; Failures:
   1/   1 tests failed in DATAFRAME-TEST::MANIPULATE
   1/   1 tests failed in DATAFRAME-TEST::SELECT
The test form   (select (make-dataframe :a '(1 2 3) :b '("a" "b" "c")) '(:a))
evaluated to    (:a (1 2 3) :b ("a" "b" "c"))
when            (:a (1 2 3))
was expected to be equal under #<FUNCTION DATAFRAME:DATAFRAME=>.

T
Screen Shot 2021-10-16 at 6 40 23 PM

Using the skip macro for the specific test form does end up skipping the test, but the warning remains.

How can I resolve this?

jdz commented 2 years ago

I have no idea what I'm doing, but changing child to subresult in https://github.com/Shinmera/parachute/blob/master/test.lisp#L214 "fixes" the problem for me. My understanding is that tests do not have the status slot, but results do.

Anyway, at least there are no error when running the test suite any more for me.

Shinmera commented 1 year ago

@jdz Your change looks correct. Not sure that what I had before was ever right.