Add Lisp-Unit2 support

[Ambrevar]

In my opinion (see https://github.com/atlas-engineer/nyxt/issues/2199#issuecomment-1146808213), Lisp-Unit2 is one of the best test framework for Common Lisp (together with Parachute). It would be great to add support for it in Slite.

It does not seem to be too hard since Lisp-Unit2 also reifies test results with a structure / class.

Also see my personal hack to implement "run test at point" for Lisp-Unit2:

(defun ambrevar/sly-run-lisp-unit-test-at-point (&optional raw-prefix-arg)
  "See `sly-compile-defun' for RAW-PREFIX-ARG."
  (interactive "P")
   (call-interactively 'sly-compile-defun)
  (let ((name `(quote ,(intern (sly-qualify-cl-symbol-name (sly-parse-toplevel-form))))))
    (sly-eval-async
        `(cl:string-trim "

�"
                         (cl:with-output-to-string (s)
                                                   (cl:let ((lisp-unit2:*test-stream* s))
                                                           (lisp-unit2:run-tests :tests ,name :run-contexts 'lisp-unit2:with-summary-context))))
      (lambda (results)
        (switch-to-buffer-other-window  (get-buffer-create "*Test Results*"))
        (erase-buffer)
        (insert results)))))

(define-key lisp-mode-map (kbd "C-c C-v") 'ambrevar/sly-run-lisp-unit-test-at-point)

[mdbergmann]

Which fork of Lisp-Unit2 should be preferred? Seems some are way beyond the 'official' one.

[Ambrevar]

I'm using https://github.com/AccelerationNet/lisp-unit2 but I have no clue about the others.

EDIT: Fixed link.

[tdrhq]

I certainly think this is possible, and should not be too much work depending on the lisp-unit2's APIs. I'm happy to add it, but I want to make sure there's going to be a user and dog-fooder before I dive into it. @Ambrevar do you use lisp-unit2 on a regular basis?

(PS. Might not get to it in the next few days though)

[Ambrevar]

I plan to transition https://github.com/atlas-engineer/nyxt/, https://github.com/atlas-engineer/nfiles and all the deps (3-4 of them for now) to it.

[Ambrevar]

Update: we've fully switched to Lisp-Unit2 :)

[tdrhq]

@Ambrevar done, as of now you can run tests, rerun in debugger. Jump to test is not implemented for any CL implementation (in theory, that could be implemented for some CL implementations with some additional changes to lisp-unit2).

You just have to load slite/lisp-unit2. I'm closing this task out for now, but let me know if things are working. I'd be particularly interested in edge cases with tests without any results, tests with only failures, tests with only errors, etc. I'll update the documentation later.

[tdrhq]

(actually keeping this open for now until I can get verification from you)

[tdrhq]

I'm not sure how lisp-unit2 is supposed to handle this case:

(unit:define-test error-test
  (:tags '(bar))
  (error "blah blah"))

run-tests seems to not be handling the error. Is that how lisp-unit2 is supposed to work? Is there a better function than run-tests that I should be typically calling?

[Ambrevar]

I think this is meant to be run from asdf:test-system, which does the right thing here (that is, not prompt the debugger).

That said, if your question is how to test this specific test programmatically without triggering the debugger, then you can do:

(let ((*debugger-hook* nil))
  (lisp-unit2:run-tests :name 'error-test
                        :run-contexts 'lisp-unit2:with-summary))

From the README:

Debugging is controlled by *debugger-hook* (as is usual in common-lisp). You can make lisp-unit simply record the error and move on by binding *debugger-hook* to nil around your run-tests call.

[tdrhq]

that's interesting. What behavior do you expect to see? My personal expectation when using slite is that when running tests the debugger should not show up unless I rerun the test with debugging enable. Would you say that is what you expect too? I could bind debugger-hook to nil if that's the case. (But that's not 100% correct, since it wouldn't hande any crashes from other threads)

I think that should be an easy change, just let me know if that would be your expected behavior

[Ambrevar]

Actually I like that the debugger shows up when running individual tests: it gives me more information. If I only wanted a summary, I would run the whole test suite, right?

So for me it's fine as it is :)

[tdrhq]

@Ambrevar Just to be clear, slite has that mode. Essentially it runs all the tests, shows which tests failed, and you can press 'r' on the any test to run it with a debugger. So you can get best of both worlds. But in any case, play around with this for a while and let me know how you feel about it.