Global history testing

[aadcg]

In d32b369b18c753b1a618b2e611e62f0342f90857 a change was introduced to deal with that fact that running these tests locally always failed. However, running the test suite twice locally results in the same problem.

Steps to reproduce:

• (asdf:load-system "nyxt/tests")
• (in-package :nyxt/tests)
• (dotimes (_ 2) (with-summary () (run-test 'global-history)))

NYXT/TESTS> (dotimes (_ 2) (with-summary () (run-test 'global-history)))

Starting: NYXT/TESTS::GLOBAL-HISTORY
 <INFO> [15:31:05] nyxt user-files.lisp (read-file :around nyxt-profile nyxt-file) -
  Loading #P"/home/aadcg/.local/share/test/history/default.lisp".
NYXT/TESTS::GLOBAL-HISTORY - PASSED (0.76s) : 7 assertions passed

Starting: NYXT/TESTS::GLOBAL-HISTORY
NYXT/TESTS::GLOBAL-HISTORY - FAILED (0.59s) : 2 assertions passed
  | FAILED (5)
  | Failed Form: (ASSERT-EQ 1
  |                         (LENGTH
  |                          (HISTORY-TREE:ALL-DATA (FILES:CONTENT FILE))))
  | Expected 1
  | but saw 2
  | Failed Form: (ASSERT-EQUALITY #'QURI:URI=
  |                               (QURI.URI:URI "http://example.org")
  |                               (URL ENTRY))
  | Expected #<QURI.URI.HTTP:URI-HTTP http://example.org>
  | but saw #<QURI.URI.HTTP:URI-HTTP http://example.org/sub>
  | Failed Form: (ASSERT-EQ 1
  |                         (LENGTH
  |                          (HISTORY-TREE:ALL-DATA (FILES:CONTENT FILE))))
  | Expected 1
  | but saw 2
  | Failed Form: (ASSERT-EQ 2
  |                         (IMPLICIT-VISITS
  |                          (FIRST
  |                           (HISTORY-TREE:ALL-DATA (FILES:CONTENT FILE)))))
  | Expected 2
  | but saw 1
  | Failed Form: (ASSERT-STRING= "foo" (TITLE ENTRY))
  | Expected "foo"
  | but saw ""
  |
  | 
NIL

[Ambrevar]

It's a different problem but yes, the test is not functional enough then! Thanks for catching this, André!

Should we have a meta-test that runs the tests twice?

[aadcg]

Should we have a meta-test that runs the tests twice?

I don't think so honestly. We'd be leaning towards a (potentially) endless chain of meta testing that goes by: program -> test(program) -> test(test(program)) -> ...

We just need to be extremely careful so that our test suite is as bug-free as we can. It's ok it's not perfect yet :)

[Ambrevar]

@aadcg Is this fixed? Wrong issue, sorry :p

[aadcg]

This test is still an abomination. I can't think of any good reason that would justify a test being stateful.