Rewrite

[tingerrr]

This is an almost complete rewrite of typst-test which aims to:

• fix most of the current bugs
• improve performance by reducing IO
• allow doing things which were previously infeasible

TODO

Currently, most of the core implementation exists as disjunct components in the library crate, which need to be wired back together to reach feature parity with the original binary crate. There are a few things that I want to outline here to make sure they are implemented before I am done.

• LIB
  • [x] running custom user hooks defined in the test configuration
  • [x] configuration reading
  • [x] default library augmentation (adding an error catching function for library authors)
  • [ ] collecting tests in parallel I'm unsure whether ignore::walkdir works well in the presence of other parallel stuff like rayon, or if this could cause the actual library consumer to spawn a hilarious number of threads.
• CLI
  • [x] font loading
  • [x] package downloads
  • [x] environment control similar to typst#4227
  • [x] fine-grained control over type of comparison
  • [x] better diff image generation
  • [x] intuitive interface for filtering tests
  • [x] reporting is sprinkled all over the code base in a way I don't particularly like
  • [x] reporting interferes with regular logging as it delets lines on the same out stream
  • [ ] project I/O operations are not yet locked, running typst-test in parallel means we can get incoherent states on disk and there are no commands that fix incomplete tests this at the moment
  • [ ] some things were copied verbatim from the typst cli, it remains to be seen if this is a problem, but it could help to identify where typst#4452 needs to be generic
  • [x] tes tset expressions are undocumented
  • [ ] error reporting of test set expressions is not very good, this is likely a by product of how I defined the grammar
  • [ ] there are lots of breaking changes that will have to be documented
  • [ ] the infra for editing test's reference kinds is there but not implemented
  • [x] the infra for running hooks is there but not implemented
  • [x] error handling is not very easy to use at this moment, especially when there are operation failures, i.e. graceful failures
  • [x] compile failures currently debug print the diagnostic, which is not very helpful

Issues

• [x] resolves #11 by adding hooks
• [x] resolves #12 by using typst directly as a crate
• [x] resolves #14 by considering tests without references to be compile-only tests
• [x] resolves #16 by doing package downloads ourselves
• [x] resolves #17
• [x] resolves #20 by adding more granular control over comparison
• [x] resolves #8 by doing diffing regardless of size mismatch

[tingerrr]

The CLI is now implemented in terms of the new lib crate, compiles and successfully runs all tests for cetz.

It's currently reusing the lib crate's internal dev World impl, which is the next thing that has to be ported to the binary crate.

[tingerrr]

This is currently blocked on typst#4452, I want to avoid copying a bunch of code when it'll likely be available as a crate soon. This includes the recently copied font loading in ea3e19be1909035c217781bc2a13966b69640037.

[tingerrr]

This branch has no reached feature parity with the typst cli, i.e. font searching and package downloads.

There are a few issues that remain, which I have added to the TODO list in the issue comment.