Refactor tests into the main repository

[zorkow]

Refactors the tests directory and parts of the old tests repository.

• Moves tests execution to the top level. While this inflates our top-level packages and package.json file, it has the advantage that jest can produce coverage information for our functions.
• The structure of the tests directory reflects roughly the structure of the ts source directory. Why roughly?
  • There is an extra src directory containing some auxiliary functionality for tests.
  • TeX packages have only one for the entire package. I.e., Ams.test.ts should cover the entire package.
  • There might be some exceptions to the above rule if there are some tests that "topically" fit together. E.g., currently there is a Tag.test.ts file that tests different tag settings.
• Currently tests rely on commonjs modules, which has a couple of consequences:
  • For files that rely on other imports (i.e., most files!) we need to explicitly import the source from cjs. An exception is for example tests/input/tex/UnitUtil.test.ts where we can directly import the ts file, without extension. Once we move to esnext as our typescript target we can use that everywhere, with NODE_OPTIONS=--experimental-vm-modules switch.
  • We need to copy a special package.json into the cjs directory to redirect all the external library paths to their commonjs targets. (@dpvc I am wondering if we should not have that file in the cjs directory by default.)

Usage

• Run tests with pnpm test or pnpm jest Path-to-test.
• Coverage is then displayed in the console. Piping can be useful with pnpm test &> /tmp/test-output or similar.

Current State

• Only very few unit tests for lengths and UnitUtil.
• The majority of tests are for TeX input packages. The tests effectively translate some input tex expression into the MathJax data structure and compare received to expected by producing MathML output.
• As such they are not really unit tests, but cover a considerable number of other MathJax functions, too. For example, the SerializedMmlVisitor.
• Comparison is achieved using a custom matcher that extends jest's expect method (getting around some type issues) that does a deep comparison using xml2json functionality from the xml-js package.

Things left to do

• Tests not yet converted from the old repository: KeyvalTest, ParserConfigMacros.
• Get coverage for every TeX package as much as possible to 100% in the correspoding .test.ts file.
• Write some tools for tests, in particular
  • update the jest-diff-test for Emacs in the MathJax-tests repository to work with the current tests
  • write a copy facility for updating test or creating new tests.

[zorkow]

Changed everything to work with mjs files now, including importing modules via #js/ paths, and without .js extensions, as this is not necessary. @dpvc PTAL.

[dpvc]

I've made a PR to this branch that moves the testing into a separate subdirectory. See what you think of that, otherwise, we can go with this as is.