Upgrades to the tests

[jasonrute]

This PR upgrades the tests in various ways:

• Added tests which checks consistency with a previous version of the model. This version is based on the network code currently in main and unless we plan to retrain the model, we should try to keep the test passing.
• Cache training of models in the training tests (which speeds up the predict server tests since they don't necessarily have to retrain a model first).
• Predict server tests can share the results from the same training test.
• Split the different steps of the pipeline into their own named tests so that once can easily say run all the server tests or all the hmodel tests.
• Added a predict_server test which doesn't update names.
• Updated the test documentation. (Please look it over and see if it makes sense. I won't go into too many details here in hope that you can get those details from the documentation.)

[jasonrute]

Note this PR doesn't do some of the other stuff we are planning to do with the tests, like align definitions in the traces, speed up the predict server tests by making them shorter, and a number of other small improvements. I'll work on those next.