keturn
commented
2 years ago This is almost a reversal from the situation we had previously, where we had too many dependency resolution errors blocking tests.
A lot of what we've done to clean that situation up has been reducing the number of methods that attempt to do module resolution. Ideally that's the responsibility of ModuleManager and nobody else should get involved.
I bet I removed some module resolution stuff that was happening earlier in the process, something that did show a higher-profile error message. With that seemingly redundant check gone, the error case presents in a different way.
Probably that StateHeadlessSetup#createGameManifest should outright fail if it can't make a manifest that contains all those modules, instead of just logging about it.
If we change it there (rather than in the MTE-specific subclass), it will affect the headless server too. I think that's probably fine and good but it's worth checking how headless servers are actually configured.
That would be an engine change, but there's probably work to do on the MTE side too to make sure an error there is presented in a useful way in the test results.
I had some tests that failed with unhelpful error messages. Among all the noise in the logs, I found a "ModuleRegistry has no latest version for module" message, but MTE was trying to forge ahead regardless.
It made for a confusing situation when there hadn't been a resolution error halting the test, yet the things I expected from that module weren't loaded.