world loading takes an extremely long time with high CPU when many source modules are present

[keturn]

What you were trying to do

Start or join a game.

What actually happened

It hangs here [TODO: insert screenshot] for minutes (or much longer?) using high CPU (one core? or all of them?).

How to reproduce

1. Clone engine repo
2. ./groovyw module init omega
3. Run the TerasologyPC task in IntelliJ IDEA.
4. Select and start CoreSampleGameplay and observe that the game does not start but is stuck in loading phase

Log details and game version

Discussion and Causes

Some earlier notes on this are on #4372, but that was a prototype PR that has since been dropped.

This is not a build configuration error.

Some things shifted around and now single-player games (in certain dev environments) are running in to the performance problem we identified in multiplayer games in #4154 back in September.

For next steps, I suggest:

• optimize this code querying subtypes in ModuleEnvironmentSandbox and/or gestalt's ModuleClassLoader. There are some preliminary ideas about that written down already.
• update to current gestalt (#4144), and then optimize that stuff, because some of it probably changes with the new version anyway.

In the meantime, the stopgap workaround is:

• don't have a ton of modules subprojects checked out in your workspace. The performance problem climbs sharply with the number of module directories.

~It's interesting that JDR encountered this from gradle –~

We now think that was a different game start-up bug, fixed by #4394. I've edited the reproduction plan to focus on IntelliJ specifically.

– previously Michael and I had seen it happen when running from IntelliJ but not gradle. I hypothesized that might be because of this:

gradle is putting the jar files on the classpath, but IntelliJ is giving the classes directory.

It's not wrong to use the directories here. It might help module and asset hot-reloading speed in development and saves a build step. gradle might well do the same thing if the configuration changes a bit.

[jdrueckert]

I would be highly interested in an analysis where this comes from. It must be a recent thing as I recall that only a few days ago I was able to start various gameplays fine from my source omega workspace.

update to current gestalt (#4144), and then optimize that stuff

Optimistically estimated, it's summer before we're done with JOML and gestalt v7 conversion... Which means we can neither work towards a release nor do any kind of playtesting or even in-game testing of PRs until then. IMO that's very bad as we're practically converting stuff for JOML and the gestalt v7 migration blind-folded.

don't have a ton of modules subprojects checked out in your workspace

That's just not an option for conversions such as we're currently doing for JOML.

So again: Where does this come from? What introduced this horrendous performance drop?

[keturn]

I believe the poorly-performing code is the same spot as in #4154 we observed in September or earlier.

The change that made us run in to it in single-player games like this instead of multi-player games was #4343, somehow.

[jdrueckert]

https://github.com/MovingBlocks/Terasology/pull/4394 seems to fix this (at least for singleplayer)

[keturn]

Which means we can neither work towards a release nor do any kind of playtesting or even in-game testing of PRs until then.

I recognize that certain types of efforts (i.e. JOML) benefit from having lots of sources simultaneously checked out in a single workspace, but for tasks like those above, the thing Michael and I encountered was no such barrier. Is this something different? I guess it's possible, given that we saw it only when running under IntelliJ and not from gradle.

[keturn]

If #4394 fixed it, I'm guessing we really are talking about different things. 🤦 Too many "unusably slow to start" bugs open at once? 👿

[DarkWeird]

Slow start from idea and not from gradle - i think it is reflections lib. idea don't gather reflection at compile time, only in runtime. every time, when you use another ModuleEnvironment - you use runtime gathered reflections info.

Module environment used permamently - in stages(loading/ingame) and temporarly - for resolving world generators and etc stuff.

High CPU usage - seems it is multithreading chunk loading/generation. (especially if it is generation part)

Which loading phase when loading stuck? catching world?