Investigate and enhance in-game module downloader, keeping extra attention on jar files in the modules directory

[Cervator]

This issue is serving as both a store for potential things we could do in the short, mid, or longer term, that possibly new contributors could even help with (validate, experiment, develop), as well as a general reminder of some notable things in the area that'll probably clear up somewhat over time. We may eventually want to close this out after partial work has completed, possibly splitting out clear defined pieces for later work.

With #4343 we "lose" the ability at development time to use jar files from modules/ in favor of having Gradle resolve binary dependencies and load them onto the classpath from the Gradle cache directory instead.

Running the game from an actual game zip should still work the same - load jar files from the modules dir. You can also simply use groovyw module recurse x to get everything as source and you should get the source dir to take precedence over any jars in the Gradle cache.

However, in at least one case there's a thing that relies on jar files being placed in modules/ - the in-game module downloader, which can be used to fetch jar files at runtime, which will then sit around in your modules dir possibly looking confusing, if they're not actually being used. The module downloader also seems to be rarely used, and could probably use some validation and fixes.

Here are some possible things that could be done on this topic - a mix of dev workspace or game zip contexts

• [ ] Validation: Check on the module downloader and see if it actually works from a game zip (start a server from one dir, connect from a client in another dir with a module jar missing, does it download correctly and work?), possibly in a state before #4343
• [ ] Feature: Find a new way to support jar file modules at run-time for a dev workspace?
• [ ] Feature: Do not try to actually update to a newer version of the engine jar ... that's not really hot swappable, but can trigger if for instance you're trying to connect to a server running a release version yet you've got the next dev version which probably should still work even at a slightly newer engine version
• [ ] Feature: Better react when a module is added at runtime. At present for instance key mappings are not re-initialized so any unique key bindings from a new module are missing until the next game restart
• [ ] Validate: Check that a source module directory does actually win in precedence over a jar - either one in the Gradle cache or even one from modules/ if support is re-added to use from there at runtime (from a dev workspace)
• [ ] Feature: Allow the distribution of a module from a developer workspace - at present this doesn't work as you can't transfer a source module to another user connecting to your running server. Not sure if it would work using a module jar from the build dir somehow?
• [ ] Validate: Really perhaps more a part of #4343 but it would be nice to be sure we have good game logging or alternatives (like gradlew dependencies) that makes it clear which exact module code is being loaded from where

Related: #2381

[keturn]

Some clarification:

4343 adds more gradle involvement in resolving module dependencies, and along with that makes sure they get on the runtime classpath when we do gradlew game or run from the IDE. Any run-time module-scanning code that is done by the engine or gestalt is still as it was.

It's being flagged here for concern because if gradle provides one version of a module and the in-game downloader puts something else in modules/, we are not clear on which one will be used in practice.

If I thought that was an easy question to answer, I'd go look it up right now to remove the confusion. But I'm afraid there are probably several places where the game does module-stuff and that's enough for me to want to table the question for later.

e.g. when the process first starts, when you go to create a new game with the Advanced Game Setup, when entering that game, and is it different for single-player vs connecting to a remote server?

I suggested this is also a thing that will be more productive to work on when we have the current version of gradle-module available (i.e. #4144).

[keturn]

Validate: Really perhaps more a part of #4343 but it would be nice to be sure we have good game logging or alternatives (like gradlew dependencies) that makes it clear which exact module code is being loaded from where

When Terasology starts, I'm seeing log messages at INFO level from

• org.terasology.engine.module.ModuleManagerImpl [engine]
• org.terasology.module.ModulePathScanner [gestalt-module]

and in the case of some conflicts, ERROR-level messages from

• org.terasology.module.TableModuleRegistry [gestalt-module]

(these are gestalt v5 paths, subject to change when we upgrade gestalt)

and then later, when we switch to the Game state, at INFO level from

• org.terasology.engine.modes.loadProcess.RegisterMods [engine]

It looks to me like all of these messages include the module version, but most of them don't include its location, so if you have e.g. a local build version 1.0.1-SNAPSHOT and a jar of the same SNAPSHOT version from the repository, there might not be enough information to tell which is which.

[keturn]

Now that we're using v7 of gestalt-module (thanks to #4622), this would be a good topic to revisit.

[keturn]

Module downloading.

[keturn]

FYI: There was a RemoteModule class that was lost in transition. https://github.com/MovingBlocks/Terasology/pull/4622/#discussion_r619557280