Anuken / Mindustry

The automation tower defense RTS
https://mindustrygame.github.io
GNU General Public License v3.0
22.69k stars 2.97k forks source link

Is it possible to speed up Mods loading process? #8105

Closed SSTentacleSS closed 1 year ago

SSTentacleSS commented 1 year ago

Platform: Linux Server

Build: v140.4

Issue: Sorry, but how long do I have to wait for one plugin to load? and i'm not kidding I have a plugin that weighs 40.9 megabytes, but the server is trying to analyze, as I understand it, all folders for the presence of plugins, is it possible to remove this? or somehow speed it up a bit

image

Steps to reproduce: Take a plugin weighing 40.9 megabytes and throw it on the server (loading on a 32-core processor, 59.7 GiB RAM, 3 TiB NVMe without debuggers and VisualVM took me 25 seconds)

Link(s) to mod(s) used: Private mods with scala runtime


Place an X (no spaces) between the brackets to confirm that you have read the line below.

SSTentacleSS commented 1 year ago

Well, now this is just funny image

SSTentacleSS commented 1 year ago

It would be beneficial to scan for dependency plugins that are described in plugin.json instead of manually iterating through huge amounts of data

SSTentacleSS commented 1 year ago

Stopped at image With this plugin image

Kieaer commented 1 year ago

Do you have the plugin source? May be it caused by Plugin. Even my 38MB plugin loads in under 3 seconds.

SSTentacleSS commented 1 year ago

Do you have the plugin source? May be it caused by Plugin. Even my 38MB plugin loads in under 3 seconds.

I am developing this plugin, the sources are all mine, the problem is that this script, due to the complexity of the structure, Jar has to go through 800 thousand directories in a cycle 100 times, which makes it simply unbearable, you can simply remove the dependency search by plugin.json, specifying the necessary dependencies manually, in json

and I think with your plugin the same problem with download speed caused by the search for dependencies VisualVM clearly showed that the plugin takes only 40 milliseconds of CPU time

Kieaer commented 1 year ago

VisualVM seems to have omitted some elements. The corresponding plugin/mod source is required to determine the cause.

SSTentacleSS commented 1 year ago

@Kieaer I analyzed the byte code in more detail - my conclusion is that ZipFi needs to be redone, since ZipFi in the current version, I assume it analyzes all folders in the jar file, which is very bad for performance, I think it might make sense to limit the ZipFi scan or do all the calculations sub-folders "lazy" thank you for your advice!

image

SSTentacleSS commented 1 year ago

(I also optimized the size of the plugin to 12 megabytes, so now it loads in 2-3 seconds)

SSTentacleSS commented 1 year ago

@Kieaer @Anuken #136

Anuken commented 1 year ago

For performance optimizations, make a PR.