prepareWorkspace automatic evaluation is very slow in big multiproject messes

[quat1024]

I'm working on a big multiproject with five instances of VanillaGradle and a couple other Minecraft plugins. When I press the "refresh Gradle" button in intellij, this procedure happens:

• gradle evaluates all subprojects
• :xplat-1.16.5 runs the prepareWorkspace task, which requires first evaluating all subprojects again
• :xplat-1.17.1 runs the prepareWorkspace task, which requires first evaluating all subprojects again
• :xplat-1.18.2 runs the prepareWorkspace task, which requires first evaluating all subprojects again
• ...

https://github.com/SpongePowered/VanillaGradle/blob/20a224ca61bcb9b12d118e39ff402c0dbcf627e6/subprojects/gradle-plugin/src/main/java/org/spongepowered/gradle/vanilla/VanillaGradle.java#L173-L174

This means the time it takes for Gradle to chug through all the projects is proportional to the number of VanillaGradle subprojects times the total number of subprojects; kinda O(n²). Exacerbating this, the DownloadAssetsTask that prepareWorkspace runs usually takes about ten or fifteen seconds to execute. Overall, Gradle-to-IntelliJ sync takes ~two minutes, which is really slow.

cursory inspection, it looks like DownloadAssetsTask always parses the asset index and reverifies all assets, which requires opening a lot of files, verifying a lot of hashes, etc. In my Gradle plugin I write a small flag that tells the asset downloader "you already got the assets for this version" and skips doing the expensive work. This obviously creates cache-coherency issues if the assets are modified without erasing that file, but it's an ok compromise.

[zml2008]

😩

but thanks for pointing that out :) we could probably toss something in the build service to cache asset state -- and maybe do some sort of datestamping/writing the current hash of the asset index to that marker file to improve performance there.

The configuration issue is a bit thornier -- of course the solution Gradle probably wants to push us towards is to use config cache now that that's 'stable', but there are still missing features (reading build services at configuration time, etc) so we'll probably have to look at alternatives, maybe some sort of task bundling thing?

[quat1024]

as a quick bandaid, would some option to not run prepareWorkspace at all be workable? In a multiloader-template-style project, you're gonna have the assets available through Loom/ForgeGradle & won't be using the runClient task at all anyway

[zml2008]

hm, i'll take a look -- I think it should be possible to say something like "only run prepareWorkspace (ed: or even add it as a task trigger) if there are run configs defined" without too much difficulty

[zml2008]

I tossed up something quick at https://github.com/SpongePowered/VanillaGradle/pull/103, no idea if it actually works cuz I don't have a testing workspace set up on my laptop yet, but i'd appreciate it if you had the time/energy to check it out :)