search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.67k stars 1.03k forks source link

Gradle builds slow to start #13730

Open dweiss opened 1 month ago

dweiss commented 1 month ago

Description

This has been mentioned by Mike Sokolov, I think. Gradle builds have become sloooow to start as we upgraded from version to version. Interestingly, I've come across this hint:

https://docs.gradle.org/current/userguide/sharing_build_logic_between_subprojects.html#sec:convention_plugins_vs_cross_configuration

which is the exact opposite to what we do. I do have some background in aspect-oriented programming so, to me, this kind of cross-configuration and separation of concerns is a way to clean up the build configuration and separate different parts of it. Well, gradle folks clearly think otherwise.

I have not debugged this intensively but when you run even the smallest task with the -debug option, you'll see a lot of this:

2024-09-06T08:31:11.679+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 11: acquired lock on worker lease
2024-09-06T08:31:11.679+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 11: released lock on worker lease
2024-09-06T08:31:11.679+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 10: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 10: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 4: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 4: released lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: acquired lock on worker lease
2024-09-06T08:31:11.680+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: acquired lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 7: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Cross-configure project :lucene:analysis:opennlp' started
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: acquired lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 6: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Cross-configure project :lucene:analysis:opennlp'
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 4: acquired lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 4: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: acquired lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 5: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: acquired lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 3: released lock on worker lease
2024-09-06T08:31:11.681+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: acquired lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 2: released lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: acquired lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 8: released lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: acquired lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 9: released lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 10: acquired lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 10: released lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 11: acquired lock on worker lease
2024-09-06T08:31:11.682+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker Thread 11: released lock on worker lease

Something must have changed in how this configuration phase is handled. It used to be super fast, now it's a hog.

Rewriting the build to gradle's official guidelines is a longer journey but we may have to do it one day.

Current workaround: use a daemon to keep the project "alive" in memory. Tweak the daemon's timeout, if you wish:

# timeout after 15 mins of inactivity.
org.gradle.daemon.idletimeout=900000

Version and environment details

No response

msokolov commented 1 month ago

Thanks for opening this @dweiss ! Indeed I reported slow test startups in IntellJ, but it makes sense that the problem is more widespread and related to gradle getting its ducks in a row

dweiss commented 1 month ago

These are related issues because Intellij starts gradle to build its project/ data model, which is then used to create Intellij project.