Reuse or cache Git properties for multi-project builds

[joschi]

I'm using the gradle-git-properties plugin in a Gradle multi-project build with around 100 subprojects and a rather long history, so that fetching the Git properties takes some time. This project lives in a single Git repository, so that the Git properties would have to be generated only once and could be passed to the subprojects. Each subproject should have its own git.properties file with the respective data.

Generating the Git properties and writing them into git.properties files takes around 2:30 minutes:

# ./gradlew generateGitProperties
Configuration on demand is an incubating feature.

BUILD SUCCESSFUL in 2m 33s
99 actionable tasks: 99 executed

What's the idiomatic way to speed this up and for the subprojects to reuse the Git properties which have already been collected in any other project? Or is there anything missing in gradle-git-properties which could be contributed?

---

I've created a repository to demonstrate the issue at https://github.com/joschi/gradle-git-properties-multi-module-test with 20,000 (small) commits.

Here's the corresponding Gradle Build Scan: https://scans.gradle.com/s/ks4g3odxjdwvg

[tha2015]

Thanks @joschi for your sample repo. It helped me a lot.

I think you are correct that the plugin is slow on repositories with many commits. I ran profiling with your provided repo and it looks like the plugin is slow when calculating "git.total.commit.count" property (I think it is related to this issue https://github.com/ajoberstar/grgit/issues/210 )

One workaround is to avoid generating that property completely by specifying only needed properties or overriding it with a custom property with same name "git.total.commit.count".

gitProperties { customProperty "git.total.commit.count", 0 }

Please test if the workaround reduces the build time for you. It will take some time to see how to replace the implementation for this property with a faster implementation. Thanks again for your contribution.

[joschi]

@tha2015 Thanks for looking into this!

I've beefed up the test repo to 50,000 commits and ran a build with build scan on it, both with the custom property for git.total.commit.count and without it.

Build Scan without custom property: https://scans.gradle.com/s/dru3ttz6f4opq (total 22s) Build Scan with custom property: https://scans.gradle.com/s/zya7xpd75qu6k (total 5s)

So overriding the git.total.commit.count seems to help. 👍

Sharing the already fetched Git properties between subprojects would be great nonetheless. 😉

[joschi]

Anecdotal Evidence: For the real-world repository, overriding git.total.commit.count brought down the time for generateGitProperties from 2m33s to 31s. 👍

# gradle generateGitProperties
Configuration on demand is an incubating feature.

BUILD SUCCESSFUL in 31s
99 actionable tasks: 99 executed

[tha2015]

@joschi Can you help to test the latest version 2.2.3 to see if it works correctly? Thanks

[joschi]

@tha2015 Works for me, thanks! 🎉

Here's the comparison between gradle-git-properties 2.2.2 and 2.2.3 in https://github.com/joschi/gradle-git-properties-multi-module-test with both of the resulting git.properties files being correct:

gradle-git-properties 2.2.2:

./gradlew build

BUILD SUCCESSFUL in 25s
8 actionable tasks: 8 executed

gradle-git-properties 2.2.3:

./gradlew build

BUILD SUCCESSFUL in 6s
8 actionable tasks: 8 executed