Retrieving the version for large repositories is slow

[balrok]

When following the recommended way to set the version in a gradle project:

project.version = scmVersion.version

it can take a considerable amount of time to compute it for larger projects.

How to reproduce (real project)

• Clone a bigger project: git clone https://github.com/spring-projects/spring-framework.git (28k commits, 10k files) (note: this is actually not that big.)
• Patch it to run scmVersion.version:

  
  diff --git a/build.gradle b/build.gradle
  index e8c3550a7c..6da12e0876 100644
  --- a/build.gradle
  +++ b/build.gradle
  @@ -1,4 +1,5 @@
  plugins {

• id 'pl.allegro.tech.build.axion-release' version '1.17.0' id 'io.freefair.aspectj' version '8.3' apply false // kotlinVersion is managed in gradle.properties id 'org.jetbrains.kotlin.plugin.serialization' version "${kotlinVersion}" apply false @@ -11,6 +12,10 @@ plugins { id 'me.champeau.mrjar' version '0.1.1' }

+def start = System.currentTimeMillis(); +scmVersion.version +println("Took " + ((System.currentTimeMillis() - start) / 1000) + "s") + ext { moduleProjects = subprojects.findAll { it.name.startsWith("spring-") } javaProjects = subprojects.findAll { !it.name.startsWith("framework-") }

* spring-framework uses a version-scheme for its tags which are not like it is configured in the plugin. Since they do not seem to have an influence on the speed, simply delete them `git tag -d $(git tag)` or configure the plugin, that it works
* run `./gradlew cV` and watch out for the "Took Xs"

### Measurements:
* directly after cloning:
    * 0.5s - 2.0s
* adding 10k files: `for i in $(git ls-files|grep -v "/$"); do touch "$i.release-plugin-test"; done`
    * 1.5s - 3.5s
* adding 30k commits: `for i in $(seq 1 30000); do git commit --allow-empty -m"$i"; done`
    * 1.4s - 3.4s

## How to reproduce (real project)

Alternatively, one can create an empty git project with only build.gradle:
```gradle

plugins {
    id 'pl.allegro.tech.build.axion-release' version '1.17.0'
}

def start = System.currentTimeMillis();
scmVersion.version
println("Took " + ((System.currentTimeMillis() - start) / 1000) + "s")

• Depending on the test scenario one can create many commits or many files:
• for i in $(seq 1 30000); do git commit --allow-empty -m"$i"; done
• for i in $(seq 1 100000); do touch "$i.tmp"; done

Measurements:

• 30k empty commits:
  • initially 2s, later 0.15s
• 100k empty uncommitted files in root directory
  • 0.8s - 1.2s

Why is it important?

The current version is calculated with each execution of gradle. This means, that it also runs for each test-execution from the ide. Also my examples here don't paint the full picture: for a different project it takes 20s on my machine. Due to gradles build-directories and maybe some other random things, I have 600k files.

Proposed solutions

I think most of the time is spent in computing things, which are not always important:

• isClean https://github.com/allegro/axion-release-plugin/pull/543
  • here it is maybe possible to add an override-flag, like the overrideBranch-flag (I will prepare a PR for it)
  • alternatively make the isClean-property to be lazily evaluated, so its only computed if it is needed
• retrieving the tags takes some time because it iterates the whole log: https://github.com/allegro/axion-release-plugin/blob/main/src/main/java/pl/allegro/tech/build/axion/release/infrastructure/git/GitRepository.java#L389
  • unclear if all the data is always needed
  • maybe lazy evaluation would also help here, or maybe it is enough to partial-compute the data, only for the relevant tags
  • here https://github.com/allegro/axion-release-plugin/blob/main/src/main/java/pl/allegro/tech/build/axion/release/domain/VersionResolver.java#L85 it is called twice. Maybe there is an optimization possible

[bgalek]

hi! sure, feel free to make a PR with those improvements.

[balrok]

Just found there is also a related issue: #182 (just referencing it in case somebody stumbles over the other)

For the performance-problem with getting the tags I figured out a different optimization: if the tags are stored inside .git/packed-refs, they are resolved super fast with jgit. If this file doesn't exist yet, it can be created with git gc --aggressive --prune (warning: not sure if it can do any harm, so one might backup the .git-folder before).

Together with the overridenIsClean flag I get the scmVersion.version-calculation down to 0.064s from initially 18s.