gradle / actions

A collection of GitHub Actions to accelerate your Gradle Builds on GitHub
https://github.com/marketplace/actions/build-with-gradle
MIT License
129 stars 28 forks source link

Failing to restore Gradle Cache because of "Error: Content-Length not found on blob response" #235

Closed buehlerjochen closed 1 month ago

buehlerjochen commented 1 month ago

Hi,

I would like to use the setup-gradle action to store and restore the Gradle caches. But the action fails every time due to errors like this: Warning: Failed to restore gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd: Error: Content-Length not found on blob response

We have a build with multiple jobs, the relevant ones are:

These are the jobs (simplified) from the workflow file:

jobs:
  build-commit:
    steps:

    # preparation, configuration

    - uses: actions/checkout@v3
      with:
        fetch-depth: 0
    - name: Set up JDK
      uses: actions/setup-java@v3
      with:
        java-version: 17
        distribution: 'zulu'

    # Use setup-gradle step to save/restore Gradle cache and speed up builds
    - name: Setup Gradle cache
      uses: gradle/actions/setup-gradle@v3
      with:
        gradle-version: wrapper
    - name: "Validate Gradle Wrapper"
      uses: gradle/wrapper-validation-action@v1

    # build & publish

    - name: Gradle build
      run: ./gradlew -i --stacktrace clean build

  compatibility-test:
    strategy:
      matrix:
        gradle-version: [ '7.3', '8.6' ]

    needs: build-commit

    steps:

      - name: Checkout
        uses: actions/checkout@v3
      - name: Set up Java
        uses: actions/setup-java@v3
        with:
          distribution: 'zulu'
          java-version: 17

      # Use setup-gradle step to save/restore Gradle cache and speed up builds
      - name: Setup Gradle cache
        uses: gradle/actions/setup-gradle@v3
        with:
          gradle-version: wrapper
          cache-read-only: true
      - name: "Validate Gradle Wrapper"
        uses: gradle/wrapper-validation-action@v1

      - name: Compatibility test
        run: ./gradlew -i --stacktrace compatibilitytest
        env:
          GRADLE_VERSION: ${{ matrix.gradle-version }}

I can see how the action stores the cache for the compile job, but then fails when restoring during:

generate-job-summary is enabled and I can see how the action stores cache entries in the compile job, but does not restore them in the compatibility test job and in subsequent build executions. For the failed restore attempts, it prints these details:

    Entry: Gradle User Home
    Requested Key : gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd
    Restored  Key : 
              Size: 
              (Entry not restored: Content-Length not found on blob response)
    Saved     Key : 
              Size: 
              (Entry not saved: cache is read-only)

We are using:

Could it be that there is a compatibility issue with GHES? Or am I doing something wrong?

buehlerjochen commented 1 month ago

@ljacomet I attended your talk about ephemeral CI on Devoxx London and we talked on the way out. Maybe you have an idea about this issue?

bigdaz commented 1 month ago

I presume this is a compatibility issue with GHES. I'm aware that some users have had success using setup-gradle with GHES, but I have no way to test this compatibility.

There are a couple of things that you could try to help diagnose this issue:

  1. See if you can use actions/cache directly to save/restore the Gradle User Home: https://github.com/actions/cache/blob/main/examples.md#java---gradle
  2. Try actions/cache@v4 as well as v3 as described in the above example
  3. Try out the previous major release of the setup-gradle action, which was gradle/gradle-build-action@v2. All input parameters listed in your example should work the same.
buehlerjochen commented 1 month ago

@bigdaz Thank you for the quick response.

I'm also afraid it might be a compatibility issue with GHES. But can it be that I am the first to report it?!

Following your suggestions, here is what I tested in order to diagnose the problem:

1) Use basic actions/cache

Save cache works a bit different, but is able to upload the cache:

Post job cleanup.
/usr/bin/tar --posix -cf cache.tgz --exclude cache.tgz -P -C /runner/_work/jira-reports-gradle-plugin/jira-reports-gradle-plugin --files-from manifest.txt -z
Cache Size: ~324 MB (339440983 B)
Cache saved successfully
Cache saved with key: Linux-gradle-95ecf835a01192c137176bcb253c71ee27a6e335f97304a3e86bee887c59dbf3

Restore also fails with Failed to restore: Content-Length not found on blob response

This is the full log output:

Run actions/cache@v3
  with:
    path: ~/.gradle/caches
  ~/.gradle/wrapper

    key: Linux-gradle-95ecf835a01192c137176bcb253c71ee27a6e335f97304a3e86bee887c59dbf3
    restore-keys: Linux-gradle-

    enableCrossOsArchive: false
    fail-on-cache-miss: false
    lookup-only: false
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.11-9/x64
    JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.11-9/x64
Warning: Failed to restore: Content-Length not found on blob response
Cache not found for input keys: Linux-gradle-95ecf835a01192c137176bcb253c71ee27a6e335f97304a3e86bee887c59dbf3, Linux-gradle-

2) Try actions/cache@v4 as well as v3

Exactly the same result as with v3.

3) Use gradle/gradle-build-action@v2

Cache upload works as before. Restore fails with the same error as with setup-gradle@v3:

Entry: Gradle User Home
    Requested Key : v8-gradle|Linux|build commit-compatibility-test[26be8a9e10540768cab7c84d80a576df]-883739c846e093442084301e65231f046dec42ff
    Restored  Key : 
              Size: 
              (Entry not restored: Content-Length not found on blob response)
    Saved     Key : 
              Size: 
              (Entry not saved: cache is read-only)

4) Run setup-gradle action with debug output

I thought this might help you with diagnosing the problem.

Output of restore action: ``` 2024-05-22T05:51:56.4690459Z ##[debug]Evaluating condition for step: 'Setup Gradle cache' 2024-05-22T05:51:56.4693431Z ##[debug]Evaluating: success() 2024-05-22T05:51:56.4694151Z ##[debug]Evaluating success: 2024-05-22T05:51:56.4694693Z ##[debug]=> true 2024-05-22T05:51:56.4695471Z ##[debug]Result: true 2024-05-22T05:51:56.4696449Z ##[debug]Starting: Setup Gradle cache 2024-05-22T05:51:56.4731246Z ##[debug]Register post job cleanup for action: gradle/actions/setup-gradle@v3 2024-05-22T05:51:56.4746191Z ##[debug]Loading inputs 2024-05-22T05:51:56.4759182Z ##[debug]Evaluating: toJSON(matrix) 2024-05-22T05:51:56.4759715Z ##[debug]Evaluating toJSON: 2024-05-22T05:51:56.4785286Z ##[debug]..Evaluating matrix: 2024-05-22T05:51:56.4785703Z ##[debug]..=> Object 2024-05-22T05:51:56.4808527Z ##[debug]=> '{ 2024-05-22T05:51:56.4808827Z ##[debug] "gradle-version": "7.3" 2024-05-22T05:51:56.4809148Z ##[debug]}' 2024-05-22T05:51:56.4809478Z ##[debug]Result: '{ 2024-05-22T05:51:56.4810032Z ##[debug] "gradle-version": "7.3" 2024-05-22T05:51:56.4810383Z ##[debug]}' 2024-05-22T05:51:56.4812022Z ##[debug]Evaluating: github.token 2024-05-22T05:51:56.4812386Z ##[debug]Evaluating Index: 2024-05-22T05:51:56.4812944Z ##[debug]..Evaluating github: 2024-05-22T05:51:56.4813323Z ##[debug]..=> Object 2024-05-22T05:51:56.4813690Z ##[debug]..Evaluating String: 2024-05-22T05:51:56.4814038Z ##[debug]..=> 'token' 2024-05-22T05:51:56.4814756Z ##[debug]=> '***' 2024-05-22T05:51:56.4815234Z ##[debug]Result: '***' 2024-05-22T05:51:56.4816294Z ##[debug]Loading env 2024-05-22T05:51:56.4824650Z ##[group]Run gradle/actions/setup-gradle@v3 2024-05-22T05:51:56.4825020Z with: 2024-05-22T05:51:56.4825306Z gradle-version: wrapper 2024-05-22T05:51:56.4825713Z cache-read-only: true 2024-05-22T05:51:56.4826039Z cache-disabled: false 2024-05-22T05:51:56.4826340Z cache-write-only: false 2024-05-22T05:51:56.4826783Z cache-overwrite-existing: false 2024-05-22T05:51:56.4827159Z gradle-home-cache-includes: caches notifications 2024-05-22T05:51:56.4827562Z gradle-home-cache-cleanup: false 2024-05-22T05:51:56.4827938Z add-job-summary: always 2024-05-22T05:51:56.4828336Z add-job-summary-as-pr-comment: never 2024-05-22T05:51:56.4828672Z dependency-graph: disabled 2024-05-22T05:51:56.4829016Z dependency-graph-continue-on-failure: true 2024-05-22T05:51:56.4829446Z build-scan-publish: false 2024-05-22T05:51:56.4829812Z validate-wrappers: false 2024-05-22T05:51:56.4830127Z generate-job-summary: true 2024-05-22T05:51:56.4830476Z gradle-home-cache-strict-match: false 2024-05-22T05:51:56.4830956Z workflow-job-context: { "gradle-version": "7.3" } 2024-05-22T05:51:56.4831471Z github-token: *** 2024-05-22T05:51:56.4831844Z env: 2024-05-22T05:51:56.4832270Z JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.11-9/x64 2024-05-22T05:51:56.4832657Z JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.11-9/x64 2024-05-22T05:51:56.4833057Z ##[endgroup] 2024-05-22T05:51:56.9406057Z ##[debug]Determined user.home from java -version output: '/home/runner' 2024-05-22T05:51:57.0060506Z ##[debug]Determined user.home from java -version output: '/home/runner' 2024-05-22T05:51:57.0179579Z Merged default JDK locations into /home/runner/.m2/toolchains.xml 2024-05-22T05:51:57.0181460Z Created a new /home/runner/.gradle/gradle.properties with --info and --stacktrace 2024-05-22T05:51:57.0182971Z ::group::Restore Gradle state from cache 2024-05-22T05:51:57.0184089Z ##[group]Restore Gradle state from cache 2024-05-22T05:51:57.0184872Z Requesting Gradle User Home with 2024-05-22T05:51:57.0186313Z key:gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd 2024-05-22T05:51:57.0188489Z restoreKeys:[gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021],gradle-home-v1|Linux|compatibility-test,gradle-home-v1|Linux] 2024-05-22T05:51:57.0190225Z Using cache paths: /home/runner/.gradle/caches,/home/runner/.gradle/notifications,/home/runner/.gradle/.setup-gradle 2024-05-22T05:51:57.0192318Z ##[debug]Resolved Keys: 2024-05-22T05:51:57.0194217Z ##[debug]["gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd","gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]","gradle-home-v1|Linux|compatibility-test","gradle-home-v1|Linux"] 2024-05-22T05:51:57.0195976Z ##[debug]Checking zstd --quiet --version 2024-05-22T05:51:57.0197801Z ##[debug]Unable to locate executable file: zstd. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable. 2024-05-22T05:51:57.0199304Z ##[debug] 2024-05-22T05:51:57.0200507Z ##[debug]zstd version: null 2024-05-22T05:51:57.0204751Z ##[debug]Resource Url: https://github.***.com/_services/artifactcache/GQJxI00YsTaFN6QmE8i6c64L5mfoAyShrcuA60wd5Oy1gGUaFc/_apis/artifactcache/cache?keys=gradle-home-v1%7CLinux%7Ccompatibility-test%5Bb2e82155fc439b8351abee85b9386021%5D-8410f0ac768ab745eaf88d7100791f8a4378a0bd%2Cgradle-home-v1%7CLinux%7Ccompatibility-test%5Bb2e82155fc439b8351abee85b9386021%5D%2Cgradle-home-v1%7CLinux%7Ccompatibility-test%2Cgradle-home-v1%7CLinux&version=cf6cbae090da8e767740cfd3b88a88e2265ea5efc16192e5d9edac5613a22021 2024-05-22T05:51:57.1123043Z ::add-mask::*** 2024-05-22T05:51:57.1126113Z ##[debug]Cache Result: 2024-05-22T05:51:57.1128847Z ##[debug]{"scope":"refs/heads/master","cacheKey":"gradle-home-v1|Linux|build-commit[c1e7d5d74766449af309af791c6faae0]-8410f0ac768ab745eaf88d7100791f8a4378a0bd","cacheVersion":"cf6cbae090da8e767740cfd3b88a88e2265ea5efc16192e5d9edac5613a22021","creationTime":"2024-05-17T04:52:09.93Z","archiveLocation":"***"} 2024-05-22T05:51:57.1138640Z ##[debug]Archive Path: /runner/_work/_temp/03ab5b18-8288-424c-82cd-6f674acc2e63/cache.tgz 2024-05-22T05:51:57.1140546Z ##[debug]Use Azure SDK: false 2024-05-22T05:51:57.1141534Z ##[debug]Download concurrency: 8 2024-05-22T05:51:57.1142388Z ##[debug]Request timeout (ms): 30000 2024-05-22T05:51:57.1143299Z ##[debug]Cache segment download timeout mins env var: undefined 2024-05-22T05:51:57.1144067Z ##[debug]Segment download timeout (ms): 600000 2024-05-22T05:51:57.1144899Z ##[debug]Lookup only: false 2024-05-22T05:51:57.1812517Z ##[warning]Failed to restore gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd: Error: Content-Length not found on blob response 2024-05-22T05:51:57.1848360Z Error: Content-Length not found on blob response 2024-05-22T05:51:57.1849131Z at /runner/_work/_actions/gradle/actions/v3/dist/setup-gradle/main/index.js:7443:23 2024-05-22T05:51:57.1849807Z at Generator.next () 2024-05-22T05:51:57.1850548Z at fulfilled (/runner/_work/_actions/gradle/actions/v3/dist/setup-gradle/main/index.js:7261:58) 2024-05-22T05:51:57.1851261Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2024-05-22T05:51:57.1852005Z Gradle User Home cache not found. Will initialize empty. 2024-05-22T05:51:57.1852805Z ::endgroup:: 2024-05-22T05:51:57.1853180Z ##[endgroup] 2024-05-22T05:51:57.1935389Z ##[debug]Node Action run completed with exit code 0 2024-05-22T05:51:57.1938119Z ##[debug]GRADLE_ACTION_ID='gradle/actions/setup-gradle' 2024-05-22T05:51:57.1938556Z ##[debug]GRADLE_BUILD_ACTION_SETUP_COMPLETED='true' 2024-05-22T05:51:57.1939053Z ##[debug]GRADLE_BUILD_ACTION_CACHE_RESTORED='true' 2024-05-22T05:51:57.1939511Z ##[debug]DEVELOCITY_INJECTION_INIT_SCRIPT_NAME='gradle-actions.inject-develocity.init.gradle' 2024-05-22T05:51:57.1940024Z ##[debug]DEVELOCITY_AUTO_INJECTION_CUSTOM_VALUE='gradle-actions' 2024-05-22T05:51:57.1940530Z ##[debug]GITHUB_DEPENDENCY_GRAPH_ENABLED='false' 2024-05-22T05:51:57.1942423Z ##[debug]Save intra-action state GRADLE_BUILD_ACTION_SETUP_COMPLETED = true 2024-05-22T05:51:57.1942959Z ##[debug]Save intra-action state USER_HOME = /home/runner 2024-05-22T05:51:57.1943465Z ##[debug]Save intra-action state GRADLE_USER_HOME = /home/runner/.gradle 2024-05-22T05:51:57.1944225Z ##[debug]Save intra-action state GRADLE_BUILD_ACTION_CACHE_RESTORED = true 2024-05-22T05:51:57.1945444Z ##[debug]Save intra-action state CACHE_LISTENER = {"cacheEntries":[{"entryName":"Gradle User Home","requestedKey":"gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]-8410f0ac768ab745eaf88d7100791f8a4378a0bd","requestedRestoreKeys":["gradle-home-v1|Linux|compatibility-test[b2e82155fc439b8351abee85b9386021]","gradle-home-v1|Linux|compatibility-test","gradle-home-v1|Linux"],"notRestored":"Content-Length not found on blob response"}],"cacheReadOnly":false,"cacheWriteOnly":false,"cacheDisabled":false,"cacheDisabledReason":"disabled"} 2024-05-22T05:51:57.1946495Z ##[debug]Save intra-action state deprecations = [] 2024-05-22T05:51:57.1947839Z ##[debug]Finishing: Setup Gradle cache ```
bigdaz commented 1 month ago

The setup-gradle action is using the actions/cache library under the covers. So if actions/cache@v3 is not able to work in your environment, then there's nothing we can do so resolve the issue with setup-gradle.

Folks are using GHES successfully with setup-gradle, so I suspect the issue is with your particular installation. I don't really know how GHES is installed and configured, but I can imagine that a some internal network infrastructure could be blocking the response, or stripping the content-length header from the response.

buehlerjochen commented 1 month ago

Yes, that is what I suspect as well. I already reached out to our operations team to clarify the issue. Thank you for your help and analysis. ♥️