Closed cloudshiftchris closed 1 month ago
Thanks for pointing out this issue and the recommendation. Yes, this would be a breaking change because workflows may already be bound to the current Gradle User Home location.
Have you found the above workaround to make a significant different to performance?
Adding in the above snippet took a GH build w/ a small-ish Gradle (really just a few custom plugins for versioning as the main build on this one was dotnet) from an erratic 9-20m to consistent 5-6m.
This (from kotest) is perhaps a more reliable approach - using a link for .gradle
instead of relocating it; this would preserve any existing use of the current gradle home location (as the location doesn't really change, at least from a consumer perspective). There may be some rough edges with consumers (other than Gradle) that don't properly process the link.
Related: setup-action 3.5.0 has the new super-fancy cache-cleanup (which works amazing btw, previous cleanup wasn't catching old gradle distributions etc, nice and clean now).
That enhancement downloads (or restores from GH cache) the latest stable Gradle distribution; it stores this under the user home, which on Windows runners is on the Very Slow C: drive (a lot of the latencies noted above were from restoring multiple gradle distributions from cache). Should likely be in runner.temp
instead (on Windows runners this is already on the D: drive).
Provision Gradle 8.9
Received 88080384 of 135856402 (64.8%), 83.9 MBs/sec
Cache Size: ~130 MB (135856402 B)
"C:\Program Files\Git\usr\bin\tar.exe" -xf D:/a/_temp/14c14b63-d717-40e3-b74c-4241c90a5343/cache.tzst -P -C D:/a/****-****/****-**** --force-local --use-compress-program "zstd -d"
Cache restored successfully
Restored Gradle distribution gradle-8.9 from cache to C:\Users\runneradmin\gradle-installations\downloads\gradle-8.9-bin.zip
"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "$ErrorActionPreference = 'Stop' ; try { Add-Type -AssemblyName System.IO.Compression.ZipFile } catch { } ; try { [System.IO.Compression.ZipFile]::ExtractToDirectory('C:\Users\runneradmin\gradle-installations\downloads\gradle-8.9-bin.zip', 'C:\Users\runneradmin\gradle-installations\installs', $true) } catch { if (($_.Exception.GetType().FullName -eq 'System.Management.Automation.MethodException') -or ($_.Exception.GetType().FullName -eq 'System.Management.Automation.RuntimeException') ){ Expand-Archive -LiteralPath 'C:\Users\runneradmin\gradle-installations\downloads\gradle-8.9-bin.zip' -DestinationPath 'C:\Users\runneradmin\gradle-installations\installs' -Force } else { throw $_ } } ;"
Received 135856402 of 135856402 (100.0%), 64.6 MBs/sec
Extracted Gradle 8.9 to C:\Users\runneradmin\gradle-installations\installs\gradle-8.9
Provisioned Gradle executable C:\Users\runneradmin\gradle-installations\installs\gradle-8.9\bin\gradle.bat
@cloudshiftchris I'm not sure that RUNNER_TEMP
is the right place to install Gradle, but it's an option if it makes a difference. Can you tell me what RUNNER_TOOL_CACHE
points to on a Windows runner?
Fixing #270 will somewhat mitigate the performance overhead added by provisioning the latest Gradle as part of cache-cleanup.
@cloudshiftchris I'm not sure that
RUNNER_TEMP
is the right place to install Gradle, but it's an option if it makes a difference. Can you tell me whatRUNNER_TOOL_CACHE
points to on a Windows runner?
Below are all the env vars for windows-latest
as of an hour ago (this is dumped out as the first step in workflow, before anything else muddies the environment).
RUNNER_TOOL_CACHE=C:\hostedtoolcache\windows
- this is why tooling such as setup-go
create symlinks in here to the D: drive, so things show up "where they are expected to be" but are actually on the fast D: drive.
Arguably the Gradle install for cache cleanup is temporary/transient - it's internal, for that sole purpose. Likely should either be in a temporary dir, or homed alongside other stuff in .gradle
. Doesn't seem worth creating another place to store "gradle stuff" if it isn't transient.
ACTIONS_CACHE_URL=https://acghubeus2.actions.githubusercontent.com/wEbjysn6J8ywOP1seEwzFimKdaeFcV2yL35oDfmCYrijQTXtFQ/
ACTIONS_ID_TOKEN_REQUEST_TOKEN=***
ACTIONS_ID_TOKEN_REQUEST_URL=https://pipelinesghubeus12.actions.githubusercontent.com/wEbjysn6J8ywOP1seEwzFimKdaeFcV2yL35oDfmCYrijQTXtFQ/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/Actions/plans/ab35149c-d335-4d83-8d1e-a9092ca9a243/jobs/e9516119-0952-567e-6e4b-347d21a7ced3/idtoken?api-version=2.0
ACTIONS_RESULTS_URL=https://results-receiver.actions.githubusercontent.com/
ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE=C:\actionarchivecache\
ACTIONS_RUNTIME_TOKEN=***
ACTIONS_RUNTIME_URL=https://pipelinesghubeus12.actions.githubusercontent.com/wEbjysn6J8ywOP1seEwzFimKdaeFcV2yL35oDfmCYrijQTXtFQ/
ALLUSERSPROFILE=C:\ProgramData
ANDROID_HOME=C:\Android\android-sdk
ANDROID_NDK=C:\Android\android-sdk\ndk\25.2.9519653
ANDROID_NDK_HOME=C:\Android\android-sdk\ndk\25.2.9519653
ANDROID_NDK_LATEST_HOME=C:\Android\android-sdk\ndk\26.3.11579264
ANDROID_NDK_ROOT=C:\Android\android-sdk\ndk\25.2.9519653
ANDROID_SDK_ROOT=C:\Android\android-sdk
ANT_HOME=C:\ProgramData\chocolatey\lib\ant\tools\apache-ant-1.10.14
APPDATA=C:\Users\runneradmin\AppData\Roaming
AZURE_CONFIG_DIR=C:\azureCli
AZURE_DEVOPS_CACHE_DIR=C:\azureDevOpsCli\cache
AZURE_EXTENSION_DIR=C:\Program Files\Common Files\AzureCliExtensionDirectory
AZ_DEVOPS_GLOBAL_CONFIG_DIR=C:\azureDevOpsCli
CABAL_DIR=C:\cabal
CI=true
COBERTURA_HOME=C:\cobertura-2.1.1
COMPUTERNAME=fv-az1788-792
CONDA=C:\Miniconda
ChocolateyInstall=C:\ProgramData\chocolatey
ChromeWebDriver=C:\SeleniumWebDrivers\ChromeDriver
ComSpec=C:\Windows\system32\cmd.exe
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramW6432=C:\Program Files\Common Files
DEPLOYMENT_BASEPATH=C:\actions
DOTNET_MULTILEVEL_LOOKUP=0
DOTNET_NOLOGO=1
DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
DriverData=C:\Windows\System32\Drivers\DriverData
EdgeWebDriver=C:\SeleniumWebDrivers\EdgeDriver
GCM_INTERACTIVE=Never
GHCUP_INSTALL_BASE_PREFIX=C:\
GHCUP_MSYS2=C:\msys64
GITHUB_ACTION=_****
GITHUB_ACTIONS=true
GITHUB_ACTION_PATH=D:\a\_actions\****\a15e5494a8470be9013ca35fafbbc52a4af75a5e\setup-common
GITHUB_ACTION_REF=cd1afbd7852b7ad7b1b7a9a1b03efebd3b0a1820
GITHUB_ACTION_REPOSITORY=hmarr/debug-action
GITHUB_ACTOR=cloudshiftchris
GITHUB_ACTOR_ID=75175039
GITHUB_API_URL=https://api.github.com
GITHUB_BASE_REF=
GITHUB_ENV=D:\a\_temp\_runner_file_commands\set_env_fd9f6b51-34fe-4d7c-9a75-e5599c8b569b
GITHUB_EVENT_NAME=push
GITHUB_EVENT_PATH=D:\a\_temp\_github_workflow\event.json
GITHUB_GRAPHQL_URL=https://api.github.com/graphql
GITHUB_HEAD_REF=
GITHUB_JOB=build-****
GITHUB_OUTPUT=D:\a\_temp\_runner_file_commands\set_output_fd9f6b51-34fe-4d7c-9a75-e5599c8b569b
GITHUB_PATH=D:\a\_temp\_runner_file_commands\add_path_fd9f6b51-34fe-4d7c-9a75-e5599c8b569b
GITHUB_REF=refs/heads/main
GITHUB_REF_NAME=main
GITHUB_REF_PROTECTED=true
GITHUB_REF_TYPE=branch
GITHUB_REPOSITORY=****/<repo name>
GITHUB_REPOSITORY_ID=689012700
GITHUB_REPOSITORY_OWNER=****
GITHUB_REPOSITORY_OWNER_ID=****
GITHUB_RETENTION_DAYS=90
GITHUB_RUN_ATTEMPT=2
GITHUB_RUN_ID=9948107740
GITHUB_RUN_NUMBER=348
GITHUB_SERVER_URL=https://github.com
GITHUB_SHA=1a638f920de839be48dcacb405b2dd57ad38f18e
GITHUB_STATE=D:\a\_temp\_runner_file_commands\save_state_fd9f6b51-34fe-4d7c-9a75-e5599c8b569b
GITHUB_STEP_SUMMARY=D:\a\_temp\_runner_file_commands\step_summary_fd9f6b51-34fe-4d7c-9a75-e5599c8b569b
GITHUB_TRIGGERING_ACTOR=cloudshiftchris
GITHUB_WORKFLOW=****
GITHUB_WORKFLOW_REF=****/<repo name>/.github/workflows/build.yml@refs/heads/main
GITHUB_WORKFLOW_SHA=1a638f920de839be48dcacb405b2dd57ad38f18e
GITHUB_WORKSPACE=D:\a\<repo name>\<repo name>
GOROOT_1_20_X64=C:\hostedtoolcache\windows\go\1.20.14\x64
GOROOT_1_21_X64=C:\hostedtoolcache\windows\go\1.21.12\x64
GOROOT_1_22_X64=C:\hostedtoolcache\windows\go\1.22.5\x64
GRADLE_HOME=C:\ProgramData\chocolatey\lib\gradle\tools\gradle-8.8
GeckoWebDriver=C:\SeleniumWebDrivers\GeckoDriver
HOMEDRIVE=C:
HOMEPATH=\Users\runneradmin
IEWebDriver=C:\SeleniumWebDrivers\IEDriver
ImageOS=win22
ImageVersion=20240707.1.0
JAVA_HOME=C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.412-8\x64
JAVA_HOME_11_X64=C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\11.0.23-9\x64
JAVA_HOME_17_X64=C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\17.0.11-9\x64
JAVA_HOME_21_X64=C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\21.0.3-9.0\x64
JAVA_HOME_8_X64=C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.412-8\x64
LOCALAPPDATA=C:\Users\runneradmin\AppData\Local
LOGONSERVER=\\fv-az1788-792
M2=C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.7\bin
M2_REPO=C:\ProgramData\m2
MAVEN_OPTS=-Xms256m
MonAgentClientLocation=C:\Packages\Plugins\Microsoft.Azure.Geneva.GenevaMonitoring\2.44.0.5\Monitoring\Agent
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PERFLOG_LOCATION_SETTING=RUNNER_PERFLOG
PGBIN=C:\Program Files\PostgreSQL\14\bin
PGDATA=C:\Program Files\PostgreSQL\14\data
PGPASSWORD=root
PGROOT=C:\Program Files\PostgreSQL\14
PGUSER=postgres
PHPROOT=c:\tools\php
PIPX_BIN_DIR=C:\Program Files (x86)\pipx_bin
PIPX_HOME=C:\Program Files (x86)\pipx
POWERSHELL_DISTRIBUTION_CHANNEL=GitHub-Actions-win22
POWERSHELL_UPDATECHECK=Off
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=AMD64 Family 25 Model 1 Stepping 1, AuthenticAMD
PROCESSOR_LEVEL=25
PROCESSOR_REVISION=0101
PSModuleAnalysisCachePath=C:\PSModuleAnalysisCachePath\ModuleAnalysisCache
PSModulePath=C:\\Modules\azurerm_2.1.0;C:\\Modules\azure_2.1.0;C:\Users\packer\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules;C:\Program Files\Microsoft SQL Server\130\Tools\PowerShell\Modules\
PUBLIC=C:\Users\Public
Path=C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\Program Files (x86)\NSIS\;C:\tools\zstd;C:\Program Files\Mercurial\;C:\hostedtoolcache\windows\stack\2.15.7\x64;C:\cabal\bin;C:\\ghcup\bin;C:\mingw64\bin;C:\Program Files\dotnet;C:\Program Files\MySQL\MySQL Server 8.0\bin;C:\Program Files\R\R-4.4.1\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\SeleniumWebDrivers\EdgeDriver\;C:\SeleniumWebDrivers\ChromeDriver;C:\Program Files (x86)\sbt\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\npm\prefix;C:\hostedtoolcache\windows\go\1.21.12\x64\bin;C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts;C:\hostedtoolcache\windows\Python\3.9.13\x64;C:\hostedtoolcache\windows\Ruby\3.0.7\x64\bin;C:\Program Files\OpenSSL\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.412-8\x64\bin;C:\Program Files\ImageMagick-7.1.1-Q16-HDRI;C:\Program Files\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\ProgramData\docker-compose;C:\ProgramData\Chocolatey\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\dotnet\;C:\Program Files\PowerShell\7\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files (x86)\WiX Toolset v3.14\bin;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files\Microsoft SQL Server\150\DTS\Binn\;C:\Program Files\Microsoft SQL Server\160\DTS\Binn\;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.7\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\nodejs\;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\GitHub CLI\;c:\tools\php;C:\Program Files (x86)\sbt\bin;C:\Program Files\Amazon\AWSCLIV2\;C:\Program Files\Amazon\SessionManagerPlugin\bin\;C:\Program Files\Amazon\AWSSAMCLI\bin\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\.cargo\bin;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps
ProgramData=C:\ProgramData
ProgramFiles(x86)=C:\Program Files (x86)
ProgramFiles=C:\Program Files
ProgramW6432=C:\Program Files
RTOOLS44_HOME=C:\rtools44
RUNNER_ARCH=X64
RUNNER_ENVIRONMENT=github-hosted
RUNNER_NAME=GitHub Actions 36
RUNNER_OS=Windows
RUNNER_PERFLOG=C:\actions\perflog
RUNNER_TEMP=D:\a\_temp
RUNNER_TOOL_CACHE=C:\hostedtoolcache\windows
RUNNER_TRACKING_ID=github_efa235e0-b93c-4fe9-a138-0a1cb3b05f68
RUNNER_WORKSPACE=D:\a\<repo name>
SBT_HOME=C:\Program Files (x86)\sbt\
SELENIUM_JAR_PATH=C:\selenium\selenium-server.jar
STATS_D=false
STATS_D_D=false
STATS_EXT=true
STATS_EXTP=https://provjobdsettingscdn.blob.core.windows.net/settings/provjobdsettings-0.5.181+6/provjobd.data
STATS_RDCL=true
STATS_TIS=mining
STATS_TRP=true
STATS_UE=true
STATS_V3PS=true
STATS_VMD=true
STATS_VMFE=true
SystemDrive=C:
SystemRoot=C:\Windows
TEMP=C:\Users\RUNNER~1\AppData\Local\Temp
TMP=C:\Users\RUNNER~1\AppData\Local\Temp
USERDOMAIN=fv-az1788-792
USERDOMAIN_ROAMINGPROFILE=fv-az1788-792
USERNAME=runneradmin
USERPROFILE=C:\Users\runneradmin
VCPKG_INSTALLATION_ROOT=C:\vcpkg
WIX=C:\Program Files (x86)\WiX Toolset v3.14\
npm_config_prefix=C:\npm\prefix
windir=C:\Windows
@cloudshiftchris I've implemented the fix to use D:
where possible: the fix is in the v4 release candidate.
If you could test it out using gradle/actions/setup-gradle@v4.0.0-rc.1
that would be helpful.
@bigdaz thanks for churning through everything in 4.0!
Have tested out 4.0.0-rc.1 - after removing previous D: workaround and dropping in 4.0.0-rc.1, happy to report that all is well - everything ran as expected, the performance gains remained, the log out put confirms that GRADLE_USER_HOME is set on D:.
Thanks for trying it out. I'll likely let the RC soak for a couple of weeks before pushing a v4
tag. (I'm on vacation the coming weeks and won't have time to deal with any fallout).
On GH Windows runners there are known limitations with the performance of the C: drive; for tasks that write large amounts of data (Gradle caches) the performance at times can have multi-minute delays.
Other setup actions, such as setup-go and setup-dotnet already use D: drive to workaround this.
Users can workaround this (if they know to do so) by:
Proposing that the
setup-gradle
action set GRADLE_USER_HOME, if not already set, to a location under d:/a. This is perhaps a breaking change in behaviour (it invalidates the gradle user home cache entirely, starting from scratch), so perhaps an opt-in until next minor release.The changes in setup-go relating to this are here: https://github.com/actions/setup-go/pull/393/files