actions / cache

Cache dependencies and build outputs in GitHub Actions
MIT License
4.39k stars 1.17k forks source link

Post cache cleanup persistently cancels operation at cache-miss failing the whole workflow #1209

Closed LuWang1983 closed 1 year ago

LuWang1983 commented 1 year ago

Hi there! I took some deep dive investigating a cache issue in our CICD pipeline and concluded that I may have to resort to you for this issue as I don't think it's anything we did on our end that caused it.

This issue is currently blocking our CICD pipeline as it fails persistently at cache-miss but if I re-run a failed workflow, sometimes it passes but only when it's cache-hit. Although logically, cache-miss shouldn't matter, it should just create a new cache when we install the dependencies but post cache cleanup doesn't seem to like it and cancels the operation failing the whole workflow.

Before we resolve this issue, any workaround would be greatly appreciated to unblock our pipeline. Would clearing all caches in the repo and create new caches help? We use dind so we have our own container running in the VM container (We use debian). Thank you so much for your help and please let me know if there's any other data you need from us!

Symptoms

Every time there's a cache-miss, post cache cleanup gets errored out with "operation cancelled" failing the whole workflow. Please see debugging logs I saved below.

5_Cache yarn node modules step:

2023-07-20T05:06:38.6401735Z ##[debug]Evaluating condition for step: 'Cache yarn node modules'
2023-07-20T05:06:38.6402963Z ##[debug]Evaluating: success()
2023-07-20T05:06:38.6403282Z ##[debug]Evaluating success:
2023-07-20T05:06:38.6403637Z ##[debug]=> true
2023-07-20T05:06:38.6403971Z ##[debug]Result: true
2023-07-20T05:06:38.6404472Z ##[debug]Starting: Cache yarn node modules
2023-07-20T05:06:38.6415030Z ##[debug]Register post job cleanup for action: actions/cache@v3
2023-07-20T05:06:38.6424462Z ##[debug]Loading inputs
2023-07-20T05:06:38.6426882Z ##[debug]Evaluating: format('{0}-storage-host-yarn-{1}', runner.os, hashFiles('yarn.lock'))
2023-07-20T05:06:38.6427215Z ##[debug]Evaluating format:
2023-07-20T05:06:38.6427445Z ##[debug]..Evaluating String:
2023-07-20T05:06:38.6427707Z ##[debug]..=> '{0}-storage-host-yarn-{1}'
2023-07-20T05:06:38.6428181Z ##[debug]..Evaluating Index:
2023-07-20T05:06:38.6428409Z ##[debug]....Evaluating runner:
2023-07-20T05:06:38.6428654Z ##[debug]....=> Object
2023-07-20T05:06:38.6428885Z ##[debug]....Evaluating String:
2023-07-20T05:06:38.6429148Z ##[debug]....=> 'os'
2023-07-20T05:06:38.6429419Z ##[debug]..=> 'Linux'
2023-07-20T05:06:38.6429699Z ##[debug]..Evaluating hashFiles:
2023-07-20T05:06:38.6466059Z ##[debug]....Evaluating String:
2023-07-20T05:06:38.6466344Z ##[debug]....=> 'yarn.lock'
2023-07-20T05:06:38.6466844Z ##[debug]Search root directory: '/home/runner/work/castify/castify'
2023-07-20T05:06:38.6467161Z ##[debug]Search pattern: 'yarn.lock'
2023-07-20T05:06:38.6469245Z ##[debug]Starting process:
2023-07-20T05:06:38.6469598Z ##[debug]  File name: '/home/runner/runners/2.306.0/externals/node16/bin/node'
2023-07-20T05:06:38.6469934Z ##[debug]  Arguments: '"/home/runner/runners/2.306.0/bin/hashFiles"'
2023-07-20T05:06:38.6470254Z ##[debug]  Working directory: '/home/runner/work/castify/castify'
2023-07-20T05:06:38.6470548Z ##[debug]  Require exit code zero: 'False'
2023-07-20T05:06:38.6470807Z ##[debug]  Encoding web name:  ; code page: ''
2023-07-20T05:06:38.6471104Z ##[debug]  Force kill process on cancellation: 'False'
2023-07-20T05:06:38.6471380Z ##[debug]  Redirected STDIN: 'False'
2023-07-20T05:06:38.6471627Z ##[debug]  Persist current code page: 'False'
2023-07-20T05:06:38.6471895Z ##[debug]  Keep redirected STDIN open: 'False'
2023-07-20T05:06:38.6472157Z ##[debug]  High priority process: 'False'
2023-07-20T05:06:38.6477182Z ##[debug]Updated oom_score_adj to 500 for PID: 2144.
2023-07-20T05:06:38.6477701Z ##[debug]Process started with process id 2144, waiting for process exit.
2023-07-20T05:06:38.7039782Z ##[debug]Match Pattern: yarn.lock
2023-07-20T05:06:38.7058893Z ##[debug]::debug::followSymbolicLinks 'false'
2023-07-20T05:06:38.7090602Z ##[debug]::debug::followSymbolicLinks 'false'
2023-07-20T05:06:38.7092528Z ##[debug]::debug::implicitDescendants 'true'
2023-07-20T05:06:38.7092896Z ##[debug]::debug::omitBrokenSymbolicLinks 'true'
2023-07-20T05:06:38.7101634Z ##[debug]::debug::Search path '/home/runner/work/castify/castify/yarn.lock'
2023-07-20T05:06:38.7129456Z ##[debug]/home/runner/work/castify/castify/yarn.lock
2023-07-20T05:06:38.7260793Z ##[debug]Found 1 files to hash.
2023-07-20T05:06:38.7266656Z ##[debug]Hash result: '04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:06:38.7273215Z ##[debug]undefined
2023-07-20T05:06:38.7342563Z ##[debug]STDOUT/STDERR stream read finished.
2023-07-20T05:06:38.7343152Z ##[debug]STDOUT/STDERR stream read finished.
2023-07-20T05:06:38.7344082Z ##[debug]Finished process 2144 with exit code 0, and elapsed time 00:00:00.0871057.
2023-07-20T05:06:38.7344917Z ##[debug]..=> '04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:06:38.7345714Z ##[debug]=> 'Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:06:38.7346378Z ##[debug]Result: 'Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:06:38.7347487Z ##[debug]Evaluating: format('{0}-storage-host-yarn-
2023-07-20T05:06:38.7348066Z ##[debug]', runner.os)
2023-07-20T05:06:38.7348513Z ##[debug]Evaluating format:
2023-07-20T05:06:38.7348738Z ##[debug]..Evaluating String:
2023-07-20T05:06:38.7348982Z ##[debug]..=> '{0}-storage-host-yarn-
2023-07-20T05:06:38.7349187Z ##[debug]'
2023-07-20T05:06:38.7349486Z ##[debug]..Evaluating Index:
2023-07-20T05:06:38.7349708Z ##[debug]....Evaluating runner:
2023-07-20T05:06:38.7349951Z ##[debug]....=> Object
2023-07-20T05:06:38.7350178Z ##[debug]....Evaluating String:
2023-07-20T05:06:38.7350382Z ##[debug]....=> 'os'
2023-07-20T05:06:38.7350729Z ##[debug]..=> 'Linux'
2023-07-20T05:06:38.7350984Z ##[debug]=> 'Linux-storage-host-yarn-
2023-07-20T05:06:38.7351194Z ##[debug]'
2023-07-20T05:06:38.7351424Z ##[debug]Result: 'Linux-storage-host-yarn-
2023-07-20T05:06:38.7351631Z ##[debug]'
2023-07-20T05:06:38.7353799Z ##[debug]Loading env
2023-07-20T05:06:38.7358036Z ##[group]Run actions/cache@v3
2023-07-20T05:06:38.7358219Z with:
2023-07-20T05:06:38.7358474Z   path: node_modules
!node_modules/.cache
~/.cache
~/.ci-cache-*

2023-07-20T05:06:38.7358927Z   key: Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e
2023-07-20T05:06:38.7359344Z   restore-keys: Linux-storage-host-yarn-

2023-07-20T05:06:38.7359598Z   enableCrossOsArchive: false
2023-07-20T05:06:38.7359832Z   fail-on-cache-miss: false
2023-07-20T05:06:38.7360046Z   lookup-only: false
2023-07-20T05:06:38.7360229Z ##[endgroup]
2023-07-20T05:06:38.7363133Z ##[command]/usr/bin/docker exec  5c29577b71b29c3a91494cfae67ad199e64e6fef0977c53f95505fefcbcd91be sh -c "cat /etc/*release | grep ^ID"
2023-07-20T05:06:38.8175732Z ##[debug]ID=debian
2023-07-20T05:06:38.8176074Z ##[debug]Running JavaScript Action with default external tool: node16
2023-07-20T05:06:39.0345339Z ##[debug]Resolved Keys:
2023-07-20T05:06:39.0351627Z ##[debug]["Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e","Linux-storage-host-yarn-"]
2023-07-20T05:06:39.0357244Z ##[debug]Checking zstd --quiet --version
2023-07-20T05:06:39.0427831Z ##[debug]Unable to locate executable file: zstd. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-07-20T05:06:39.0433542Z ##[debug]
2023-07-20T05:06:39.0442286Z ##[debug]zstd version: null
2023-07-20T05:06:39.0458476Z ##[debug]Resource Url: https://artifactcache.actions.githubusercontent.com/rpb3cHBtlDJ1vrVvbGJM7rAeOhBuka2z2j4L7qNC6oGjyZIEfx/_apis/artifactcache/cache?keys=Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e%2CLinux-storage-host-yarn-&version=5f46fbd1755235cf014663450000ef2ac971f679cfef790c2ef56617a5213896
2023-07-20T05:06:39.1826705Z ::add-mask::***
2023-07-20T05:06:39.1828804Z ##[debug]Cache Result:
2023-07-20T05:06:39.1830415Z ##[debug]{"scope":"refs/heads/main","cacheKey":"Linux-storage-host-yarn-a0c56a88571454c8c28c0e0cdf2ccfb13715a67dfa4fe3ad16c32cc1b252d282","cacheVersion":"5f46fbd1755235cf014663450000ef2ac971f679cfef790c2ef56617a5213896","creationTime":"2023-06-21T17:56:54.2433333Z","archiveLocation":"***"}
2023-07-20T05:06:39.1844183Z ##[debug]Archive Path: /__w/_temp/efa7e828-2f6b-4145-8d33-fde60a78a2ea/cache.tgz
2023-07-20T05:06:39.1848121Z ##[debug]Use Azure SDK: true
2023-07-20T05:06:39.1848542Z ##[debug]Download concurrency: 8
2023-07-20T05:06:39.1852533Z ##[debug]Request timeout (ms): 30000
2023-07-20T05:06:39.1853035Z ##[debug]Cache segment download timeout mins env var: undefined
2023-07-20T05:06:39.1853514Z ##[debug]Segment download timeout (ms): 600000
2023-07-20T05:06:39.1853922Z ##[debug]Lookup only: false
...
2023-07-20T05:08:04.0683856Z Cache Size: ~2038 MB (2137044058 B)
2023-07-20T05:08:04.0684183Z [command]/bin/tar -xf /__w/_temp/efa7e828-2f6b-4145-8d33-fde60a78a2ea/cache.tgz -P -C /__w/castify/castify -z
2023-07-20T05:09:26.9155226Z Cache restored successfully
2023-07-20T05:09:27.0383979Z Cache restored from key: Linux-storage-host-yarn-a0c56a88571454c8c28c0e0cdf2ccfb13715a67dfa4fe3ad16c32cc1b252d282
2023-07-20T05:09:27.0592960Z ##[debug]Node Action run completed with exit code 0
2023-07-20T05:09:27.0595196Z ##[debug]Save intra-action state CACHE_KEY = Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e
2023-07-20T05:09:27.0596067Z ##[debug]Save intra-action state CACHE_RESULT = Linux-storage-host-yarn-a0c56a88571454c8c28c0e0cdf2ccfb13715a67dfa4fe3ad16c32cc1b252d282
2023-07-20T05:09:27.0621006Z ##[debug]Set output cache-hit = false
2023-07-20T05:09:27.0621703Z ##[debug]Finishing: Cache yarn node modules

6_Install node dependencies if not cache hit - installing node modules and a few other steps that all passed

22_Post Cache yarn node modules: See error towards the bottom of the logs where it says "The operation was canceled." I did notice that there's a message above it "Unable to locate executable file: zstd. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable."

2023-07-20T05:11:04.1849561Z ##[debug]Evaluating condition for step: 'Post Cache yarn node modules'
2023-07-20T05:11:04.1850675Z ##[debug]Evaluating: success()
2023-07-20T05:11:04.1850992Z ##[debug]Evaluating success:
2023-07-20T05:11:04.1851338Z ##[debug]=> true
2023-07-20T05:11:04.1851681Z ##[debug]Result: true
2023-07-20T05:11:04.1852175Z ##[debug]Starting: Post Cache yarn node modules
2023-07-20T05:11:04.1869363Z ##[debug]Loading inputs
2023-07-20T05:11:04.1870634Z ##[debug]Evaluating: format('{0}-storage-host-yarn-{1}', runner.os, hashFiles('yarn.lock'))
2023-07-20T05:11:04.1870966Z ##[debug]Evaluating format:
2023-07-20T05:11:04.1871204Z ##[debug]..Evaluating String:
2023-07-20T05:11:04.1871467Z ##[debug]..=> '{0}-storage-host-yarn-{1}'
2023-07-20T05:11:04.1871780Z ##[debug]..Evaluating Index:
2023-07-20T05:11:04.1872002Z ##[debug]....Evaluating runner:
2023-07-20T05:11:04.1872234Z ##[debug]....=> Object
2023-07-20T05:11:04.1872464Z ##[debug]....Evaluating String:
2023-07-20T05:11:04.1872670Z ##[debug]....=> 'os'
2023-07-20T05:11:04.1872940Z ##[debug]..=> 'Linux'
2023-07-20T05:11:04.1873168Z ##[debug]..Evaluating hashFiles:
2023-07-20T05:11:04.1873416Z ##[debug]....Evaluating String:
2023-07-20T05:11:04.1873630Z ##[debug]....=> 'yarn.lock'
2023-07-20T05:11:04.1873998Z ##[debug]Search root directory: '/home/runner/work/castify/castify'
2023-07-20T05:11:04.1874293Z ##[debug]Search pattern: 'yarn.lock'
2023-07-20T05:11:04.1874736Z ##[debug]Starting process:
2023-07-20T05:11:04.1875026Z ##[debug]  File name: '/home/runner/runners/2.306.0/externals/node16/bin/node'
2023-07-20T05:11:04.1875362Z ##[debug]  Arguments: '"/home/runner/runners/2.306.0/bin/hashFiles"'
2023-07-20T05:11:04.1875684Z ##[debug]  Working directory: '/home/runner/work/castify/castify'
2023-07-20T05:11:04.1876101Z ##[debug]  Require exit code zero: 'False'
2023-07-20T05:11:04.1876369Z ##[debug]  Encoding web name:  ; code page: ''
2023-07-20T05:11:04.1876654Z ##[debug]  Force kill process on cancellation: 'False'
2023-07-20T05:11:04.1876918Z ##[debug]  Redirected STDIN: 'False'
2023-07-20T05:11:04.1877191Z ##[debug]  Persist current code page: 'False'
2023-07-20T05:11:04.1877467Z ##[debug]  Keep redirected STDIN open: 'False'
2023-07-20T05:11:04.1877736Z ##[debug]  High priority process: 'False'
2023-07-20T05:11:04.1916126Z ##[debug]Updated oom_score_adj to 500 for PID: 3157.
2023-07-20T05:11:04.1916997Z ##[debug]Process started with process id 3157, waiting for process exit.
2023-07-20T05:11:04.2461789Z ##[debug]Match Pattern: yarn.lock
2023-07-20T05:11:04.2472092Z ##[debug]::debug::followSymbolicLinks 'false'
2023-07-20T05:11:04.2502844Z ##[debug]::debug::followSymbolicLinks 'false'
2023-07-20T05:11:04.2504372Z ##[debug]::debug::implicitDescendants 'true'
2023-07-20T05:11:04.2505512Z ##[debug]::debug::omitBrokenSymbolicLinks 'true'
2023-07-20T05:11:04.2513432Z ##[debug]::debug::Search path '/home/runner/work/castify/castify/yarn.lock'
2023-07-20T05:11:04.2555114Z ##[debug]/home/runner/work/castify/castify/yarn.lock
2023-07-20T05:11:04.2685264Z ##[debug]Found 1 files to hash.
2023-07-20T05:11:04.2687738Z ##[debug]Hash result: '04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:11:04.2692084Z ##[debug]undefined
2023-07-20T05:11:04.2713217Z ##[debug]STDOUT/STDERR stream read finished.
2023-07-20T05:11:04.2713399Z ##[debug]STDOUT/STDERR stream read finished.
2023-07-20T05:11:04.2716189Z ##[debug]Finished process 3157 with exit code 0, and elapsed time 00:00:00.0837545.
2023-07-20T05:11:04.2720798Z ##[debug]..=> '04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:11:04.2721400Z ##[debug]=> 'Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:11:04.2722045Z ##[debug]Result: 'Linux-storage-host-yarn-04d1fa0cac8a12f0fbf0f0f1b3b74c72b7ab091f03f417f62586433aa0be9e7e'
2023-07-20T05:11:04.2723051Z ##[debug]Evaluating: format('{0}-storage-host-yarn-
2023-07-20T05:11:04.2723302Z ##[debug]', runner.os)
2023-07-20T05:11:04.2723531Z ##[debug]Evaluating format:
2023-07-20T05:11:04.2724139Z ##[debug]..Evaluating String:
2023-07-20T05:11:04.2724397Z ##[debug]..=> '{0}-storage-host-yarn-
2023-07-20T05:11:04.2724614Z ##[debug]'
2023-07-20T05:11:04.2724866Z ##[debug]..Evaluating Index:
2023-07-20T05:11:04.2725098Z ##[debug]....Evaluating runner:
2023-07-20T05:11:04.2725334Z ##[debug]....=> Object
2023-07-20T05:11:04.2725565Z ##[debug]....Evaluating String:
2023-07-20T05:11:04.2725884Z ##[debug]....=> 'os'
2023-07-20T05:11:04.2726125Z ##[debug]..=> 'Linux'
2023-07-20T05:11:04.2726376Z ##[debug]=> 'Linux-storage-host-yarn-
2023-07-20T05:11:04.2726589Z ##[debug]'
2023-07-20T05:11:04.2726822Z ##[debug]Result: 'Linux-storage-host-yarn-
2023-07-20T05:11:04.2727043Z ##[debug]'
2023-07-20T05:11:04.2729172Z ##[debug]Loading env
2023-07-20T05:11:04.2733260Z Post job cleanup.
2023-07-20T05:11:04.2736254Z ##[command]/usr/bin/docker exec  5c29577b71b29c3a91494cfae67ad199e64e6fef0977c53f95505fefcbcd91be sh -c "cat /etc/*release | grep ^ID"
2023-07-20T05:11:04.3581674Z ##[debug]ID=debian
2023-07-20T05:11:04.3582023Z ##[debug]Running JavaScript Action with default external tool: node16
2023-07-20T05:11:04.5855663Z ##[debug]Cache state/key: Linux-storage-host-yarn-a0c56a88571454c8c28c0e0cdf2ccfb13715a67dfa4fe3ad16c32cc1b252d282
2023-07-20T05:11:04.5939723Z ##[debug]Checking zstd --quiet --version
2023-07-20T05:11:04.5968981Z ##[debug]Unable to locate executable file: zstd. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-07-20T05:11:04.5969617Z ##[debug]
2023-07-20T05:11:04.5976126Z ##[debug]zstd version: null
2023-07-20T05:11:04.5983359Z ##[debug]implicitDescendants 'false'
2023-07-20T05:11:04.6019749Z ##[debug]followSymbolicLinks 'true'
2023-07-20T05:11:04.6020232Z ##[debug]implicitDescendants 'false'
2023-07-20T05:11:04.6023518Z ##[debug]omitBrokenSymbolicLinks 'true'
2023-07-20T05:11:04.6024944Z ##[debug]Search path '/__w/castify/castify/node_modules'
2023-07-20T05:11:04.6031924Z ##[debug]Search path '/github/home'
2023-07-20T05:11:04.6042461Z ##[debug]Matched: node_modules
2023-07-20T05:11:04.6484937Z ##[debug]Matched: ../../../github/home/.cache
2023-07-20T05:11:04.6501480Z ##[debug]Matched: ../../../github/home/.ci-cache-yarn
2023-07-20T05:11:04.6508361Z ##[debug]Cache Paths:
2023-07-20T05:11:04.6513132Z ##[debug]["node_modules","../../../github/home/.cache","../../../github/home/.ci-cache-yarn"]
2023-07-20T05:11:04.6525896Z ##[debug]Archive Path: /__w/_temp/64656404-8155-4579-8d53-7feb632cf9cc/cache.tgz
2023-07-20T05:11:04.6558476Z [command]/bin/tar --posix -cf cache.tgz --exclude cache.tgz -P -C /__w/castify/castify --files-from manifest.txt -z
2023-07-20T05:15:38.4562979Z ##[debug]Re-evaluate condition on job cancellation for step: 'Post Cache yarn node modules'.
2023-07-20T05:15:38.4695455Z ##[error]The operation was canceled.
2023-07-20T05:15:38.4705781Z ##[debug]System.OperationCanceledException: The operation was canceled.
2023-07-20T05:15:38.4706284Z ##[debug]   at System.Threading.CancellationToken.ThrowOperationCanceledException()
2023-07-20T05:15:38.4707225Z ##[debug]   at GitHub.Runner.Sdk.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
2023-07-20T05:15:38.4708529Z ##[debug]   at GitHub.Runner.Common.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
2023-07-20T05:15:38.4732781Z ##[debug]   at GitHub.Runner.Worker.Handlers.ContainerStepHost.ExecuteAsync(IExecutionContext context, String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, String standardInInput, CancellationToken cancellationToken)
2023-07-20T05:15:38.4733973Z ##[debug]   at GitHub.Runner.Worker.Handlers.NodeScriptActionHandler.RunAsync(ActionRunStage stage)
2023-07-20T05:15:38.4734534Z ##[debug]   at GitHub.Runner.Worker.ActionRunner.RunAsync()
2023-07-20T05:15:38.4734989Z ##[debug]   at GitHub.Runner.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
LuWang1983 commented 1 year ago

Btw, I cleared all the caches and triggered a manual run on our Lint Test Build CI job. It failed with the same exact logs but didn't create caches. According to this GHA documentation, cache-miss will automatically create a new cache if the job succeeds. I guess because it failed at post cache cleanup, no new cache is created. Below is the step in our workflow yaml file.

...(omitting for brevity)
runs-on: ubuntu-20.04
    steps:
      - name: Checkout repo
        uses: actions/checkout@v3

      - name: Cache node modules
        uses: actions/cache@v3
        id: yarn-cache
        with:
          # Note: cache does not respect default work-directory,
          # cache node_modules to speed up ngcc (executed as postinstall step on yarn install)
          path: |
            node_modules
            !node_modules/.cache
            ~/.cache
            ~/.ci-cache-*
          key: ${{ runner.os }}-storage-host-yarn-${{ hashFiles('yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-storage-host-yarn-

      - name: Install dependencies
        if: ${{ steps.yarn-cache.outputs.cache-hit }} != 'true'
        run: yarn --frozen-lockfile --non-interactive
        shell: bash
LuWang1983 commented 1 year ago

We are also intermittently running into no disk space errors and it may be related.

LuWang1983 commented 1 year ago

I added the dependency zstd in our job runner docker image to rule out that it was the missing dependency that caused the operation to be canceled. It failed with the same error FYI.

LuWang1983 commented 1 year ago

Closing issue. Github Support helped me resolved it.

shinji62 commented 11 months ago

@LuWang1983 What was the issue ? I am facing something similar.

LuWang1983 commented 11 months ago

For us, it timed out so we needed to increase the timeout minutes. It doesn't really tell you what caused the cancellation in logs but if you go to the summary page, it gives you the reason. The summary page usually gives pretty good context of the error. I just didn't look closely.