Size limits?

[jwalton]

Are there size limits on artifacts? Are there quotas for total number of artifacts? Do these count towards my 100GB hard limit on the size of my repo? (And, if you know, a similar question about pushing docker images to Github package registry?)

I'm looking to port a buildkite build to Github Actions - at the moment it generates some largish artifacts when it archives node_modules (to save time doing npm ci from one parallel task to the next). I was thinking about pushing a dev docker image instead, but right now our image is an astonishing 1G, which seems like an excessively large thing to push to either artifacts or to a package registry on each build. :P

[Godin]

FWIW this is quite interesting question, especially given that there is no way to remove uploaded artifacts (#5).

[GXTX]

Max upload size is 5GB due to a bug when chunking, unsure if it counts towards hard limit as I stopped at 55GB uploaded.

It would be nice to have an official answer so I don't have to risk my account testing.

[madhavtummala]

I tried to upload a 2gb file. I breaks it into chunks and fails multiple times trying to upload. This is bad.

PS: The upload file is ubuntu-19 image file

[pixsaoul]

Is there an official information concerning the total amount of artifact uploaded ?

[jens-maus]

I would also be interested in getting some official information on the size limits of uploading artifcats in GitHub Actions as one of my projects also just outputed a nice Unhandled exception. System.IO.IOException: No space left on device when using actions/upload-artifact.

See here: https://github.com/jens-maus/RaspberryMatic/actions/runs/37347250

[jens-maus]

BTW: There seems to be a new API call to programmatically delete artifacts. See here: https://github.community/t5/GitHub-Actions/Delete-artifacts/m-p/44548#M5818

Are there any plans on enhancing "actions/upload-artifact" to directly delete old artifacts which would help a lot for managing nightly build retention schemes.

[jens-maus]

Please note. I just adapted a nice gist provided helper script from @lelegard (https://gist.github.com/lelegard/6a428f67ee08e86d0c2f1af3f4a633d0) for my own purposes and added it to my github workflows so that the runners itself are purging old artifacts during its initialization (see https://github.com/jens-maus/RaspberryMatic/blob/master/.github/workflows/purge_artifacts.sh)

[jens-maus]

Hmmm, strange enough I am still getting the mentioned "Unhandled exception" excpetion for one of my build jobs even thought I cleared out all build artifcats. See here:

Unhandled exception. System.IO.IOException: No space left on device
   at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
   at System.IO.FileStream.FlushWriteBuffer()
   at System.IO.FileStream.Flush(Boolean flushToDisk)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at System.Diagnostics.TraceSource.Flush()
   at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
   at GitHub.Runner.Common.TraceManager.Dispose()
   at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
   at GitHub.Runner.Common.HostContext.Dispose()
   at GitHub.Runner.Worker.Program.Main(String[] args)
System.IO.IOException: No space left on device
   at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
   at System.IO.FileStream.FlushWriteBuffer()
   at System.IO.FileStream.Flush(Boolean flushToDisk)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
System.IO.IOException: No space left on device
   at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
   at System.IO.FileStream.FlushWriteBuffer()
   at System.IO.FileStream.Flush(Boolean flushToDisk)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Common.Tracing.Error(Exception exception)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)

See here for more details: https://github.com/jens-maus/RaspberryMatic/actions/runs/37920129

The other 5 build jobs could perfectly upload the artifact. Its just that single "ova" build job that fails with that strange No space left on device exception.

Does anyone has an idea where this disk space exception is coming from?!? To me this currently looks like a github issue, isn't it?

[yacaovsnc]

@jens-maus is this hosted runner? If not, could you please verify how much space is left on the runner device? The stacktrace seems to indicate we failed to write trace events locally.

[jens-maus]

@yacaovsnc Nope, this is not a self-hosted runner but uses githubs' own runners. And I also tried to verify if there isn't enough space left after the build process, but this isn't the case. There seem to be enough space left on the devices within the runner. So I still don't know why these No space left on device errors happen. See here for some recent error output, for example:

https://github.com/jens-maus/RaspberryMatic/actions/runs/43667982

Hope that this helps to investigate the issue...

[tarunmangukiya]

Facing same issues with simple html files too. I think this isn't related to the size of the files!

[thisismydesign]

Our team has created an Action to periodically clean up old artifacts: https://github.com/c-hive/gha-remove-artifacts

It also supports keeping tagged (release) artifacts.

[ahdbilal]

You can now set artifact and log retention duration to less than 90 days(minimum of 1 day) and we will clean and delete artifacts right after they expire. You can learn more about this feature here. Feedback is welcome!

[chrisjsewell]

Max upload size is 5GB due to a bug when chunking I tried to upload a 2gb file. I breaks it into chunks and fails multiple times trying to upload.

Is this a known bug then? Should it be raised in a separate issue? I was trying to upload an archive of a few Gb, but it always fails on the final chunk

Uploading /Users/runner/work/quantum-mobile/quantum-mobile/dist/quantum_mobile_20.06.1.ova (99.8%)
Error: Unexpected response. Unable to upload chunk to pipelines.actions.githubusercontent.com/yExJFZLG3LmnOuH7nWUxsQVTOQFMk9X0eyXPOmEUS8Lzfm9DD5/_apis/resources/Containers/343210?itemPath=distribution%2Fquantum_mobile_20.06.1.ova
##### Begin Diagnostic HTTP information #####
Status Code: 500
Status Message: Internal Server Error
Header Information: {
  "cache-control": "no-store,no-cache",
  "pragma": "no-cache",
  "content-length": "265",
  "content-type": "application/json; charset=utf-8",
  "strict-transport-security": "max-age=2592000",
  "x-tfs-processid": "826003cc-287e-4561-beeb-c63f5ce49acb",
  "activityid": "ff26414b-5c0b-4b3b-87b7-3f463a4639b4",
  "x-tfs-session": "ff26414b-5c0b-4b3b-87b7-3f463a4639b4",
  "x-vss-e2eid": "ff26414b-5c0b-4b3b-87b7-3f463a4639b4",
  "x-vss-senderdeploymentid": "a07ab14e-025a-39c3-8d53-788cd7ce207f",
  "x-frame-options": "SAMEORIGIN",
  "x-msedge-ref": "Ref A: 298EE8828CBE4D23838DD84E02073233 Ref B: ATAEDGE1718 Ref C: 2020-10-10T00:14:26Z",
  "date": "Sat, 10 Oct 2020 00:14:32 GMT"
}

(Edit: I see now there is a closed issue https://github.com/actions/upload-artifact/issues/29)

[heavelock]

Is there also some kind of a chunking in case of directory-as-artifact?

I uploaded a docs html as artifact and interface shows the size as 300MB. However, the downloaded zip is only 44.1MB and lacks a lot of files. Checked on two different jobs with multiple download attempts.

See here: https://github.com/obspy/obspy/actions/runs/568966900

[lhl2617]

@heavelock I'm facing the same issue as you and created #190

[Zibri]

I wonder what is the total disk quota for RELEASES + artifacts. (the default free plan I mean)

Anyway to be on the safe side I have a github action to remove everything :D

  #!/bin/bash
  for a in $(curl -qs "https://api.github.com/repos/Zibri/VICE/actions/runs"|jq -r .workflow_runs[].id); do 
  curl   -X DELETE   -H "Accept: application/vnd.github.v3+json"  \
  -H "Authorization: token TOKEN " \
  "https://api.github.com/repos/OWNER/REPOSITORY/actions/runs/$a";
  done

you can further customize it by using selectors in "jq" command (if needed: pip install jq)