search

nektos / act

Run your GitHub Actions locally 🚀
https://nektosact.com
MIT License
55.1k stars 1.38k forks source link

Slim variant of nektos/act-environments-ubuntu:18.04 #196

Closed aidansteele closed 4 years ago

aidansteele commented 4 years ago

My feeling reading some of the issues in this repo is that a lot of folks get stuck on the fact that we default to not using the enormous image - which is a reasonable IMHO. I've been thinking about whether there's a compromise we can reach, where we get e.g. 90% of the benefit at 30% of the size.

There would be pros and cons to this approach. Pros would likely be fewer issues on this repo, an easier getting-started process for most people, etc. Cons would be that the environment would be so close to the Github-provided one, but crucially not identical - which would cause a lot of head-scratching. Disregarding this for now, I started to take a cursory look at how we might want to shrink the image size.

All these numbers are on the uncompressed (~18GB total) image size:

What are your thoughts on this selective purge? I'm thinking that we'd likely want to automate the creation of the image + purging as well. If you're happy with the idea in general, I'll start experimenting with some approaches.

cplee commented 4 years ago

@aidansteele I'm totally onboard with this idea 🚀

cplee commented 4 years ago

btw - here's the repo for the image i built: https://github.com/nektos/act-environments

aidansteele commented 4 years ago

Great, good to hear you think it's a reasonable idea. I'll try report back with my findings / PR in the next few days - week.

phaze9 commented 4 years ago

I took the opposite approach by starting with act's default node:12.6-buster-slim image and adding what I need, which is currently docker.

If you're interested in creating a slim image tailored to your use-case, feel free to fork https://github.com/phaze9/action-runner Or grab the slim image including docker-ce from https://hub.docker.com/repository/docker/phaze9/action-runner

aidansteele commented 4 years ago

I got a chance today to start looking into this. I created a tool to convert the upstream ubuntu1804.json into a Dockerfile (I'll release that later) and attempted to build it. A few things:

The total size is now just under 50 GB 🤯

It looks like some new things have been added to the environment since @cplee's image was built.

There are some steps I skipped as they failed and I didn't have time to look into why yet. The skipped steps were:

These were all the steps that resulted in a layer on disk greater than 50 MB.

SIZE CREATED BY
20.4 GB scripts/installers/dotnetcore-sdk.sh
9.3 GB scripts/installers/haskell.sh
7.14 GB scripts/installers/hosted-tool-cache.sh
1.6 GB scripts/installers/swift.sh
1.37 GB scripts/installers/java-tools.sh
1.37 GB scripts/installers/clang.sh
921 MB scripts/installers/azpowershell.sh
527 MB scripts/installers/mono.sh
503 MB scripts/installers/rust.sh
500 MB scripts/installers/azure-cli.sh
382 MB scripts/installers/miniconda.sh
367 MB scripts/installers/nodejs.sh
366 MB scripts/installers/julia.sh
364 MB scripts/installers/google-cloud-sdk.sh
302 MB scripts/installers/mongodb.sh
287 MB scripts/installers/hhvm.sh
271 MB scripts/installers/bazel.sh
260 MB scripts/installers/google-chrome.sh
203 MB scripts/installers/1804/basic.sh
202 MB scripts/installers/1804/basic.sh
190 MB scripts/installers/1804/containers.sh
164 MB scripts/installers/powershellcore.sh
162 MB scripts/installers/powershellcore.sh
155 MB scripts/installers/packer.sh
155 MB scripts/installers/azcopy.sh
130 MB scripts/installers/python.sh
128 MB scripts/installers/ansible.sh
110 MB scripts/installers/cmake.sh
103 MB scripts/installers/heroku.sh
96 MB scripts/installers/phantomjs.sh
89 MB scripts/installers/1804/kubernetes-tools.sh
75.5 MB scripts/installers/aws.sh
68.4 MB apt-get -y install sudo lsb-core software-properties-common apt-utils
67.3 MB scripts/installers/ruby.sh
53.1 MB scripts/installers/git.sh
52.1 MB scripts/installers/terraform.sh
github-actions[bot] commented 4 years ago

Issue is stale and will be closed in 7 days unless there is new activity

github-actions[bot] commented 4 years ago

Issue is stale and will be closed in 7 days unless there is new activity

kiptoomm commented 4 years ago

I got a chance today to start looking into this. I created a tool to convert the upstream ubuntu1804.json into a Dockerfile (I'll release that later) and attempted to build it. A few things:

The total size is now just under 50 GB 🤯

It looks like some new things have been added to the environment since @cplee's image was built.

There are some steps I skipped as they failed and I didn't have time to look into why yet. The skipped steps were:

  • scripts/installers/docker-moby.sh (test failed due to dockerd not running inside container)
  • scripts/helpers/containercache.sh (same as above)
  • scripts/installers/homebrew.sh (refused to be installed as root)
  • scripts/installers/aws-sam-cli.sh (uses homebrew)
  • scripts/installers/mysql.sh (test failed to mysqld not running inside container)
  • scripts/installers/1804/android.sh
  • scripts/installers/vcpkg.sh
  • scripts/installers/1804/php.sh

These were all the steps that resulted in a layer on disk greater than 50 MB.

SIZE CREATED BY 20.4 GB scripts/installers/dotnetcore-sdk.sh 9.3 GB scripts/installers/haskell.sh 7.14 GB scripts/installers/hosted-tool-cache.sh 1.6 GB scripts/installers/swift.sh 1.37 GB scripts/installers/java-tools.sh 1.37 GB scripts/installers/clang.sh 921 MB scripts/installers/azpowershell.sh 527 MB scripts/installers/mono.sh 503 MB scripts/installers/rust.sh 500 MB scripts/installers/azure-cli.sh 382 MB scripts/installers/miniconda.sh 367 MB scripts/installers/nodejs.sh 366 MB scripts/installers/julia.sh 364 MB scripts/installers/google-cloud-sdk.sh 302 MB scripts/installers/mongodb.sh 287 MB scripts/installers/hhvm.sh 271 MB scripts/installers/bazel.sh 260 MB scripts/installers/google-chrome.sh 203 MB scripts/installers/1804/basic.sh 202 MB scripts/installers/1804/basic.sh 190 MB scripts/installers/1804/containers.sh 164 MB scripts/installers/powershellcore.sh 162 MB scripts/installers/powershellcore.sh 155 MB scripts/installers/packer.sh 155 MB scripts/installers/azcopy.sh 130 MB scripts/installers/python.sh 128 MB scripts/installers/ansible.sh 110 MB scripts/installers/cmake.sh 103 MB scripts/installers/heroku.sh 96 MB scripts/installers/phantomjs.sh 89 MB scripts/installers/1804/kubernetes-tools.sh 75.5 MB scripts/installers/aws.sh 68.4 MB apt-get -y install sudo lsb-core software-properties-common apt-utils 67.3 MB scripts/installers/ruby.sh 53.1 MB scripts/installers/git.sh 52.1 MB scripts/installers/terraform.sh

@aidansteele @cplee what happened to this? I am not so familiar with using Docker images so would love it if you can point me to any docs on how to use this slim variant. I tried using ubuntu-latest=nektos/act-environments-ubuntu:18.04 but it looks too large as it's taking forever to load. All I'm trying to do is to use act on paths-filter, a GH Action that requires git

pimterry commented 4 years ago

I've found an interesting alternative approach. I'm using this as my base image: https://github.com/httptoolkit/act-build-base/blob/main/Dockerfile

In effect, I'm sticking with a plain node base image, and just mixing in individual installers for the specific parts of the standard image I need, directly pulling individual scripts from act-environments.

Maybe there's a route through here? The installer scripts aren't quite as easy to use standalone right now as they could be (e.g. they don't work without document.sh preloaded into the right path) but with a few small tweaks there I can imagine an Act readme that said you can either:

For people like me, that'd be pretty useful - doing this seems to work perfectly, it's very easy, and it saves me an enormous amount of disk space.

For GitHub themselves it makes sense to provide an actions base that includes the kitchen sink, but no individual devs need all of that locally for every build.