Open fwilhe opened 4 years ago
See https://github.com/nektos/act#runners.
You can customize the image used for the runs-on. I've created an image that is a match to GitHub runners, but its like 15GB!
Hi @cplee
thanks for the work you put into this.
I know that the image is configurable, and I've locally tested with an image where I installed sudo. So this is kind of a workaround, but the reason I opened this issue is that this limits compatibility with workflows that run on GitHub. My intention was to point out where the documented (and actual) behavior of Actions is unequal the default of act
, which makes it less useful compared to what it could be.
As mentioned I'm not sure what solution I would propose, as maintaining separate images seems inappropriate.
Tough problem. The runners that GitHub Actions offers are huge. I didn't feel right using an 18GB image as the default, requiring users of act
to download that large of an image to run it.
I chose the node
image because it was required to run node actions and still light enough. Not sure how to proceed...not sure what good enough looks like for the default docker image.
Open to ideas ๐
I wonder if there's a way we could make an intermediary image that has 90% of the needed CLI tools for 10% of the footprint of the giant image. The only way we could reasonably do that (beyond intuition) is scanning a corpus of public GH Actions and seeing what works.
Then again, maybe it would give people a false sense of security as these issues would crop up less often - so it would be "stranger" and more likely to result in bug reports. ๐ค
Maybe act could use a ~/.act/
or the current project directory?
see #196 for discussion on creating a slim runner
I think at least the documentation should be updated, it currently states "The environment variables and filesystem are all configured to match what GitHub provides." but should say that that is only the case if you use this non-default image.
I think that docker for mac/windows is actually not very good with large images, but that an 18GB image in general isn't that crazy given the high speed internet and large disks available today.
I would at least add common tools that are used in CI builds. For example, make
is currently missing. I think that adding it doesn't change the size of the image that much but saves time and makes it much more convenient for a big crowd of people.
Or maybe I'm doing something wrong without realizing it and make is actually in the image :)
I would at least add common tools that are used in CI builds
I guess there is not enough agreement on what is "common". In the end you would include everything that github installs in the runner, which @cplee already did in that 20 gig image.
I have to say that building my own "runtime" image worked well for me.
The nice thing with that image is that you can also use it to run the whole action in via
jobs:
build:
runs-on: ubuntu-latest
container: myimage
and to not rely on any "setup" actions. That makes it much easier to reproduce the github build on your local machine using act
.
Hope that helps.
I guess there is not enough agreement on what is "common"
Yes, I agree. But the initial argument against using the huge github image seems to be its size. Such tools as make
are tiny and are incredibly popular in this context, so it's not clear to me what's the practical motivation of not including it into the default image.
That makes it much easier to reproduce the github build on your local machine using act.
In my opinion, most of the people interested in this project are interested in it because they already use github actions in their projects, not the other way around. Asking them to change their existing CI configuration (especially on a company/organization level) just to be able to test it with a particular tool doesn't seem to be practical.
Just my opinion though.
so it's not clear to me what's the practical motivation of not including it into the default image.
I'm not the author of act, so I can only guess that the motivation is to use an off-the-shelf image, and not to build one and then discuss with everyone what should be included or not.
Asking them to change their existing CI configuration (especially on a company level) just to be able to test it with a particular tool doesn't seem to be practical.
No, I'm not asking anyone to do that, I just say what worked for me.
Issue is stale and will be closed in 7 days unless there is new activity
seems like git is not inside the node:12.6-buster-slim image either. Which is counter intuitive.
Got the same issue on NixOS, running this simple workflow.
[Build/Build] ๐งช Matrix: map[java:jdk11]
[Build/Build] ๐ Start image=node:12.6-buster-slim
[Build/Build] ๐ณ docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Build/Build] ๐ณ docker cp src=/home/gvolpe/workspace/scala3-typed-holes/. dst=/github/workspace
[Build/Build] โญ Run actions/checkout@v2.3.2
[Build/Build] โ
Success - actions/checkout@v2.3.2
[Build/Build] โญ Run Install Nix โ๏ธ
[Build/Build] โ git clone 'https://github.com/cachix/install-nix-action' # ref=v12
[Build/Build] ๐ณ docker cp src=/home/gvolpe/.cache/act/cachix-install-nix-action@v12 dst=/actions/
| /actions/cachix-install-nix-action@v12/lib/install-nix.sh: line 11: sudo: command not found
| child_process.js:644
| throw err;
| ^
|
| Error: Command failed: /actions/cachix-install-nix-action@v12/lib/install-nix.sh
| at checkExecSyncError (child_process.js:623:11)
| at Object.execFileSync (child_process.js:641:15)
| at Object.<anonymous> (/actions/cachix-install-nix-action@v12/lib/main.js:4:17)
| at Module._compile (internal/modules/cjs/loader.js:776:30)
| at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
| at Module.load (internal/modules/cjs/loader.js:643:32)
| at Function.Module._load (internal/modules/cjs/loader.js:556:12)
| at Function.Module.runMain (internal/modules/cjs/loader.js:839:10)
| at internal/main/run_main_module.js:17:11 {
| status: 127,
| signal: null,
| output: [ null, null, null ],
| pid: 28,
| stdout: null,
| stderr: null
| }
[Build/Build] โ Failure - Install Nix โ๏ธ
Error: exit with `FAILURE`: 1
I propose we offer the large or slim image upon initial install (I used choco) and also include a configuration command (act --default-image slim/github/other
) with the addition of --global
affecting all images not the current workspace. Or something along those lines.
I'm just here to avoid never using act again because like mentioned above the readme clearly states it just works out the box and matches github actions like for like but locally in docker. This is not the case, and as a lazy user (I'd imagine 90% of your userbase) I would be happy to be asked once during install or just have to set my preferred default once and forget it. But to be honest I'm confused. In my head if you take the same docker image as github does and the action specify then yes it would be large but that's what the user signed up for. Still I feel your pain and appreciate the idea of slim by default. Also I dislike installers that don't have command line options like -y
to answer yes to everything.
Separately, nice to see a healthy community and thanks for all the hard work / input all. Can't wait to see this move forward, but I'd be keen to hear if anyone had similar utility by using the official local github build agent, I imagine that's probably accessible/hackable enough (able to get access to terminal and files).
I'd be keen to hear if anyone had similar utility by using the official local github build agent, I imagine that's probably accessible/hackable enough (able to get access to terminal and files).
Don't use GitHub actions or run full GitHub Actions docker image then.
How's about we enhance the readme to mention how to run the full image in the installation instructions. That will remove most of these "duplicate" issues. Basically it needs to be more than obvious that 99% of users will need to do this or face errors...
Having the sections after detailed usage instructions is a case of TLDR, and I did RTFM (mostly initially then fully after experiencing issues) but still missed it and then got my syntax wrong for my already instantiated act instance once I'd finally spotted it. The configuration file eventually solved my snafu ๐คฆ
@tyeth Would something like this be okay?
I think that's fair @CatTheHacker . If possible, I would like to see another Small image with essential Unix tools such as sudo
, curl
, awk
, etc.
@CatTheHacker Brilliant yes, and maybe worth mentioning in that comment its using the Github official image for big (somehow more concisely). Thanks for taking a look at windows Path slash direction too.
It might make sense to also base the "small" image on Ubuntu. I noticed that the setup-python action for example relies on ubuntu specifically to find out which archive to download, and fails on the node image that is based on debian.
I think that's fair @CatTheHacker . If possible, I would like to see another Small image with essential Unix tools such as
sudo
,curl
,awk
, etc.
@gvolpe that's not in my power, to me it also seems weird that image without sudo
or curl
was chosen as default. I can make such as default in my fork but it will be up to act
owner if they will chose to implement that in the end.
@CatTheHacker Brilliant yes, and maybe worth mentioning in that comment its using the Github official image for big (somehow more concisely). Thanks for taking a look at windows Path slash direction too.
@tyeth There isn't any official GitHub image for Docker unless you mean act
big one. I'm open for suggestions on how to write/convey the message ๐ since I'm usually not the best in describing things to people.
It might make sense to also base the "small" image on Ubuntu. I noticed that the setup-python action for example relies on ubuntu specifically to find out which archive to download, and fails on the node image that is based on debian.
@fwilhe Just, Yes. (even though I'm always for Debian and hate strongly dislike Ubuntu, but when using ubuntu-xx.xx
runner, I expect it to be actually Ubuntu)
feel free to make any suggestions, I'm trying to do as much as possible for this repo in free time
@tyeth Would something like this be okay?
My english brain is not firing on all cylinders right now, but having seen the merge progress today I thought I'd better get back here sharpish ๐ Maybe just add to the brackets describing big, from (includes more tools) to (included tools match Github Actions)
But I'd feel better with something else ๐คฆ Any suggestions people?
Ok, so I've been thinking a bit and came up with this
node:12.6-buster-slim
(we could build upon that image to add below image env vars but leave it with just node.js, to make it a bit more compatible)ubuntu:act-*
, includes required env vars for some software setup (like ruby/setup-ruby
), includes more tools (gawk
,curl
,git
,ssh
,etc.)nektos/act-environments-ubuntu:18.04
big
and mine as large
(quay.io/catthehacker/virtual-environments | https://github.com/CatTheHacker/virtual-environments)Looking for feedback/suggestions
It would be really nice if there were clear instructions on how a user can build and enable an image of their own that matches the one used by github. Here's are two clear question to answer:
@xloem
Modifications to remove or install inaccessible commands. I'll ask the first question a different way:
I infer the way to run a custom docker image is:
Dockerfile
in the folder based on a simple template (needs link to example).actrc
to reference the name of the image like nektos/act-environments-ubuntu:18.04 in -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04
How do I build a complete docker github workflow image?
It's just not possible due to the way how Docker works. See https://github.com/actions/virtual-environments/discussions/2320#discussioncomment-328445
way to run a custom docker image
I'd say it's not our responsibility to teach how to make Docker images, there is plenty of (much better) sources that go into details on how to do that where official Docker documentation is quite good source on that.
README.md
describes:
Eventually there could be link to Docker documentation regarding how to make Docker images in README.md
.
I was debugging another issue and wanted to check if the large image would solve it. It looks like I'm stuck with the micro image that I've chosen at the very first install. Is there a way to force-reset to the large one?
I've tried a bunch of things:
act
~/.actrc
~/.cache/act
-P ...
with the commandAfter reinstalling it fails almost instantly and never attempts to download the large image, and it looks like containers
section of the workflow overrides the runs-on
.
@killthekitten
You are using python:3.8.5
according to your log.
To get default image survey, .actrc
cannot exist in ~
, ~/.config/
and current directory where act
is run or you can just replace image yourself, those are described in README
@catthehacker thanks for the fast response!
To get default image survey, .actrc cannot exist in ~,
Did you mean it can exist or cannot exist?
I tried replacing the python runtime with no big success:
act -j test --secret-file .act.secrets -v -P python:3.8.5=nektos/act-environments-ubuntu:18.04
Maybe it has to do with runs-on
and container
defined at the same time? It comes from this section of my workflow:
jobs:
test:
runs-on: ubuntu-latest
container:
image: python:3.8.5
container
will use the image you defined in container:
. -P
is used to replace image used in runs-on:
.
@catthehacker : First of all I like to show my fullest appreciation for all the work you were doing to provide access to multiple image sizes. I was wondering whether in order to avoid many duplicate issues here - one could create symlinks to tools that have not been included in the smaller images to a file that once executed provides a message explaining that you need a larger image, point to this issue and then error out?
@jonico that's an amazing idea, I'll try to work on that.
@jonico that's an amazing idea, I'll try to work on that.
If one were to use an unusual exit code for that script, one may also have special treatment of this error code one the docker container exited - will definitely try out any suggestions you do ๐
I would use already existing exit code 127
which stands for command not found but I will use command-not-found
or write small Go program to make proper error message and add it to image.
@catthehacker: I will feature nektos/act in GitHub's upcoming summer fest. I figured you are in a similar time zone and might be interested in listening in. Here is the registration link (everybody is welcome - and there will be :octocat: swag for the first 200 registrants): https://resources.github.com/webcasts/Summerfest-2021-South-Europe/
@jonico I've tried to work on the not installed tools and:
bash
command_not_found_handle
won't work because like in GHA, we don't load profiles and rc
filesWould it be possible to intercept the 128 / command not found when the docker container already exited?
Sounds possible but haven't got around that yet.
@jonico
@catthehacker: Wonderful ๐ป - I would suggest to change the error message to "Please see https://github.com/nektos/act/issues/107 for potential reasons and solutions." but this is probably just ๐ฒ shedding ๐
Why isn't the gh
command installed though?
So I tried using the medium instead of slim for git, but noticed medium doesn't have yarn :/
Not sure if this is useful for anyone, but out of curiosity I've tried to build a 'minimal' runner image based on github's packer scripts. The repo is here, the image is published to DockerHub both for x86 and arm.
The idea is to keep the pre-installed amount of tools low because I assume most workflows anyway use setup-*
actions.
I know about existing images, but maybe mine fill a gap and are useful for some people?
Let me know if this is useful to you.
Why isn't the
gh
command installed though?
Were you able to resolve this? Should I install gh
first via brew install
?
Is this the command I'm supposed to use please? It doesn't seem to spend time pulling any bigger image though -
./act -P catthehacker/ubuntu:custom-latest-20240215 -s GITHUB_TOKEN=abIUV8V push
I still get this error
[Test FusionAuth login/run-tests] โ
Success - Main Get docker-compose.yaml
[Test FusionAuth login/run-tests] โญ Run Main Start FusionAuth
[Test FusionAuth login/run-tests] ๐ณ docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/1-composite-1.sh] user= workdir=faDockerComposeFilePath
| /var/run/act/workflow/1-composite-1.sh: line 2: docker: command not found
[Test FusionAuth login/run-tests] โ Failure - Main Start FusionAuth
[Test FusionAuth login/run-tests] exitcode '127': command not found, please refer to https://github.com/nektos/act/issues/107 for more information
same as when I tried running
./act -P ubuntu-22.04=-self-hosted
What's interesting is the runner links are broken above, and I would love to know how to use the 18GB runner right now.
For those looking, the docs on runners have apparently moved here: https://nektosact.com/usage/runners.html also my issue is one of surprise. The first run I was asked how large an env would I like. I chose micro, because I'm just starting. By making that uninformed choice, I spent the last couple hours trying to figure out what issue 107 meant and how to do -P ubuntu-latest=catthehacker/ubuntu:act-latest
once I finally found the documentation. Forcing a user on first run to make a size choice with no obvious way to change said choice is a great way to trap folks in this non-intuitive error.
So I tried using the medium instead of slim for git, but noticed medium doesn't have yarn :/
I added these steps and act was able to add yarn
with that
- name: Set Node.js 16.x
uses: actions/setup-node@v3
with:
node-version: 16.x
- name: Run install
uses: Borales/actions-yarn@v5.0.0
with:
cmd: install # will run `yarn install` command
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} # if needed
- name: ๐ใBuild bundle
uses: Borales/actions-yarn@v5.0.0
with:
cmd: run build
dir: ./
Hi,
as documented here GitHub configures passwordless sudo, so I can have sudo in my shell scripts to perform individual commands as root.
Given this minimal workflow:
The result on Actions is
uid=0(root) gid=0(root) groups=0(root)
Running this workflow in
act
yields this result:Not sure what the most elegant solution would be without re-building the
node:12.6-buster-slim
image to install sudo and configure it passwordless. But I think the goal should be to be 1:1 compatible with actions, if possible.