search

jetify-com / devbox-install-action

31 stars 15 forks source link

Action does not work with `act` #10

Closed mikenikles closed 11 months ago

mikenikles commented 1 year ago

I'm working on a follow-up blog post to Test GitHub Actions locally, but with a focus on how to use act (GitHub repo) locally and Devbox on GitHub Actions.

The work-in-progress pull request is at https://github.com/mikenikles/devbox-github-actions/pull/1.

Setup

  1. Clone the repo and change to the configure-devbox-on-ci branch
  2. devbox shell
  3. devbox run act

I run into two issues: 1) If I remove the refresh-cli: true input, devbox run act fails with exec: "node": executable file not found in $PATH: unknown 2) If I keep the refresh-cli: true input, the installation of the Devbox CLI is stuck with an interactive prompt "Nix is not installed. Devbox will attempt to install it. Press enter to continue or ctrl-c to exit." This is because of this line in the installation script.

I'm opening this in case someone on the Devbox team has insights into how we can make this work, while in parallel I continue to investigate too.

mikenikles commented 1 year ago

I wonder if this line should be as follows:

- if isatty.IsTerminal(os.Stdout.Fd()) {
+ if isatty.IsTerminal(os.Stdout.Fd()) || isatty.IsCygwinTerminal(os.Stdout.Fd()) {

This is a bit of a guess, based on the fact that act does that here.

LucilleH commented 1 year ago

I haven't encountered/tried this combination of devbox > act > devbox scenario. What I've seen the most is testing locally with devbox run and CICD using the installer + running the same script, assuming the same script will produce the same result in local and CICD.

I'm not sure how act works, and whether it installs a fresh nix + devbox every time you run locally (I hope not, or else the experience would be very slow). Thoughts? @mikenikles

mikenikles commented 1 year ago

I'm on the same page in terms of devbox run locally and on CI / CD.

act emulates a GitHub actions environment locally within a Docker container. The benefit of that is you can iterate on your .github/workflows/*.yml files locally, without pushing changes to GitHub to kick off a new workflow run.

I was hoping to get this to work to showcase how someone can use a devbox.json for their dev environment, their CI environment, and also run the GitHub workflows locally with act to iterate quickly.

Caching the Devbox CLI works the same as it does in a GitHub workflow, so that helps with the experience locally.

This is a low priority issue from my point of view. I figured I'd open it to see if anyone on the team goes like "Ah... I think I have a fix for that" 😄. Happy to even close it and if I ever get back to using act in a project, I may re-test and re-open this issue if necessary.

airtonix commented 9 months ago

this is now a problem, because I cannot debug why a goreleaser run fails to run due to the devbox/installer inexplicably not making goreleaser available as a command.

having to do a release, wait for a build, discover that yet again the problem isn't fixed gets very very tiring.

on:
  push:
    branches:
      - master

permissions:
  contents: write
  pull-requests: write
  packages: write

name: ReleaseManagement

jobs:
  Release:
    name: Release Please
    runs-on: ubuntu-22.04
    outputs:
      release_created: ${{ steps.release.outputs.release_created }}
      releases_created: ${{ steps.release.outputs.releases_created }}
      tag_name: ${{ steps.release.outputs.tag_name }} # e.g. v1.0.0
      version: ${{ steps.release.outputs.version }} # e.g. 1.0.0
      all: ${{ toJSON(steps.release.outputs) }}
    steps:
      - uses: google-github-actions/release-please-action@v3
        id: release-please
        # if we're using ACT, skip this
        if: ${{ !github.event.act }}
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          command: manifest
          release-type: go
          extra-files: |
            meta/package.go
      - id: release
        env:
          NOT_ACT: ${{ !github.event.act }}
        run: |
          // if we're using ACT, output that a release was created
          // otherwise output all the release info from the release-please action
          if [ "${NOT_ACT}" == "true" ]; then
            echo "name=release_created::${{ steps.release-please.outputs.release_created }}" >> $GITHUB_OUTPUT
            echo "name=releases_created::${{ steps.release-please.outputs.releases_created }}" >> $GITHUB_OUTPUT
            echo "name=tag_name::${{ steps.release-please.outputs.tag_name }}" >> $GITHUB_OUTPUT
            echo "name=version::${{ steps.release-please.outputs.version }}" >> $GITHUB_OUTPUT
          else
            echo "name=release_created::true" >> $GITHUB_OUTPUT
            echo "name=releases_created::true" >> $GITHUB_OUTPUT
            echo "name=tag_name::snapshot" >> $GITHUB_OUTPUT
            echo "name=version::snaphot" >> $GITHUB_OUTPUT
          fi

  Build:
    if: needs.Release.outputs.releases_created

    runs-on: ubuntu-22.04
    needs: [Release]

    env:
      REGISTRY: ghcr.io
      IMAGE_NAME: ${{ github.repository }}

    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - run: git fetch --force --tags

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2
      - uses: docker/setup-buildx-action@v2
      - name: Docker Login
        uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: setup jetpack devbox
        uses: jetpack-io/devbox-install-action@v0.7.0

      - name: Run GoReleaser
        env:
          registry: ${{ env.REGISTRY }}
          IMAGE_NAME: ${{ env.IMAGE_NAME }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          devbox run -- gorelease release --clean
airtonix commented 9 months ago

related: https://github.com/catthehacker/docker_images/issues/119

LucilleH commented 9 months ago

Hi @airtonix,

The error is caused by determinate systems nix installer trying to setup a systemd service for nix by default, but a docker image doesn't have systemd.

Can you try adding skip-nix-installation: 'true' to the devbox action, add the following nix installer step before the devbox step:

    - name: Install nix in container
       uses: DeterminateSystems/nix-installer-action@v4
       with:
         logger: pretty
         extra-conf: experimental-features = ca-derivations fetch-closure
         init: 'none'

    - name: setup jetpack devbox
       uses: jetpack-io/devbox-install-action@v0.7.0
       with:
         skip-nix-installation: 'true'
airtonix commented 9 months ago

@LucilleH that got me a bit further, but then ran into this: https://github.com/DeterminateSystems/nix-installer-action/issues/33

so with : 

      - name: Install nix in container
        uses: DeterminateSystems/nix-installer-action@v4
        with:
          logger: pretty
          planner: linux
          extra-conf: experimental-features = ca-derivations fetch-closure
          init: 'none'

      - name: setup jetpack devbox
        uses: jetpack-io/devbox-install-action@v0.7.0
        env:
          DEVBOX_DEBUG: 1
        with:
          skip-nix-installation: 'true'

      - name: Run GoReleaser
        env:
          registry: ${{ env.REGISTRY }}
          IMAGE_NAME: ${{ env.IMAGE_NAME }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          if [ "${{ !github.event.act }}" == "true" ]; then
            devbox run -- gorelease release --clean
          else
            devbox run -- gorelease build --clean --snapshot
          fi

i'm not getting : 

| 2023/11/21 09:59:23 findProjectDir: path is 
| 2023/11/21 09:59:23 finding devbox.json in dir: /mnt/Store/Projects/Mine/Github/bank-downloaders
| 2023/11/21 09:59:23 findProjectDir: path is 
| 2023/11/21 09:59:23 finding devbox.json in dir: /mnt/Store/Projects/Mine/Github/bank-downloaders
| Error: exit status 1
| 
| 2023/11/21 09:59:23 Command stderr: error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted
| 
| 2023/11/21 09:59:23 
| ExecutionID:377bd372c58c4ed68376268366d6f25f
| <nil>
| 2023/11/21 09:59:23 findProjectDir: path is
airtonix commented 9 months ago

just checked that the image for act that I'm using has root access (some of them don't 🤷🏻 ).

it does.

airtonix commented 9 months ago

aha, i get further by running act like : 

  act push \
    --platform ubuntu-22.04=catthehacker/ubuntu:act-22.04 \
    --eventpath .actevent.json \
    --workflows .github/workflows/release.yml \
    --job Build

.actevent.json is

{
  "act": true
}
airtonix commented 9 months ago

and now another error... looks to be something something something github api no auth error

| Found devbox latest version 0.8.2.
[ReleaseManagement/Build  ]   ✅  Success - Main Get devbox version
[ReleaseManagement/Build  ]   ⚙  ::set-env:: latest_version=0.8.2
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] ⭐ Run Main Mount devbox cli cache
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ]   🐳  docker cp src=/home/zenobius/.cache/act/actions-cache-restore@v3/ dst=/var/run/act/actions/actions-cache-restore@v3/
[ReleaseManagement/Build  ]   🐳  docker exec cmd=[node /var/run/act/actions/actions-cache-restore@v3/dist/restore-only/index.js] user= workdir=
[ReleaseManagement/Build  ]   💬  ::debug::Resolved Keys:
[ReleaseManagement/Build  ]   💬  ::debug::["Linux-devbox-cli-0.8.2"]
[ReleaseManagement/Build  ]   💬  ::debug::Checking zstd --quiet --version
[ReleaseManagement/Build  ]   💬  ::debug::1.4.8
[ReleaseManagement/Build  ]   💬  ::debug::zstd version: 1.4.8
[ReleaseManagement/Build  ]   💬  ::debug::Resource Url: http://192.168.86.43:44323/_apis/artifactcache/cache?keys=Linux-devbox-cli-0.8.2&version=10c3673137184e8fc98ee7019556b49e59a4412bda7618d1ecdeb71d8913cc9a
[ReleaseManagement/Build  ]   ⚙  ***
[ReleaseManagement/Build  ]   💬  ::debug::Cache Result:
[ReleaseManagement/Build  ]   💬  ::debug::{"archiveLocation":"***","cacheKey":"linux-devbox-cli-0.8.2","result":"hit"}
[ReleaseManagement/Build  ]   💬  ::debug::Archive Path: /tmp/c6dbdcbb-4b92-4edf-b7d6-d70d42b08b59/cache.tzst
[ReleaseManagement/Build  ]   💬  ::debug::Use Azure SDK: false
[ReleaseManagement/Build  ]   💬  ::debug::Download concurrency: 8
[ReleaseManagement/Build  ]   💬  ::debug::Request timeout (ms): 30000
[ReleaseManagement/Build  ]   💬  ::debug::Cache segment download timeout mins env var: undefined
[ReleaseManagement/Build  ]   💬  ::debug::Segment download timeout (ms): 600000
[ReleaseManagement/Build  ]   💬  ::debug::Lookup only: false
| Cache Size: ~8 MB (8449799 B)
| [command]/usr/bin/tar -xf /tmp/c6dbdcbb-4b92-4edf-b7d6-d70d42b08b59/cache.tzst -P -C /mnt/Store/Projects/Mine/Github/bank-downloaders --use-compress-program unzstd
| Cache restored successfully
| Cache restored from key: linux-devbox-cli-0.8.2
[ReleaseManagement/Build  ]   ✅  Success - Main Mount devbox cli cache
[ReleaseManagement/Build  ]   ⚙  ::set-output:: cache-primary-key=Linux-devbox-cli-0.8.2
[ReleaseManagement/Build  ]   ⚙  ::set-output:: cache-matched-key=linux-devbox-cli-0.8.2
[ReleaseManagement/Build  ]   ⚙  ::set-output:: cache-hit=true
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] ⭐ Run Main Configure nix access-tokens
[ReleaseManagement/Build  ]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/2-composite-5.sh] user= workdir=
[ReleaseManagement/Build  ]   ✅  Success - Main Configure nix access-tokens
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ]   🐳  docker exec cmd=[node /var/run/act/workflow/hashfiles/index.js] user= workdir=
[ReleaseManagement/Build  ] 'runs-on' key not defined in ReleaseManagement/Build
[ReleaseManagement/Build  ] ⭐ Run Main Install devbox packages
[ReleaseManagement/Build  ]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/2-composite-8.sh] user= workdir=
| 2023/11/21 10:20:39 findProjectDir: path is 
| 2023/11/21 10:20:39 finding devbox.json in dir: /mnt/Store/Projects/Mine/Github/bank-downloaders
| 2023/11/21 10:20:39 findProjectDir: path is 
| 2023/11/21 10:20:39 finding devbox.json in dir: /mnt/Store/Projects/Mine/Github/bank-downloaders
| 2023/11/21 10:20:39 script: echo
| 2023/11/21 10:20:39 script args: [Packages installed!]
| 2023/11/21 10:20:39 findProjectDir: path is .
| Ensuring packages are installed.
| 
| Installing 6 packages: go@1.20.5, goreleaser@1.22.1, just@1.16.0, gopass@1.15.8, go-tools@2023.1.6, act@0.2.54.
| 
| [1/6] go@1.20.5
| 2023/11/21 10:20:39 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 6 /nix/store/nvydgjdaff1i887x2fk8kygcrbxyfyiy-go-1.20.5 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[1/6] go@1.20.5: Success
| [2/6] goreleaser@1.22.1
| 2023/11/21 10:21:07 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 7 /nix/store/c7kx0d7lkqcd7gfn1ybkc91fd1x7var1-goreleaser-1.22.1 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[2/6] goreleaser@1.22.1: Success
| [3/6] just@1.16.0
| 2023/11/21 10:21:13 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 8 /nix/store/c39cbcadvxjpy2rgw0gkgykfx8r2bcix-just-1.16.0 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[3/6] just@1.16.0: Success
| [4/6] gopass@1.15.8
| 2023/11/21 10:21:15 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 9 /nix/store/z6npmwq6k5y658cdmcp8nzb8la9bav5v-gopass-1.15.8 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[4/6] gopass@1.15.8: Success
| [5/6] go-tools@2023.1.6
| 2023/11/21 10:21:35 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 10 /nix/store/5df6gjp8lv6ndixw58ayihc8394nfzq4-go-tools-2023.1.6 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[5/6] go-tools@2023.1.6: Success
| [6/6] act@0.2.54
| 2023/11/21 10:21:38 running command: /nix/var/nix/profiles/default/bin/nix profile install --profile /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/nix/profile/default --impure --priority 11 /nix/store/g4834zf11rr214wfnbz6iy324bznf1nf-act-0.2.54 --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure
[6/6] act@0.2.54: Success
| hint: Using 'master' as the name for the initial branch. This default branch name
| hint: is subject to change. To configure the initial branch name to use in all
| hint: of your new repositories, which will suppress this warning, call:
| hint: 
| hint:         git config --global init.defaultBranch <name>
| hint: 
| hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
| hint: 'development'. The just-created branch can be renamed via this command:
| hint: 
| hint:         git branch -m <name>
| Initialized empty Git repository in /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake/.git/
| 2023/11/21 10:21:42 current environment PATH is: /nix/var/nix/profiles/default/bin:/opt/hostedtoolcache/node/18.18.2/x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
| 2023/11/21 10:21:42 Running print-dev-env cmd: /nix/var/nix/profiles/default/bin/nix print-dev-env /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure --json
| Error: error running script "echo" in Devbox: Command: /nix/var/nix/profiles/default/bin/nix print-dev-env /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure --json: exit status 1
| 
| 2023/11/21 10:21:42 Command stderr: warning: Git tree '/mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake' is dirty
| error:
|        … while updating the lock file of flake 'git+file:///mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake'
| 
|        … while updating the flake input 'nixpkgs'
| 
|        … while fetching the input 'github:NixOS/nixpkgs/75a52265bda7fd25e06e3a67dee3f0354e73243c'
| 
|        error: unable to download 'https://api.github.com/repos/NixOS/nixpkgs/tarball/75a52265bda7fd25e06e3a67dee3f0354e73243c': HTTP error 401
| 
|        response body:
| 
|        {
|          "message": "Bad credentials",
|          "documentation_url": "https://docs.github.com/rest"
|        }
| 
| 2023/11/21 10:21:42 
| ExecutionID:6b8aac534b0e4a6f8003be61a7bc6df6
| exit status 1
| Command: /nix/var/nix/profiles/default/bin/nix print-dev-env /mnt/Store/Projects/Mine/Github/bank-downloaders/.devbox/gen/flake --extra-experimental-features ca-derivations --option experimental-features nix-command flakes fetch-closure --json
| go.jetpack.io/devbox/internal/nix.(*Nix).PrintDevEnv
|       go.jetpack.io/devbox/internal/nix/nix.go:78
| go.jetpack.io/devbox/internal/impl.(*Devbox).computeNixEnv
|       go.jetpack.io/devbox/internal/impl/devbox.go:803
| go.jetpack.io/devbox/internal/impl.(*Devbox).ensurePackagesAreInstalledAndComputeEnv
|       go.jetpack.io/devbox/internal/impl/devbox.go:965
| go.jetpack.io/devbox/internal/impl.(*Devbox).RunScript
|       go.jetpack.io/devbox/internal/impl/devbox.go:211
| go.jetpack.io/devbox/internal/boxcli.runScriptCmd
|       go.jetpack.io/devbox/internal/boxcli/run.go:111
| go.jetpack.io/devbox/internal/boxcli.runCmd.func1
|       go.jetpack.io/devbox/internal/boxcli/run.go:43
| github.com/spf13/cobra.(*Command).execute
|       github.com/spf13/cobra@v1.7.0/command.go:940
| github.com/spf13/cobra.(*Command).ExecuteC
|       github.com/spf13/cobra@v1.7.0/command.go:1068
| github.com/spf13/cobra.(*Command).Execute
|       github.com/spf13/cobra@v1.7.0/command.go:992
| go.jetpack.io/devbox/internal/boxcli/midcobra.(*midcobraExecutable).Execute
|       go.jetpack.io/devbox/internal/boxcli/midcobra/midcobra.go:61
| go.jetpack.io/devbox/internal/boxcli.Execute
|       go.jetpack.io/devbox/internal/boxcli/root.go:112
| go.jetpack.io/devbox/internal/boxcli.Main
|       go.jetpack.io/devbox/internal/boxcli/root.go:135
| main.main
|       ./main.go:11
| runtime.main
|       runtime/proc.go:267
| runtime.goexit
|       runtime/asm_amd64.s:1650
| 2023/11/21 10:21:42 findProjectDir: path is 
| 2023/11/21 10:21:42 finding devbox.json in dir: /mnt/Store/Projects/Mine/Github/bank-downloaders
airtonix commented 9 months ago

ok updated command gets me past that: 

  act push \
    -s GITHUB_TOKEN="$(gh auth token)" \
    --platform ubuntu-22.04=catthehacker/ubuntu:act-22.04 \
    --eventpath .actevent.json \
    --workflows .github/workflows/release.yml \
    --job Build

and now i'm at the same error i get as when run in githubs infrastructure 👍🏻

debugging time

airtonix commented 9 months ago

as usual... i'm the muppet.

original problem was that the command i'm supposed to run is goreleaser, not gorelease

LucilleH commented 9 months ago

@airtonix glad it all worked! 😃