search

nektos / act

Run your GitHub Actions locally 🚀
https://nektosact.com
MIT License
54.22k stars 1.35k forks source link

Bug: Works in github actions but not in act runner. binfmt_misc not enabled (Log Provided) #2342

Open mofosyne opened 4 months ago

mofosyne commented 4 months ago

Just recently added ci to llamafile, but would like the capability to test this setup locally, but it's not working as it appears that the binfmt_misc is missing.

Bug report info

$ act --bug-report
act version:            0.2.62
GOOS:                   linux
GOARCH:                 amd64
NumCPU:                 8
Docker host:            DOCKER_HOST environment variable is not set
Sockets found:
        /var/run/docker.sock
Config files:           
        /home/mofosyne/.actrc:
                -P ubuntu-latest=catthehacker/ubuntu:act-latest
                -P ubuntu-22.04=catthehacker/ubuntu:act-22.04
                -P ubuntu-20.04=catthehacker/ubuntu:act-20.04
                -P ubuntu-18.04=catthehacker/ubuntu:act-18.04
Build info:
        Go version:            go1.22.2
        Module path:           command-line-arguments
        Main version:          
        Main path:             
        Main checksum:         
        Build settings:
                -buildmode:           exe
                -compiler:            gc
                -ldflags:             -X main.version=0.2.62
                DefaultGODEBUG:       httplaxcontentlength=1,httpmuxgo121=1,tls10server=1,tlsrsakex=1,tlsunsafeekm=1
                CGO_ENABLED:          1
                CGO_CFLAGS:           
                CGO_CPPFLAGS:         
                CGO_CXXFLAGS:         
                CGO_LDFLAGS:          
                GOARCH:               amd64
                GOOS:                 linux
                GOAMD64:              v1
Docker Engine:
        Engine version:        24.0.5
        Engine runtime:        runc
        Cgroup version:        2
        Cgroup driver:         systemd
        Storage driver:        overlay2
        Registry URI:          https://index.docker.io/v1/
        OS:                    Linux Mint 21.3
        OS type:               linux
        OS version:            21.3
        OS arch:               x86_64
        OS kernel:             5.15.0-101-generic
        OS CPU:                8
        OS memory:             31929 MB
        Security options:
                name=apparmor
                name=seccomp,profile=builtin
                name=cgroupns

Command used with act

act -j ubuntu-focal-make

Describe issue

To be able to cleanly and successfully run act -j ubuntu-focal-make as the same workflow is already working in github actions itself.

Seems like github actions environment has binfmt_misc enabled by default

Link to GitHub repository

https://github.com/Mozilla-Ocho/llamafile/commit/31419d0b718f318ab23ab40eeb10a170e0eb2edc

Workflow content

name: CI
on:
  push:
    branches: [ master, main, fix ]
  pull_request:
    branches: [ master, main, fix ]

jobs:
  ubuntu-focal-make:
    timeout-minutes: 60
    runs-on: ubuntu-latest

    steps:
      - name: Clone
        id: checkout
        uses: actions/checkout@v4

      - name: Dependencies
        id: depends
        run: |
          sudo apt-get update
          sudo apt-get install make

      - name: Cache cosmocc toolchain
        id: cache-cosmocc-toolchain
        uses: actions/cache@v4
        env:
          cache-name: cache-cosmocc-toolchain
        with:
          path: |
            .cosmocc
            o/depend
            o/depend.test
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/config.mk') }}
          restore-keys: |
            ${{ runner.os }}-build-${{ env.cache-name }}

      - name: Setup cosmocc and ape loader
        run: |
          sudo make cosmocc-ci PREFIX=/usr

      - name: Build
        run: |
          sudo make -j $(nproc)

      - name: Make Llamafile
        run: |
          cp ./models/TinyLLama-v0.1-5M-F16.gguf tinyllama.gguf
          cat << EoF > .args
          -m
          tinyllama.gguf
          ...
          EoF
          cp o//llama.cpp/main/main \
            tinyllama.llamafile
          o//llamafile/zipalign -j0 \
            tinyllama.llamafile \
            tinyllama.gguf \
            .args

      - name: Execute LLM CLI CPU  # GA doesn't have "support_simdgroup_reduction" for RMS_NORM :'(
        run: |
          ./tinyllama.llamafile -e -p '## Famous Speech\n\nFour score and seven' -n 50 -ngl 0

Relevant log output

[CI/ubuntu-focal-make] 🚀  Start image=catthehacker/ubuntu:act-latest
[CI/ubuntu-focal-make]   🐳  docker pull image=catthehacker/ubuntu:act-latest platform= username= forcePull=true
[CI/ubuntu-focal-make]   🐳  docker create image=catthehacker/ubuntu:act-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[CI/ubuntu-focal-make]   🐳  docker run image=catthehacker/ubuntu:act-latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[CI/ubuntu-focal-make]   ☁  git clone 'https://github.com/actions/cache' # ref=v4
[CI/ubuntu-focal-make] ⭐ Run Main Clone
[CI/ubuntu-focal-make]   🐳  docker cp src=/home/mofosyne/git/llamafile/. dst=/home/mofosyne/git/llamafile
[CI/ubuntu-focal-make]   ✅  Success - Main Clone
[CI/ubuntu-focal-make] ⭐ Run Main Dependencies
[CI/ubuntu-focal-make]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/depends] user= workdir=
[CI/ubuntu-focal-make]   | Get:1 https://packages.microsoft.com/ubuntu/22.04/prod jammy InRelease [3632 B]
[CI/ubuntu-focal-make]   | Get:2 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main amd64 Packages [155 kB]
[CI/ubuntu-focal-make]   | Get:3 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main all Packages [1035 B]
... truncated for brevity...
[CI/ubuntu-focal-make]   | Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2125 kB]
[CI/ubuntu-focal-make]   | Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [35.0 kB]
[CI/ubuntu-focal-make]   | Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [110 kB]
[CI/ubuntu-focal-make]   | Fetched 32.1 MB in 19s (1705 kB/s)
[CI/ubuntu-focal-make]   | Reading package lists...
[CI/ubuntu-focal-make]   | Reading package lists...
[CI/ubuntu-focal-make]   | Building dependency tree...
[CI/ubuntu-focal-make]   | Reading state information...
[CI/ubuntu-focal-make]   | make is already the newest version (4.3-4.1build1).
[CI/ubuntu-focal-make]   | 0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
[CI/ubuntu-focal-make]   ✅  Success - Main Dependencies
[CI/ubuntu-focal-make]   🐳  docker exec cmd=[node /var/run/act/workflow/hashfiles/index.js] user= workdir=
[CI/ubuntu-focal-make] ⭐ Run Main Cache cosmocc toolchain
[CI/ubuntu-focal-make]   🐳  docker cp src=/home/mofosyne/.cache/act/actions-cache@v4/ dst=/var/run/act/actions/actions-cache@v4/
[CI/ubuntu-focal-make]   🐳  docker exec cmd=[node /var/run/act/actions/actions-cache@v4/dist/restore/index.js] user= workdir=
[CI/ubuntu-focal-make]   💬  ::debug::Resolved Keys:
[CI/ubuntu-focal-make]   💬  ::debug::["Linux-build-cache-cosmocc-toolchain-0a4a7cf6822444f8c0c6ce1593cda1a50cc1389b579d74fd3a1e0ebe5b72c8ef","Linux-build-cache-cosmocc-toolchain"]
[CI/ubuntu-focal-make]   💬  ::debug::Checking zstd --quiet --version
[CI/ubuntu-focal-make]   💬  ::debug::1.4.8
[CI/ubuntu-focal-make]   💬  ::debug::zstd version: 1.4.8
[CI/ubuntu-focal-make]   💬  ::debug::Resource Url: http://192.168.152.100:43737/_apis/artifactcache/cache?keys=Linux-build-cache-cosmocc-toolchain-0a4a7cf6822444f8c0c6ce1593cda1a50cc1389b579d74fd3a1e0ebe5b72c8ef%252CLinux-build-cache-cosmocc-toolchain&version=696a13bfbed4a3f18de2467f3b40f080fbb8a8be981cf6fdbfe35ca22d5055cd
[CI/ubuntu-focal-make]   💬  ::debug::Failed to delete archive: Error: ENOENT: no such file or directory, unlink ''
[CI/ubuntu-focal-make]   | Cache not found for input keys: Linux-build-cache-cosmocc-toolchain-0a4a7cf6822444f8c0c6ce1593cda1a50cc1389b579d74fd3a1e0ebe5b72c8ef, Linux-build-cache-cosmocc-toolchain
[CI/ubuntu-focal-make]   ✅  Success - Main Cache cosmocc toolchain
[CI/ubuntu-focal-make] ⭐ Run Main Setup cosmocc and ape loader
[CI/ubuntu-focal-make]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/3] user= workdir=
[CI/ubuntu-focal-make]   | make: .cosmocc/3.3.8/bin/mkdeps: No such file or directory
[CI/ubuntu-focal-make]   | .cosmocc/3.3.8/bin/mkdeps -o o//depend -r o// llamafile/addnl.c llamafile/bincompare.c llamafile/check_cpu.c llamafile/cuda.c llamafile/extract.c llamafile/get_app_dir.c llamafile/get_tmp_dir.c llamafile/gpu.c llamafile/has.c llamafile/help.c llamafile/is_file_newer_than.c llamafile/launch_browser.c llamafile/llamafile.c llamafile/log.c llamafile/metal.c llamafile/pick_a_warp_kernel.c llamafile/schlep.c llamafile/security.c llamafile/slicehf.c llamafile/x.c llamafile/zip.c llamafile/zipalign.c llamafile/zipcheck.c llamafile/debug.cpp llamafile/explain_a_warp_kernel.cpp llamafile/flags.cpp llamafile/govern.cpp llamafile/iqk_mul_mat_amd_avx2.cpp llamafile/iqk_mul_mat_amd_zen4.cpp llamafile/sgemm.cpp llamafile/sgemm_matmul_test.cpp llamafile/sgemm_sss_test.cpp llamafile/sgemm_vecdot_test.cpp llamafile/simple.cpp llamafile/tinyblas_cpu_mixmul_amd_avx.cpp llamafile/tinyblas_cpu_mixmul_amd_avx2.cpp llamafile/tinyblas_cpu_mixmul_amd_avx512f.cpp llamafile/tinyblas_cpu_mixmul_amd_avxvnni.cpp llamafile/tinyblas_cpu_mixmul_amd_fma.cpp llamafile/tinyblas_cpu_mixmul_amd_zen4.cpp llamafile/tinyblas_cpu_mixmul_arm80.cpp llamafile/tinyblas_cpu_mixmul_arm82.cpp llamafile/tinyblas_cpu_sgemm_amd_avx.cpp llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp llamafile/tinyblas_cpu_sgemm_amd_fma.cpp llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp llamafile/tinyblas_cpu_sgemm_arm80.cpp llamafile/tinyblas_cpu_sgemm_arm82.cpp llamafile/tinyblas_cpu_unsupported.cpp llamafile/tokenize.cpp llamafile/compcap.cu llamafile/cudaprops.cu llamafile/tester.cu llamafile/tinyblas.cu llamafile/tinyblas_test.cu llama.cpp/ggml-alloc.c llama.cpp/ggml-backend.c llama.cpp/ggml-quants-amd-avx.c llama.cpp/ggml-quants-amd-avx2.c llama.cpp/ggml-quants-amd-avx512.c llama.cpp/ggml-quants-arm80.c llama.cpp/ggml-vector-amd-avx.c llama.cpp/ggml-vector-amd-avx2.c llama.cpp/ggml-vector-amd-avx512.c llama.cpp/ggml-vector-amd-avx512bf16.c llama.cpp/ggml-vector-amd-f16c.c llama.cpp/ggml-vector-amd-fma.c llama.cpp/ggml-vector-arm80.c llama.cpp/ggml-vector-arm82.c llama.cpp/ggml.c llama.cpp/stb_image.c llama.cpp/build-info.cpp llama.cpp/common.cpp llama.cpp/console.cpp llama.cpp/ggml-quants.cpp llama.cpp/ggml-vector.cpp llama.cpp/grammar-parser.cpp llama.cpp/json-schema-to-grammar.cpp llama.cpp/llama.cpp llama.cpp/sampling.cpp llama.cpp/unicode-data.cpp llama.cpp/unicode.cpp llama.cpp/llava/clip.cpp llama.cpp/llava/llava-cli.cpp llama.cpp/llava/llava-quantize.cpp llama.cpp/llava/llava.cpp llama.cpp/server/macsandbox.cpp llama.cpp/server/server.cpp llama.cpp/main/embedding.cpp llama.cpp/main/main.cpp llama.cpp/imatrix/imatrix.cpp llama.cpp/quantize/quantize.cpp llama.cpp/perplexity/perplexity.cpp  llamafile/ansiblas.h llamafile/bench.h llamafile/cuda.h llamafile/debug.h llamafile/float.h llamafile/fp16.h llamafile/gemm.h llamafile/half.h llamafile/llamafile.h llamafile/log.h llamafile/macros.h llamafile/micros.h llamafile/naive.h llamafile/numba.h llamafile/sgemm.h llamafile/tester.h llamafile/tinyblas.h llamafile/tinyblas_cpu.h llamafile/version.h llamafile/x.h llamafile/zip.h llama.cpp/base64.h llama.cpp/common.h llama.cpp/console.h llama.cpp/ggml-alloc.h llama.cpp/ggml-backend-impl.h llama.cpp/ggml-backend.h llama.cpp/ggml-common.h llama.cpp/ggml-cuda.h llama.cpp/ggml-impl.h llama.cpp/ggml-metal.h llama.cpp/ggml-quants.h llama.cpp/ggml-vector.h llama.cpp/ggml.h llama.cpp/grammar-parser.h llama.cpp/json-schema-to-grammar.h llama.cpp/json.h llama.cpp/llama.h llama.cpp/llamafile.h llama.cpp/log.h llama.cpp/sampling.h llama.cpp/stb_image.h llama.cpp/unicode-data.h llama.cpp/unicode.h llama.cpp/llava/clip.h llama.cpp/llava/llava.h llama.cpp/server/httplib.h llama.cpp/server/macsandbox.h llama.cpp/server/oai.h llama.cpp/server/server.h llama.cpp/server/utils.h      llamafile/iqk_mul_mat.inc llamafile/tinyblas_cpu_mixmul.inc llamafile/tinyblas_cpu_sgemm.inc llama.cpp/ggml-quants.inc llama.cpp/ggml-vector.inc       
[CI/ubuntu-focal-make]   | build/download-cosmocc.sh .cosmocc/3.3.8 3.3.8 61208872dea249fb9621e950a15f438d2db70b0ca3aa3e91f5e8d0b078fc328d
[CI/ubuntu-focal-make]   | --2024-05-29 10:19:10--  https://github.com/jart/cosmopolitan/releases/download/3.3.8/cosmocc-3.3.8.zip
[CI/ubuntu-focal-make]   | Resolving github.com (github.com)... 20.248.137.48
[CI/ubuntu-focal-make]   | Connecting to github.com (github.com)|20.248.137.48|:443... connected.
[CI/ubuntu-focal-make]   | HTTP request sent, awaiting response... 302 Found
[CI/ubuntu-focal-make]   | Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/272457606/effa5af7-054b-43cc-a17f-1d4a96ffa5f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240529%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240529T101911Z&X-Amz-Expires=300&X-Amz-Signature=9b20c2c03e3a4600e3748f1879a639f682e760f8467079d801321c812e3e9b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=272457606&response-content-disposition=attachment%3B%20filename%3Dcosmocc-3.3.8.zip&response-content-type=application%2Foctet-stream [following]
[CI/ubuntu-focal-make]   | --2024-05-29 10:19:11--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/272457606/effa5af7-054b-43cc-a17f-1d4a96ffa5f5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240529%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240529T101911Z&X-Amz-Expires=300&X-Amz-Signature=9b20c2c03e3a4600e3748f1879a639f682e760f8467079d801321c812e3e9b71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=272457606&response-content-disposition=attachment%3B%20filename%3Dcosmocc-3.3.8.zip&response-content-type=application%2Foctet-stream
[CI/ubuntu-focal-make]   | Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...
[CI/ubuntu-focal-make]   | Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
[CI/ubuntu-focal-make]   | HTTP request sent, awaiting response... 200 OK
[CI/ubuntu-focal-make]   | Length: 186345203 (178M) [application/octet-stream]
[CI/ubuntu-focal-make]   | Saving to: ‘cosmocc.zip’
[CI/ubuntu-focal-make]   | 
[CI/ubuntu-focal-make]   |      0K .......... .......... .......... .......... ..........  0% 2.90M 61s
[CI/ubuntu-focal-make]   |     50K .......... .......... .......... .......... ..........  0% 3.04M 60s
[CI/ubuntu-focal-make]   |    100K .......... .......... .......... .......... ..........  0% 2.84M 61s
... truncated for brevity...
[CI/ubuntu-focal-make]   | 181800K .......... .......... .......... .......... .......... 99% 3.20M 0s
[CI/ubuntu-focal-make]   | 181850K .......... .......... .......... .......... .......... 99% 3.35M 0s
[CI/ubuntu-focal-make]   | 181900K .......... .......... .......... .......... .......... 99% 3.33M 0s
[CI/ubuntu-focal-make]   | 181950K .......... .......... .......                         100% 3.14M=69s
[CI/ubuntu-focal-make]   | 
[CI/ubuntu-focal-make]   | 2024-05-29 10:20:21 (2.56 MB/s) - ‘cosmocc.zip’ saved [186345203/186345203]
[CI/ubuntu-focal-make]   | 
[CI/ubuntu-focal-make]   | cosmocc.zip: OK
[CI/ubuntu-focal-make]   | Archive:  cosmocc.zip
[CI/ubuntu-focal-make]   |    creating: bin/
[CI/ubuntu-focal-make]   |   inflating: bin/ape-x86_64.elf      
[CI/ubuntu-focal-make]   |     linking: bin/x86_64-unknown-cosmo-c++filt  -> x86_64-linux-cosmo-c++filt 
[CI/ubuntu-focal-make]   |     linking: bin/aarch64-unknown-cosmo-objdump  -> aarch64-linux-cosmo-objdump 
[CI/ubuntu-focal-make]   |     linking: bin/x86_64-unknown-cosmo-cc  -> cosmocross 
[CI/ubuntu-focal-make]   |   inflating: bin/x86_64-linux-cosmo-readelf  
[CI/ubuntu-focal-make]   |   inflating: bin/ape-m1.c            
... truncated for brevity...
[CI/ubuntu-focal-make]   |  extracting: aarch64-linux-cosmo/lib/libgomp.a  
[CI/ubuntu-focal-make]   |  extracting: aarch64-linux-cosmo/lib/libgcc_s.a  
[CI/ubuntu-focal-make]   |  extracting: aarch64-linux-cosmo/lib/libresolv.a  
[CI/ubuntu-focal-make]   |   inflating: LICENSE.gpl3            
[CI/ubuntu-focal-make]   | finishing deferred symbolic links:
[CI/ubuntu-focal-make]   |   bin/x86_64-unknown-cosmo-c++filt -> x86_64-linux-cosmo-c++filt
[CI/ubuntu-focal-make]   |   bin/aarch64-unknown-cosmo-objdump -> aarch64-linux-cosmo-objdump
[CI/ubuntu-focal-make]   |   bin/x86_64-unknown-cosmo-cc -> cosmocross
... truncated for brevity...
[CI/ubuntu-focal-make]   |   bin/x86_64-unknown-cosmo-strip -> x86_64-linux-cosmo-strip
[CI/ubuntu-focal-make]   |   libexec/gcc/x86_64-linux-cosmo/12.3.0/ld -> ld.bfd
[CI/ubuntu-focal-make]   |   libexec/gcc/aarch64-linux-cosmo/12.3.0/ld -> ld.bfd
[CI/ubuntu-focal-make]   | # Install ape loader
[CI/ubuntu-focal-make]   | install .cosmocc/3.3.8/bin/ape-x86_64.elf /usr/bin/ape
[CI/ubuntu-focal-make]   | # Config binfmt_misc to use ape loader for ape.elf files
[CI/ubuntu-focal-make]   | echo ':APE:M::MZqFpD::/usr/bin/ape:' > /proc/sys/fs/binfmt_misc/register
[CI/ubuntu-focal-make]   | /bin/sh: 1: cannot create /proc/sys/fs/binfmt_misc/register: Directory nonexistent
[CI/ubuntu-focal-make]   | make: *** [build/rules.mk:47: /usr/bin/ape] Error 2
[CI/ubuntu-focal-make]   | make: *** Deleting file '/usr/bin/ape'
[CI/ubuntu-focal-make]   ❌  Failure - Main Setup cosmocc and ape loader
[CI/ubuntu-focal-make] exitcode '2': failure
[CI/ubuntu-focal-make]   🐳  docker exec cmd=[node /var/run/act/workflow/hashfiles/index.js] user= workdir=
[CI/ubuntu-focal-make] 🏁  Job failed

Additional information

No response

mofosyne commented 4 months ago

If you check the CI log in here https://github.com/Mozilla-Ocho/llamafile/actions/runs/9281962364/job/25538973568 you can observe that the failing step echo ':APE:M::MZqFpD::/usr/bin/ape:' > /proc/sys/fs/binfmt_misc/register passes in github actions in stage 'Setup cosmocc and ape loader'

Fryguy commented 3 months ago

I see bimfmt_misc in my image when I docker run a bash prompt into it.

I'm not an expert on binfmt_misc, but I have hit issues before, particularly with running with images for a different architecture (such as ppc64le or s390x). In those cases, the problem was that binfmt emulators were not installed on the host. I'm a Mac user, and Docker Desktop had this built-in, but when I switched to using colima, it turned out they didn't have binfmt installed by default. The command to add that into your host is [ref]

docker run --privileged --rm tonistiigi/binfmt --install all
mofosyne commented 3 months ago

So I ran that command then attempted to run act again with the same script above, but it still has the same issue. Is there something else you did to make neok act use this binfmt as the base perhaps? @Fryguy