search

chainguard-dev / melange

build APKs from source code
Apache License 2.0
426 stars 109 forks source link

cargo pipeline error #1582

Open ckoehler opened 3 weeks ago

ckoehler commented 3 weeks ago

melange is giving an error even tho the build is successful.

Log

❯ melange build --arch aarch64 test.yaml
2024/10/21 15:46:45 INFO melange is building:
2024/10/21 15:46:45 INFO   configuration file: test.yaml
2024/10/21 15:46:45 INFO   workspace dir: /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/melange-workspace-1968109458
2024/10/21 15:46:45 INFO evaluating pipelines for package requirements
2024/10/21 15:46:45 INFO --cache-dir ./melange-cache/ not a dir; skipping
2024/10/21 15:46:45 INFO populating workspace /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/melange-workspace-1968109458 from .
2024/10/21 15:46:45 INFO building workspace in '/var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/melange-guest-4117346961' with apko
2024/10/21 15:46:46 INFO setting apk repositories: [https://packages.wolfi.dev/os]
2024/10/21 15:46:46 INFO image configuration:
2024/10/21 15:46:46 INFO   contents:
2024/10/21 15:46:46 INFO     build repositories: []
2024/10/21 15:46:46 INFO     runtime repositories: [https://packages.wolfi.dev/os]
2024/10/21 15:46:46 INFO     keyring:      [https://packages.wolfi.dev/os/wolfi-signing.rsa.pub]
2024/10/21 15:46:46 INFO     packages:     [build-base busybox cargo-auditable rust]
2024/10/21 15:46:46 INFO   accounts:
2024/10/21 15:46:46 INFO     runas:
2024/10/21 15:46:46 INFO     users:
2024/10/21 15:46:46 INFO       - uid=1000(build) gid=1000
2024/10/21 15:46:46 INFO     groups:
2024/10/21 15:46:46 INFO       - gid=1000(build) members=[build]
2024/10/21 15:46:46 INFO auth configured for: []
2024/10/21 15:46:47 INFO installing ca-certificates-bundle (20241010-r0)
2024/10/21 15:46:47 INFO installing wolfi-baselayout (20230201-r15)
2024/10/21 15:46:47 INFO installing glibc (2.40-r3)
2024/10/21 15:46:47 INFO installing libgcc (14.2.0-r3)
2024/10/21 15:46:47 INFO installing ld-linux (2.40-r3)
2024/10/21 15:46:47 INFO installing glibc-locale-posix (2.40-r2)
2024/10/21 15:46:47 INFO installing make (4.4.1-r3)
2024/10/21 15:46:47 INFO installing libzstd1 (1.5.6-r5)
2024/10/21 15:46:47 INFO installing libstdc++ (14.2.0-r3)
2024/10/21 15:46:47 INFO installing binutils (2.43.1-r1)
2024/10/21 15:46:47 INFO installing pkgconf (2.3.0-r1)
2024/10/21 15:46:47 INFO installing posix-cc-wrappers (1-r3)
2024/10/21 15:46:47 INFO installing libgo (14.2.0-r3)
2024/10/21 15:46:47 INFO installing gmp (6.3.0-r2)
2024/10/21 15:46:47 INFO installing isl (0.26-r4)
2024/10/21 15:46:47 INFO installing mpfr (4.2.1-r4)
2024/10/21 15:46:47 INFO installing mpc (1.3.1-r5)
2024/10/21 15:46:47 INFO installing zlib (1.3.1-r4)
2024/10/21 15:46:47 INFO installing libstdc++-dev (14.2.0-r3)
2024/10/21 15:46:47 INFO installing libatomic (14.2.0-r3)
2024/10/21 15:46:47 INFO installing libgomp (14.2.0-r3)
2024/10/21 15:46:47 INFO installing gcc (14.2.0-r3)
2024/10/21 15:46:47 INFO installing libxcrypt (4.4.36-r8)
2024/10/21 15:46:47 INFO installing libxcrypt-dev (4.4.36-r8)
2024/10/21 15:46:47 INFO installing linux-headers (6.6.29-r1)
2024/10/21 15:46:47 INFO installing nss-db (2.40-r3)
2024/10/21 15:46:47 INFO installing nss-hesiod (2.40-r3)
2024/10/21 15:46:47 INFO installing glibc-dev (2.40-r3)
2024/10/21 15:46:48 INFO installing build-base (1-r8)
2024/10/21 15:46:48 INFO installing libcrypt1 (2.40-r3)
2024/10/21 15:46:48 INFO installing busybox (1.37.0-r0)
2024/10/21 15:46:48 INFO installing cargo-auditable (0.6.4-r1)
2024/10/21 15:46:48 INFO installing libffi (3.4.6-r5)
2024/10/21 15:46:48 INFO installing xz (5.6.3-r0)
2024/10/21 15:46:48 INFO installing libxml2 (2.13.4-r1)
2024/10/21 15:46:48 INFO installing libLLVM-18 (18.1.8-r3)
2024/10/21 15:46:48 INFO installing libcrypto3 (3.3.2-r2)
2024/10/21 15:46:48 INFO installing libssl3 (3.3.2-r2)
2024/10/21 15:46:48 INFO installing libunistring (1.3-r0)
2024/10/21 15:46:48 INFO installing libidn2 (2.3.7-r2)
2024/10/21 15:46:48 INFO installing libpsl (0.21.5-r3)
2024/10/21 15:46:48 INFO installing libbrotlicommon1 (1.1.0-r4)
2024/10/21 15:46:48 INFO installing libbrotlidec1 (1.1.0-r4)
2024/10/21 15:46:48 INFO installing krb5-conf (1.0-r3)
2024/10/21 15:46:48 INFO installing keyutils-libs (1.6.3-r4)
2024/10/21 15:46:48 INFO installing libverto (0.3.2-r4)
2024/10/21 15:46:48 INFO installing libcom_err (1.47.1-r0)
2024/10/21 15:46:48 INFO installing krb5-libs (1.21.3-r1)
2024/10/21 15:46:48 INFO installing gdbm (1.24-r0)
2024/10/21 15:46:48 INFO installing ncurses-terminfo-base (6.5_p20241006-r3)
2024/10/21 15:46:48 INFO installing ncurses (6.5_p20241006-r3)
2024/10/21 15:46:48 INFO installing readline (8.2.13-r0)
2024/10/21 15:46:48 INFO installing sqlite-libs (3.46.1-r0)
2024/10/21 15:46:48 INFO installing heimdal-libs (7.8.0-r7)
2024/10/21 15:46:48 INFO installing cyrus-sasl (2.1.28-r4)
2024/10/21 15:46:48 INFO installing libevent (2.1.12-r6)
2024/10/21 15:46:48 INFO installing libldap (2.6.8-r5)
2024/10/21 15:46:48 INFO installing libnghttp2-14 (1.63.0-r0)
2024/10/21 15:46:48 INFO installing libcurl-openssl4 (8.10.1-r1)
2024/10/21 15:46:48 INFO installing rust-1.81 (1.81.0-r2)
2024/10/21 15:46:50 INFO built image layer tarball as /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/apko-temp-3050087311/apko-aarch64.tar.gz
2024/10/21 15:46:50 INFO using /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/apko-temp-3050087311/apko-aarch64.tar.gz for image layer
2024/10/21 15:46:50 INFO OCI layer digest: sha256:2cded9e21fef217b0e8f54c94df2fe2d4955b49044217043503dba9b02b18866
2024/10/21 15:46:50 INFO OCI layer diffID: sha256:d30442f425b1b230ab863f52025ecb1dc66fb124c60e8abea85bf41b17576d4a
2024/10/21 15:46:50 INFO saving OCI image locally: apko.local/cache:bb863e2da1cbf21b67f908c8bf88c50b6622b8375be5d7d6f12d5d2be83f94bf
2024/10/21 15:46:50 WARN skipping local domain tagging apko.local/cache:bb863e2da1cbf21b67f908c8bf88c50b6622b8375be5d7d6f12d5d2be83f94bf as index.docker.io/library/melange:latest
2024/10/21 15:46:50 INFO ImgRef = apko.local/cache:bb863e2da1cbf21b67f908c8bf88c50b6622b8375be5d7d6f12d5d2be83f94bf
2024/10/21 15:46:50 INFO running step "cargo/build"
2024/10/21 15:46:50 WARN    Compiling melange-test v0.1.0 (/home/build) uses=cargo/build
2024/10/21 15:46:50 WARN     Finished `release` profile [optimized] target(s) in 0.38s uses=cargo/build
2024/10/21 15:46:50 WARN install: omitting directory 'target/release/build' uses=cargo/build
2024/10/21 15:46:50 WARN install: omitting directory 'target/release/deps' uses=cargo/build
2024/10/21 15:46:50 WARN install: omitting directory 'target/release/examples' uses=cargo/build
2024/10/21 15:46:50 WARN install: omitting directory 'target/release/incremental' uses=cargo/build
2024/10/21 15:46:50 INFO pod a36ad1fa8fff2c0493938ff4b27a9fd4331494ca7c5dad1f753da7ad0d0410f8 terminated
2024/10/21 15:46:51 INFO deleting guest dir /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/melange-guest-4117346961
2024/10/21 15:46:51 INFO deleting workspace dir /var/folders/pl/pj8rn0910g75pms3yx__nkt00000gn/T/melange-workspace-1968109458
2024/10/21 15:46:51 INFO deleting image apko.local/cache:bb863e2da1cbf21b67f908c8bf88c50b6622b8375be5d7d6f12d5d2be83f94bf
2024/10/21 15:46:51 INFO untagged apko.local/cache:bb863e2da1cbf21b67f908c8bf88c50b6622b8375be5d7d6f12d5d2be83f94bf
2024/10/21 15:46:51 INFO deleted sha256:eb944017111f335376f14158d99dad1a1144b15270cb8ee2c3ddbb267f46cae5
2024/10/21 15:46:51 INFO deleted sha256:d30442f425b1b230ab863f52025ecb1dc66fb124c60e8abea85bf41b17576d4a
2024/10/21 15:46:51 ERRO failed to build package: unable to run package cargo-test pipeline: unable to run pipeline: unable to run pipeline: task exited with code 1

Reproduce

Create a new Rust project, the default works.

cargo new melange-test; cd melange-test

Create a melange file with this content:

package:
  name: cargo-test
  version: 0.1.0
  description: Test cargo pipeline
  copyright:
    - license: MIT
  dependencies:
    runtime:

environment:
  contents:
    keyring:
      - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    repositories:
      - https://packages.wolfi.dev/os
    packages: 
      - build-base 

pipeline:
  - uses: cargo/build

Build: melange build --arch amd64 test.yaml Arch doesn't matter, just keeping the output shorter. aarch64 also fails.

Version

  __  __   _____   _          _      _   _    ____   _____
 |  \/  | | ____| | |        / \    | \ | |  / ___| | ____|
 | |\/| | |  _|   | |       / _ \   |  \| | | |  _  |  _|
 | |  | | | |___  | |___   / ___ \  | |\  | | |_| | | |___
 |_|  |_| |_____| |_____| /_/   \_\ |_| \_|  \____| |_____|
melange

GitVersion:    0.13.7
GitCommit:     brew
GitTreeState:  clean
BuildDate:     2024-10-21T16:50:36Z
GoVersion:     go1.23.2
Compiler:      gc
Platform:      darwin/arm64
ckoehler commented 2 weeks ago

Same thing with Docker:

❯ docker run --rm --privileged -v "$(pwd)":/work cgr.dev/chainguard/melange build --arch aarch64 test.yaml

❯ docker run --rm --privileged -v "$(pwd)":/work  cgr.dev/chainguard/melange version
  __  __   _____   _          _      _   _    ____   _____
 |  \/  | | ____| | |        / \    | \ | |  / ___| | ____|
 | |\/| | |  _|   | |       / _ \   |  \| | | |  _  |  _|
 | |  | | | |___  | |___   / ___ \  | |\  | | |_| | | |___
 |_|  |_| |_____| |_____| /_/   \_\ |_| \_|  \____| |_____|
melange

GitVersion:    v0.14.7
GitCommit:     71c88c32251b71894867172d355a13ee703f86ec
GitTreeState:  clean
BuildDate:     '2024-10-23T15:39:27Z'
GoVersion:     go1.23.2
Compiler:      gc
Platform:      linux/arm64
ckoehler commented 2 weeks ago

A few more debug logs:

[...]
2024/10/23 18:08:11 DEBU finished building filesystem
2024/10/23 18:08:13 INFO built image layer tarball as /tmp/apko-temp-916801685/apko-aarch64.tar.gz
2024/10/23 18:08:13 INFO using /tmp/apko-temp-916801685/apko-aarch64.tar.gz for image layer
2024/10/23 18:08:18 DEBU pushed /tmp/apko-temp-916801685/apko-aarch64.tar.gz as /tmp/melange-guest-3950364476
2024/10/23 18:08:18 DEBU successfully built workspace with apko
2024/10/23 18:08:18 INFO ImgRef = /tmp/melange-guest-3950364476
2024/10/23 18:08:18 DEBU executing: bwrap --bind /tmp/melange-guest-3950364476 / --bind /tmp/melange-workspace-3540092415 /home/build --bind /etc/resolv.conf /etc/resolv.conf --unshare-pid --die-with-parent --dev /dev --proc /proc --ro-bind /sys /sys --chdir /home/build --clearenv --new-session --setenv SOURCE_DATE_EPOCH 0 --setenv GOMODCACHE /var/cache/melange/gomodcache --setenv HOME /home/build --setenv GOPATH /home/build/.cache/go /bin/sh -c [ -x /sbin/ldconfig ] && /sbin/ldconfig /lib || true
2024/10/23 18:08:19 DEBU running the main pipeline
2024/10/23 18:08:19 INFO running step "cargo/build"
2024/10/23 18:08:19 DEBU executing: bwrap --bind /tmp/melange-guest-3950364476 / --bind /tmp/melange-workspace-3540092415 /home/build --bind /etc/resolv.conf /etc/resolv.conf --unshare-pid --die-with-parent --dev /dev --proc /proc --ro-bind /sys /sys --chdir /home/build --clearenv --new-session --setenv GOPATH /home/build/.cache/go --setenv SOURCE_DATE_EPOCH 0 --setenv GOMODCACHE /var/cache/melange/gomodcache --setenv HOME /home/build --setenv PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /bin/sh -c set -e
[ -d '/home/build' ] || mkdir -p '/home/build'
cd '/home/build'

exit 0 uses=cargo/build
2024/10/23 18:08:19 DEBU executing: bwrap --bind /tmp/melange-guest-3950364476 / --bind /tmp/melange-workspace-3540092415 /home/build --bind /etc/resolv.conf /etc/resolv.conf --unshare-pid --die-with-parent --dev /dev --proc /proc --ro-bind /sys /sys --chdir /home/build --clearenv --new-session --setenv SOURCE_DATE_EPOCH 0 --setenv GOMODCACHE /var/cache/melange/gomodcache --setenv HOME /home/build --setenv GOPATH /home/build/.cache/go --setenv PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /bin/sh -c set -e
[ -d '/home/build' ] || mkdir -p '/home/build'
cd '/home/build'
# Installation directory should always be bin as we are producing a binary
INSTALL_PATH="/home/build/melange-out/cargo-test/usr/bin"
OUTPUT_PATH="target/release"

# Enter target package directory
cd "."

# Build and install package(s)
cargo auditable build "--release"
if [[ ! -z "" ]]; then
  install -Dm755 "${OUTPUT_PATH}/" "${INSTALL_PATH}/"
else
  install -Dm755 "${OUTPUT_PATH}"/* -t "${INSTALL_PATH}"
fi

exit 0 uses=cargo/build
2024/10/23 18:08:19 WARN    Compiling melange-test v0.1.0 (/home/build) uses=cargo/build
2024/10/23 18:08:19 WARN     Finished `release` profile [optimized] target(s) in 0.21s uses=cargo/build
2024/10/23 18:08:19 WARN install: omitting directory 'target/release/build' uses=cargo/build
2024/10/23 18:08:19 WARN install: omitting directory 'target/release/deps' uses=cargo/build
2024/10/23 18:08:19 WARN install: omitting directory 'target/release/examples' uses=cargo/build
2024/10/23 18:08:19 WARN install: omitting directory 'target/release/incremental' uses=cargo/build
2024/10/23 18:08:19 INFO deleting guest dir /tmp/melange-guest-3571777626
2024/10/23 18:08:19 INFO deleting workspace dir /tmp/melange-workspace-3540092415
2024/10/23 18:08:19 INFO removing image path /tmp/melange-guest-3950364476
2024/10/23 18:08:19 ERRO failed to build package: unable to run package cargo-test pipeline: unable to run pipeline: unable to run pipeline: exit status 1
kranurag7 commented 2 weeks ago

@ckoehler where is the source that you're trying to build, generally we checkout a project using git-checkout that we want to build. I don't see a git-checkout in your config, you can use maybe fetch too if you want to pull a tarball. You'll be able to find examples for fetch in wolfi repository. Let me know if you are able to make some more progress after this.

There's also a useful make debug/geckodriver make target that'll put your in shell if the build fails and then you can inspect more. Something like you can ensure if you're in right directory or invoke cargo auditable build directly.

ckoehler commented 2 weeks ago

It's the current directory, mapped into work. The steps to reproduce this are in the issue above. The cargo build step succeeds, but the package generation fails.

ckoehler commented 2 weeks ago

Maybe there's also a workflow problem? I am usually packaging local apps in CI, so the source is available from a git checkout already, no need to fetch anything remotely. What's the best way to do that?

kranurag7 commented 2 weeks ago

@ckoehler the honest answer for now is that I lack this context as of now. I've never tried building local packages, I can look into this tomorrow if it's possible but generally whole packages fetch the source code from internet and build it from source.

I'll look into this tomorrow if this is still an issue that's not solved.

ckoehler commented 2 weeks ago

Here's a self-contained yaml file that pulls and builds a project from git. The cargo build is successful, but the package generation isn't.

package:
  name: probe
  version: 0.1.0
  description: probe
  copyright:
    - license: MIT
  dependencies:
    runtime:

environment:
  contents:
    keyring:
      - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    repositories:
      - https://packages.wolfi.dev/os
    packages: 

pipeline:
  - uses: git-checkout
    with:
      expected-commit: 5092046571b4791f68a47c29527a14d6d57aee3a
      repository: https://github.com/ckoehler/probe.git
      tag: v0.2.2
  - uses: cargo/build

I built it with Docker and the dev-container method, including make debug/probe, which puts me in a shell; there, cargo auditable build succeeds as well (the warning can be ignored). It's something after the build step, in the package generation or something.

kranurag7 commented 2 weeks ago

@ckoehler this should keep you moving. 

package:
  name: probe
  version: 0.1.0
  description: probe
  copyright:
    - license: MIT

environment:
  contents:
    packages:
      - build-base

pipeline:
  - uses: git-checkout
    with:
      expected-commit: 5092046571b4791f68a47c29527a14d6d57aee3a
      repository: https://github.com/ckoehler/probe.git
      tag: v0.2.2
  - uses: cargo/build
    with:
      output: probe

Notes about why it was failing can be read in this PR notes: https://github.com/chainguard-dev/melange/pull/1596

I saw you've two binaries pub and probe but we will not be able to move with two binaries installation for now. Maybe there's a workaround.

But this is something where we plan to improve in the future as we plan to adopt cargo/build more and more.

Thanks for being patient.

ckoehler commented 2 weeks ago

Ah interesting! The actual repo I have trouble with actually has no binaries, but a couple of libraries (rlib). Does the same thing apply to that case?

ckoehler commented 2 weeks ago

I guess it does, I didn't know the install command would exit with 1 if it encounters directories. So that wildcard install never would've worked.