search

threefoldtech / 0-initramfs

Zero-OS Initramfs Builder
Apache License 2.0
2 stars 1 forks source link

Build using docker container does not work: no output in staging #67

Closed Totodebrant closed 1 year ago

Totodebrant commented 1 year ago

Following: https://github.com/threefoldtech/0-initramfs#build-using-a-docker-container

Exact steps taken:

  1. docker run --platform=linux/amd64 -ti -v "/Users/toto/dev/naiein/threefold/0-initramfs":/0-initramfs ubuntu:18.04 /bin/bash
  2. cd 0-initramfs/
  3. autobuild/tf-build.sh

These are the final output lines:

Processing triggers for ca-certificates (20211016ubuntu0.18.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  117M  100  117M    0     0  14.5M      0  0:00:08  0:00:08 --:--:-- 16.0M
/usr/bin/head: error reading '/proc/self/exe': Bad file descriptor
/usr/bin/head: failed to close '/proc/self/exe': Bad file descriptor
rustup: unknown platform bitness
/bin/sh: 375: [: Illegal number:
info: downloading installer
info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2023-03-28, rust version 1.68.2 (9eb3afe9e 2023-03-27)
info: downloading component 'cargo'
  6.7 MiB /   6.7 MiB (100 %)   3.6 MiB/s in  2s ETA:  0s
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)   8.7 MiB/s in  2s ETA:  0s
info: downloading component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)  13.1 MiB/s in  2s ETA:  0s
info: downloading component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)  19.4 MiB/s in  3s ETA:  0s
info: downloading component 'rustfmt'
info: installing component 'cargo'
  6.7 MiB /   6.7 MiB (100 %)   6.0 MiB/s in  1s ETA:  0s
info: installing component 'clippy'
info: installing component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)   1.9 MiB/s in 14s ETA:  0s
info: installing component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)   4.9 MiB/s in  6s ETA:  0s
info: installing component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)   6.2 MiB/s in 11s ETA:  0s
info: installing component 'rustfmt'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu installed - rustc 1.68.2 (9eb3afe9e 2023-03-27)

Rust is installed now. Great!

To get started you may need to restart your current shell.
This would reload your PATH environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: installing component 'rust-std' for 'x86_64-unknown-linux-musl'
 43.5 MiB /  43.5 MiB (100 %)   4.8 MiB/s in  8s ETA:  0s

Nothing in staging (expected: staging/vmlinuz.efi)

Any idea what's going wrong?

maxux commented 1 year ago

It seems that script didn't executes the build process. Can you try again (on a fresh docker) but using bash -x autobuild/tf-build.sh to debug a bit the process, thanks :)

Totodebrant commented 1 year ago

So this is the final output with -x:

Setting up liblvm2cmd2.02:amd64 (2.02.176-4.1ubuntu3.18.04.3) ...
Setting up dmeventd (2:1.02.145-4.1ubuntu3.18.04.3) ...
Setting up dh-autoreconf (17) ...
Setting up libicu-le-hb-dev:amd64 (1.0.3+git161113-4) ...
Setting up lvm2 (2.02.176-4.1ubuntu3.18.04.3) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libicu-dev (60.2-3ubuntu3.2) ...
Setting up libxml2-dev:amd64 (2.9.4+dfsg1-6.1ubuntu1.8) ...
Setting up dh-strip-nondeterminism (0.040-1.1~build1) ...
Setting up libharfbuzz-dev:amd64 (1.7.2-1ubuntu1) ...
Setting up debhelper (11.1.6ubuntu2) ...
Setting up xtables-addons-source (3.0-0.1ubuntu5) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.6) ...
Processing triggers for ca-certificates (20211016ubuntu0.18.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
++ GOVER=1.14.1
++ curl -L https://dl.google.com/go/go1.14.1.linux-amd64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  117M  100  117M    0     0  14.2M      0  0:00:08  0:00:08 --:--:-- 20.2M
++ tar -C /usr/local -xzf /tmp/go1.14.1.linux-amd64.tar.gz
++ mkdir -p /gopath
++ curl https://sh.rustup.rs -sSf
++ sh -s -- -y
/usr/bin/head: error reading '/proc/self/exe': Bad file descriptor
/usr/bin/head: failed to close '/proc/self/exe': Bad file descriptor
rustup: unknown platform bitness
/bin/sh: 375: [: Illegal number:
info: downloading installer
info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2023-03-28, rust version 1.68.2 (9eb3afe9e 2023-03-27)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)  17.4 MiB/s in  1s ETA:  0s
info: downloading component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)  20.9 MiB/s in  1s ETA:  0s
info: downloading component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)  19.6 MiB/s in  3s ETA:  0s
info: downloading component 'rustfmt'
info: installing component 'cargo'
  6.7 MiB /   6.7 MiB (100 %)   5.8 MiB/s in  1s ETA:  0s
info: installing component 'clippy'
info: installing component 'rust-docs'
 19.5 MiB /  19.5 MiB (100 %)   1.9 MiB/s in 14s ETA:  0s
info: installing component 'rust-std'
 29.9 MiB /  29.9 MiB (100 %)   5.0 MiB/s in  5s ETA:  0s
info: installing component 'rustc'
 68.1 MiB /  68.1 MiB (100 %)   6.3 MiB/s in 10s ETA:  0s
info: installing component 'rustfmt'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu installed - rustc 1.68.2 (9eb3afe9e 2023-03-27)

Rust is installed now. Great!

To get started you may need to restart your current shell.
This would reload your PATH environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"
++ source /root/.cargo/env
+++ case ":${PATH}:" in
+++ export PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+++ PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ rustup target add x86_64-unknown-linux-musl
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: installing component 'rust-std' for 'x86_64-unknown-linux-musl'
 43.5 MiB /  43.5 MiB (100 %)   5.2 MiB/s in  8s ETA:  0s
++ dirname autobuild/tf-build.sh
+ . autobuild/tf-build-settings.sh
++ export PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin
++ PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin
++ export GOPATH=/gopath
++ GOPATH=/gopath
++ source /root/.cargo/env
+++ case ":${PATH}:" in
+ arguments='--all --compact '
+ '[' '' = release ']'
+ export INTERACTIVE=false
+ INTERACTIVE=false
++ dirname autobuild/tf-build.sh
+ cd autobuild/..
+ bash initramfs.sh --all --compact
Totodebrant commented 1 year ago

initramfs.sh is not in the autobuild folder, but in /0-initramfs, I suspect this might be the problem.

maxux commented 1 year ago

Can you run bash -x initramfs.sh --all --compact ? That's weird that this scripts doesn't returns anything

maxux commented 1 year ago

initramfs.sh is not in the autobuild folder, but in /0-initramfs, I suspect this might be the problem.

There is a:

+ cd autobuild/..

Which should be fine

Totodebrant commented 1 year ago

Output:

root@01383ee4fc60:/0-initramfs# bash -x initramfs.sh --all --compact
+ set -e
+ BUILDMODE=debug
+ DISTFILES=/0-initramfs/archives
+ WORKDIR=/0-initramfs/staging
+ CONFDIR=/0-initramfs/config
+ ROOTDIR=/0-initramfs/root
+ TMPDIR=/0-initramfs/tmp
+ PKGDIR=/0-initramfs/packages
+ EXTENDIR=/0-initramfs/extensions
+ PATCHESDIR=/0-initramfs/patches
+ TOOLSDIR=/0-initramfs/tools
+ MUSLWORKDIR=/0-initramfs/staging/musl
+ MUSLROOTDIR=/0-initramfs/staging/musl/root
+ MIRRORSRC=https://download.grid.tf/initramfs-mirror/
++ grep -c bogomips /proc/cpuinfo
+ JOBS=1
maxux commented 1 year ago

I suspect some buffering, there is no reasons script stop there, does it returns an error ? with echo $?

Totodebrant commented 1 year ago 
root@01383ee4fc60:/0-initramfs# echo $?
1
Totodebrant commented 1 year ago

So root@01383ee4fc60:/0-initramfs# grep -c bogomips /proc/cpuinfo gives 0, I suspect this might have something to do with the fact I'm running an m1 processor.

maxux commented 1 year ago

Ahhh, your host is a macOS ? That's possible. I wonder what's going to happen by crosscompiling everything as well.

But that line should not be an issue, grep -c returns 0 then script adds 1, and we have confirmation it works:

++ grep -c bogomips /proc/cpuinfo
+ JOBS=1

JOBS is well set, 1 is not optimal for sure but it's not an issue. I don't see why it fails without giving any output.

Totodebrant commented 1 year ago

Yes, it's macos. Running --platform=linux/amd64 for my Docker as well. Anyway we can debug this further?

maxux commented 1 year ago

Except running each commands from initramfs.sh manually to see which one could cause trouble, I don't really see :/

Totodebrant commented 1 year ago

Alright, so I found the issue on this line: JOBS=$(($(grep -c 'bogomips' /proc/cpuinfo) + 1)). Since set -e was used, it exits when any command exits with a non zero status. This is not an issue for the job calculation but it still exits. Using the following code solved this issue: JOBS=$(($(grep -c 'bogomips' /proc/cpuinfo || true) + 1)).

Now it gets a lot further but hangs on:

+ main
+ info ===============================
+ echo -e '\033[34;1m===============================\033[0m'
===============================
+ info '=  Zero-OS Initramfs Builder  ='
+ echo -e '\033[34;1m=  Zero-OS Initramfs Builder  =\033[0m'
=  Zero-OS Initramfs Builder  =
+ info ===============================
+ echo -e '\033[34;1m===============================\033[0m'
===============================
+ echo ''

+ prepare
+ '[' '!' -d /0-initramfs/config ']'
+ '[' -z ']'
+ command -v go
+ echo '[-] variable GOPATH not defined'
[-] variable GOPATH not defined
+ exit 1

This one should have been set by the script, no?