Kernel build, etc.

[djkaosun]

The code reflects what is described in this issue. https://github.com/runborg/vyos-pi-builder/issues/14

See also https://docs.vyos.io/en/latest/contributing/build-vyos.html

[runborg]

Hi @djkaosun , and thanks for a extensive patch-set for this project!

Most of these patches should be easy to get documented to be upstreamed into the main vyos repositories in some form. and i will allow those patches to be merged. But i see you've added references to influxdb apt repos in here. and i cant find any referenced to those repositories from the main vyos repositories. what is the need for those repositories? if they're only local modifications for your build, please remove them from this PR as i dont want to add anything other than mandatory fixes to get vyos built into this project. if there are other significant reasons for adding those please state them and i'll consider it.

Cheers, Runar

[djkaosun]

VyOS uses InfluxData Teregraf package internally, and the lack of a reference to this repository (and its GPG key) causes live-build to fail.

In amd64.toml it is specified as a required package. So maybe I am legit to update arm64.toml in vyos-build.

https://github.com/vyos/vyos-build/blob/current/data/architectures/amd64.toml

[runborg]

Hi! Thanks for a good description of the problem!

I've looked into the issue, and it looks like the vyos apt repository have a copy of telegraf inside it, but only for amd64. packages for arm64 is missing. https://dev.packages.vyos.net/repositories/current/dists/current/main/binary-amd64/Packages vs https://dev.packages.vyos.net/repositories/current/dists/current/main/binary-arm64/Packages

The source of this issue is located here: https://github.com/vyos/vyos-build/blob/dcdf45c216147744d5bc1c5a0d70a8bf3d848ad9/packages/telegraf/Jenkinsfile#L32 , the forth parameter here is "build multiarch", and is here set to "false"... The real fix this to get this changed to "true", but to get a running version of this without waiting for this i would have compiled the package manually using https://github.com/vyos/vyos-build/blob/current/packages/telegraf/build.sh and manually add the package to the build packages directory. this will ensure that the package is the correct version intended to be used.

[runborg]

There have until nov 2022 been hardcoded references to x86 inside this package, but this should now be fixed.. but multiarch is still not enabled by default. https://phabricator.vyos.net/T4831

[runborg]

an unverified patch for this:

$ git diff
diff --git a/build-image.sh b/build-image.sh
index bb08d36..7bfe786 100755
--- a/build-image.sh
+++ b/build-image.sh
@@ -5,6 +5,16 @@ ROOTDIR=$(pwd)
 # Clean out the build-repo and copy all custom packages
 rm -rf vyos-build
 git clone http://github.com/vyos/vyos-build vyos-build
+
+if [ ! -f build/telegraf*.deb ]; then
+       pushd vyos-build/packages/telegraf
+        git clone https://github.com/influxdata/telegraf.git -b v1.23.1 telegraf
+       bash -x ./build.sh
+       popd
+       mkdir build
+       cp vyos-build/packages/telegraf/telegraf/build/dist/telegraf_1.23.1-1_arm64.deb build/
+fi
+
 for a in $(find build -type f -name "*.deb" | grep -v -e "-dbgsym_" -e "libnetfilter-conntrack3-dbg"); do
        echo "Copying package: $a"
        cp $a vyos-build/packages/
@@ -16,9 +26,6 @@ patch -t -u vyos-build/scripts/build-vyos-image < patches/0001_build-vyos-image.
 # Build to arm64.toml
 patch -t -u vyos-build/data/architectures/arm64.toml < patches/0002_arm64.toml.patch

-# Set GPG key of InfluxData repository
-curl https://repos.influxdata.com/influxdb.key > vyos-build/data/live-build-config/archives/influxdb.key.chroot
-
 cd vyos-build

 echo "Copy new default configuration to the vyos image"
diff --git a/patches/0002_arm64.toml.patch b/patches/0002_arm64.toml.patch
index 22c9595..2012bba 100644
--- a/patches/0002_arm64.toml.patch
+++ b/patches/0002_arm64.toml.patch
@@ -1,14 +1,10 @@
 --- a/vyos-build/data/architectures/arm64.toml 2022-12-31 21:26:39.310760935 +0900
 +++ b/vyos-build/data/architectures/arm64.toml 2022-12-31 21:27:31.443059203 +0900
-@@ -1,3 +1,13 @@
+@@ -1,3 +1,9 @@
 -# Packages included in ARM64 images by default
 -packages = ["grub-efi-arm"]
 -bootloaders = "grub-efi"
 \ No newline at end of file
-+additional_repositories = [
-+  "deb [arch=arm64] https://repos.influxdata.com/debian bullseye stable"
-+]
-+
 +kernel_flavor = "arm64-vyos"
 +
 +# Packages added to images for x86 by default

[djkaosun]

Thank you very much. I have applied the patch you gave me.

If we leave make kernel-local and make kernel-registry aside and talk only about patches, it seems to me that the only modifications that are really needed to the vyos-pi-builder project are related to the -v option of the docker build command in Makefile. Perhaps the more correct approach to the remaining issues would be to change vyos-build.

On the other hand, these patches are needed to make vyos-pi-builder work right now. Depending on which is more important for this project, to work now or to be stable under the assumption that vyos-build works properly, will determine how to deal with these issues (It may be the latter).

[runborg]

Thnx, for "fixing" local build of the telegraf package! based on that the other fixes seams to vyos-build seams to be upstreamable, i'll merge this PR, so we have a working image builder again!