Closed rluzuriaga closed 1 year ago
Sounds good to me!
@rluzuriaga I just stumbled upon an interesting difference when building with GH Actions:
./build.sh -nfa x64
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
Checking packages needed for building
* WARNING: Did not find a patch file building filesystem without patches!
Preparing buildroot 2022.02.9 on x64 build:
* Downloading buildroot source package........................Done
* Extracting buildroot sources................................Done
* Preparing code..............................................mkdir -p /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/lxdialog
PKG_CONFIG_PATH="" make CC="/usr/bin/gcc" HOSTCC="/usr/bin/gcc" \
obj=/home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config -C support/kconfig -f Makefile.br conf
make[1]: Entering directory '/home/runner/work/fos/fos/fssourcex64/support/kconfig'
/usr/bin/gcc -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DLOCALE -I/home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -MM *.c > /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/.depend 2>/dev/null || :
/usr/bin/gcc -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DLOCALE -I/home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -c conf.c -o /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/conf.o
/usr/bin/gcc -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DLOCALE -I/home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -I. -c /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/zconf.tab.c -o /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/zconf.tab.o
/usr/bin/gcc -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DLOCALE -I/home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/conf.o /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/zconf.tab.o -o /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/conf
rm /home/runner/work/fos/fos/fssourcex64/output/build/buildroot-config/zconf.tab.c
make[1]: Leaving directory '/home/runner/work/fos/fos/fssourcex64/support/kconfig'
Error in reading or end of file.
*
* Restart config...
*
*
* mpd
*
mpd (BR2_PACKAGE_MPD) [N/y/?] (NEW)
Error in reading or end of file.
Error in reading or end of file.
*
Error in reading or end of file.
* Interpreter languages and scripting
*
4th (BR2_PACKAGE_4TH) [N/y/?] n
enscript (BR2_PACKAGE_ENSCRIPT) [N/y/?] n
erlang (BR2_PACKAGE_ERLANG) [N/y/?] n
execline (BR2_PACKAGE_EXECLINE) [N/y/?] n
ficl (BR2_PACKAGE_FICL) [N/y/?] n
gauche (BR2_PACKAGE_GAUCHE) [N/y/?] n
guile (BR2_PACKAGE_GUILE) [N/y/?] n
haserl (BR2_PACKAGE_HASERL) [N/y/?] n
janet (BR2_PACKAGE_JANET) [N/y/?] n
jimtcl (BR2_PACKAGE_JIMTCL) [N/y/?] n
lua (BR2_PACKAGE_LUA) [N/y/?] n
luajit (BR2_PACKAGE_LUAJIT) [N/y/?] n
micropython (BR2_PACKAGE_MICROPYTHON) [N/y/?] n
moarvm (BR2_PACKAGE_MOARVM) [N/y/?] n
mono (BR2_PACKAGE_MONO) [N/y/?] n
nodejs (BR2_PACKAGE_NODEJS) [N/y/?] (NEW)
*
* openjdk needs X.Org
*
perl (BR2_PACKAGE_PERL) [N/y/?] n
php (BR2_PACKAGE_PHP) [N/y/?] n
python3 (BR2_PACKAGE_PYTHON3) [N/y/?] n
quickjs (BR2_PACKAGE_QUICKJS) [N/y/?] n
ruby (BR2_PACKAGE_RUBY) [N/y/?] n
tcl (BR2_PACKAGE_TCL) [N/y/?] n
*
* Host utilities
*
host abootimg (BR2_PACKAGE_HOST_ABOOTIMG) [N/y/?] n
host aespipe (BR2_PACKAGE_HOST_AESPIPE) [N/y/?] n
host android-tools (BR2_PACKAGE_HOST_ANDROID_TOOLS) [N/y/?] n
host asn1c (BR2_PACKAGE_HOST_ASN1C) [N/y/?] n
host babeltrace2 (BR2_PACKAGE_HOST_BABELTRACE2) [N/y/?] n
host bmap-tools (BR2_PACKAGE_HOST_BMAP_TOOLS) [N/y/?] n
host btrfs-progs (BR2_PACKAGE_HOST_BTRFS_PROGS) [N/y/?] n
host checkpolicy (BR2_PACKAGE_HOST_CHECKPOLICY) [N/y/?] n
host checksec (BR2_PACKAGE_HOST_CHECKSEC) [N/y/?] n
host cmake (BR2_PACKAGE_HOST_CMAKE) [N/y/?] n
host cramfs (BR2_PACKAGE_HOST_CRAMFS) [N/y/?] n
host cryptsetup (BR2_PACKAGE_HOST_CRYPTSETUP) [N/y/?] n
host dbus-python (BR2_PACKAGE_HOST_DBUS_PYTHON) [N/y/?] n
host delve (BR2_PACKAGE_HOST_DELVE) [N/y/?] n
host dfu-util (BR2_PACKAGE_HOST_DFU_UTIL) [N/y/?] n
host dos2unix (BR2_PACKAGE_HOST_DOS2UNIX) [N/y/?] n
host dosfstools (BR2_PACKAGE_HOST_DOSFSTOOLS) [N/y/?] n
host doxygen (BR2_PACKAGE_HOST_DOXYGEN) [N/y/?] n
host dtc (BR2_PACKAGE_HOST_DTC) [N/y/?] n
host e2fsprogs (BR2_PACKAGE_HOST_E2FSPROGS) [Y/?] y
host e2tools (BR2_PACKAGE_HOST_E2TOOLS) [N/y/?] n
host environment-setup (BR2_PACKAGE_HOST_ENVIRONMENT_SETUP) [N/y/?] n
host erofs-utils (BR2_PACKAGE_HOST_EROFS_UTILS) [N/y/?] n
host exfatprogs (BR2_PACKAGE_HOST_EXFATPROGS) [N/y/?] n
host f2fs-tools (BR2_PACKAGE_HOST_F2FS_TOOLS) [N/y/?] n
host faketime (BR2_PACKAGE_HOST_FAKETIME) [N/y/?] n
host fatcat (BR2_PACKAGE_HOST_FATCAT) [N/y/?] n
host firmware-utils (BR2_PACKAGE_HOST_FIRMWARE_UTILS) [N/y/?] n
host fwup (BR2_PACKAGE_HOST_FWUP) [N/y/?] n
host genext2fs (BR2_PACKAGE_HOST_GENEXT2FS) [Y/n/?] y
host genimage (BR2_PACKAGE_HOST_GENIMAGE) [N/y/?] n
host genpart (BR2_PACKAGE_HOST_GENPART) [N/y/?] n
host gnupg (BR2_PACKAGE_HOST_GNUPG) [N/y/?] n
host gptfdisk (BR2_PACKAGE_HOST_GPTFDISK) [Y/n/?] y
host imagemagick (BR2_PACKAGE_HOST_IMAGEMAGICK) [N/y/?] n
host imx-mkimage (BR2_PACKAGE_HOST_IMX_MKIMAGE) [N/y/?] n
host jq (BR2_PACKAGE_HOST_JQ) [Y/n/?] y
host jsmin (BR2_PACKAGE_HOST_JSMIN) [N/y/?] n
host kmod (BR2_PACKAGE_HOST_KMOD) [N/y/?] n
host libp11 (BR2_PACKAGE_HOST_LIBP11) [N/y/?] n
host lld (BR2_PACKAGE_HOST_LLD) [N/y/?] n
host lpc3250loader (BR2_PACKAGE_HOST_LPC3250LOADER) [N/y/?] n
host lttng-babeltrace (BR2_PACKAGE_HOST_LTTNG_BABELTRACE) [N/y/?] n
host mender-artifact (BR2_PACKAGE_HOST_MENDER_ARTIFACT) [N/y/?] n
host meson-tools (BR2_PACKAGE_HOST_MESON_TOOLS) [N/y/?] n
host mkpasswd (BR2_PACKAGE_HOST_MKPASSWD) [N/y/?] n
host mtd, jffs2 and ubi/ubifs tools (BR2_PACKAGE_HOST_MTD) [N/y/?] n
host mtools (BR2_PACKAGE_HOST_MTOOLS) [N/y/?] n
host nodejs (BR2_PACKAGE_HOST_NODEJS) [N/y/?] (NEW)
host-odb (BR2_PACKAGE_HOST_ODB) [N/y/?] n
host openocd (BR2_PACKAGE_HOST_OPENOCD) [N/y/?] n
host opkg-utils (BR2_PACKAGE_HOST_OPKG_UTILS) [N/y/?] n
host pahole (BR2_PACKAGE_HOST_PAHOLE) [N/y/?] n
host parted (BR2_PACKAGE_HOST_PARTED) [N/y/?] n
host pigz (BR2_PACKAGE_HOST_PIGZ) [N/y/?] n
Wed Feb 22 20:41:10 UTC 2023 - building ...
Wed Feb 22 20:41:40 UTC 2023 - building ...
Wed Feb 22 20:42:10 UTC 2023 - building ...
Wed Feb 22 20:42:40 UTC 2023 - building ...
...
But when manually building from the latest I see this:
./build.sh -nfa x64
Checking packages needed for building
* WARNING: Did not find a patch file building filesystem without patches!
Preparing buildroot 2022.02.9 on x64 build:
* Extracting buildroot sources................................Done
* Preparing code..............................................mkdir -p /home/sebastian/fos/fssourcex64/output/build/buildroot-config/lxdialog
PKG_CONFIG_PATH="" make CC="/usr/bin/gcc" HOSTCC="/usr/bin/gcc" \
obj=/home/sebastian/fos/fssourcex64/output/build/buildroot-config -C support/kconfig -f Makefile.br conf
make[1]: Entering directory '/home/sebastian/fos/fssourcex64/support/kconfig'
/usr/bin/gcc -DCURSES_LOC="<ncurses.h>" -DLOCALE -I/home/sebastian/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -MM *.c > /home/sebastian/fos/fssourcex64/output/build/buildroot-config/.depend 2>/dev/null || :
/usr/bin/gcc -DCURSES_LOC="<ncurses.h>" -DLOCALE -I/home/sebastian/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -c conf.c -o /home/sebastian/fos/fssourcex64/output/build/buildroot-config/conf.o
/usr/bin/gcc -DCURSES_LOC="<ncurses.h>" -DLOCALE -I/home/sebastian/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" -I. -c /home/sebastian/fos/fssourcex64/output/build/buildroot-config/zconf.tab.c -o /home/sebastian/fos/fssourcex64/output/build/buildroot-config/zconf.tab.o
/usr/bin/gcc -DCURSES_LOC="<ncurses.h>" -DLOCALE -I/home/sebastian/fos/fssourcex64/output/build/buildroot-config -DCONFIG_=\"\" /home/sebastian/fos/fssourcex64/output/build/buildroot-config/conf.o /home/sebastian/fos/fssourcex64/output/build/buildroot-config/zconf.tab.o -o /home/sebastian/fos/fssourcex64/output/build/buildroot-config/conf
rm /home/sebastian/fos/fssourcex64/output/build/buildroot-config/zconf.tab.c
make[1]: Leaving directory '/home/sebastian/fos/fssourcex64/support/kconfig'
#
# configuration written to /home/sebastian/fos/fssourcex64/.config
#
Done
Wed Feb 22 15:19:23 CST 2023 - building ...
Wed Feb 22 15:19:53 CST 2023 - building ...
Wed Feb 22 15:20:23 CST 2023 - building ...
Wed Feb 22 15:20:53 CST 2023 - building ...
...
Looks like it's unable to use our existing configs....
Same with the kernels:
./build.sh -nka x64
Checking packages needed for building
Preparing kernel 5.15.93 on x64 build:
* Downloading kernel source...................................Done
* Extracting kernel source....................................Done
* Preparing kernel source.....................................Done
* WARNING: Did not find a patch file building vanilla kernel without patches!
* Cloning Linux firmware repository...........................Done
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*
* Restart config...
*
*
* Mitigations for speculative execution vulnerabilities
*
...
@Sebastian-Roth That's odd. When I run it manually I get the exact same output as GitHub Actions. I just thought that was the normal process. How are you running it manually?
@rluzuriaga said:
How are you running it manually?
Exactly as seen above: ./build.sh -nfa x64
- this is on Debian Linux
@Sebastian-Roth Oh maybe it's an Ubuntu thing. I'll look into it. From my testing, I've been building on Ubuntu without any issues and the kernels and inits all work fine.
I am just worried about the "Restarting config" message and I guess the "Error in reading or end of file" is part of that as well. I am not exactly sure but I guess this happens when you run make oldconfig
but the .config
file is missing or corrupt. But on the other hand you'd think that a default kernel config would never work to boot up a FOS machine...
@Sebastian-Roth So I created a brand-new Debian 11 VM to test the difference in building between it and Ubuntu. I get the exact same thing on Debian and Ubuntu. I'm wondering if you have any special setup on your machine.
Don't mind the characters at the end, that was an accident.
@Sebastian-Roth Okay I did some more testing and the same appeared on Debian 11 and 10. I tried Debian 9 and then I get the same output as your local machine. Is the machine running Debian 9?
@rluzuriaga Right, got me there! Using an old Debian 9 system for building. I will test on an up to date system soon. I think we can just leave this topic open for the time being. I will get back to it when I have more time.
Thanks heaps for your testing!
@Sebastian-Roth I think I know why Debian 9 doesn't give that "issue". When building the kernels and inits on Debian 9 or Ubuntu 18.04 and bellow, it doesn't ask for the following packages. On newer versions of Debian and Ubuntu we would need to add these 4 lines to the filesystem config files.
# BR2_PACKAGE_MPD is not set
# BR2_PACKAGE_NODEJS is not set
# BR2_PACKAGE_HOST_NODEJS is not set
# BR2_PACKAGE_HOST_QEMU is not set
MPD (Music Player Daemon) shouldn't be needed neither should nodeJS or QEMU.
These are the lines that need to be added to the kernel configs.
# CONFIG_SLS is not set
# CONFIG_ZERO_CALL_USED_REGS is not set
# CONFIG_KCSAN is not set
# CONFIG_KASAN is not set
CONFIG_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR_STRONG=y
# CONFIG_ZERO_CALL_USED_REGS is not set
CONFIG_ARM64_PTR_AUTH_KERNEL=y
CONFIG_ARM64_TLB_RANGE=y
CONFIG_ARM64_MTE=y
# CONFIG_ZERO_CALL_USED_REGS is not set
# CONFIG_KASAN is not set
How did I get these configs? It's just the default that the builder gives.
@rluzuriaga I just updated the kernel and buildroot configs to build on Debian 10 without those messages (9632dcc). We should definitely be aware of those changes (mentioned by you as well):
Stack Protector buffer overflow detection (STACKPROTECTOR) [Y/n/?] (NEW)
Strong Stack Protector (STACKPROTECTOR_STRONG) [Y/n/?] (NEW)
I think it's ok to enable those but we need to keep that in mind in case people complain about issues with the 32 bit kernel.
@rluzuriaga To be sure we don't miss this kind of config changes we need to make sure we are using the same OS and version when preparing the configs as the Github Actions. I am wondering if it's wise to set runs-on
(refernce) to a specific version so we are in control instead of GH Actions switching to a newer Ubuntu version without us noticing.
Currently ubuntu-22.04
makes sense I think (full list of options).
@Sebastian-Roth I agree, I'll change it to 22.04. I think that as soon as a new LTS version comes out, we should try building on it so that we don't fall behind.
Since I'll be making this change, I could make the changes so that the Kernel and Buildroot versions are automatically retrieved from build.sh
and we don't have to enter it manually. Also, I can make the release name the same as how they show up now: Latest from MM.DD.YYY
and the tag as YYYYMMDD
without asking for that input.
@Sebastian-Roth I also saw this commit that you made to release.sh
. I can make another workflow that does that too, or I can modify the current workflow to have a checkbox to make an official release build.
What do you think?
@rluzuriaga said:
Since I'll be making this change, I could make the changes so that the Kernel and Buildroot versions are automatically retrieved from build.sh and we don't have to enter it manually. Also, I can make the release name the same as how they show up now: Latest from MM.DD.YYY and the tag as YYYYMMDD without asking for that input.
Yeah, that'd be great. I can't see why we would need manually enter that information. If it needs adjustment at some point (e.g. we want other tag names), then we can simply change the YAML again. Great!
I also saw this commit that you made to release.sh. I can make another workflow that does that too, or I can modify the current workflow to have a checkbox to make an official release build. What do you think?
Sounds great as well. I think adding a checkbox (default unchecked) and an extra field named "Release Version" would work perfectly fine.
@rluzuriaga As well you can remove the "Pre-release" checkbox. While it can be useful I think it can confuse people as well now that we add the "Official release" checkbox. On the Github fos releases page we can manually edit a release and change it to pre-release if wanted. The only case this is playing a role is, that pre-releases are hidden from the FOG web UI kernel update view.
@Sebastian-Roth Sounds good to me. Maybe it is a good idea to change the release name to full date. Instead of MM.DD.YYYY
it could be MONTH DD, YYYY
?
For the official release checkbox, I would have to add a text field for the release name/FOG version, since there is nowhere in the code to get that info.
If changing the format, should it be YYYY-MM-DD? This way we can see at a glance what is newer
On Mon, Mar 13, 2023, 6:22 p.m. Rodrigo Luzuriaga @.***> wrote:
@Sebastian-Roth https://github.com/Sebastian-Roth Sounds good to me. Maybe it is a good idea to change the release name to full date. Instead of MM.DD.YYYY it could be MONTH DD, YYYY?
— Reply to this email directly, view it on GitHub https://github.com/FOGProject/fos/issues/65#issuecomment-1467201778, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTQQFUG7B4VMMFO4E7SAVLW37B57ANCNFSM6AAAAAAU4HPJ5A . You are receiving this because you are subscribed to this thread.Message ID: @.***>
@rluzuriaga said:
Maybe it is a good idea to change the release name to full date. Instead of MM.DD.YYYY it could be MONTH DD, YYYY?
I side with the format @lukebarone suggested, YYYY-MM-DD
. This makes it easy to recognize as a date but is still sortable. While we don't need those sorted yet I am sure there will be a situation in the future where this is going to be handy.
For the official release checkbox, I would have to add a text field for the release name/FOG version, since there is nowhere in the code to get that info.
Yes, would need an extra text field.
I think the best way to incorporate GitHub Actions is by having a manual workflow. The manual workflow would have a few questions to fill out the release information. Have it look something like this:
The workflow would build all kernels and buildinits, then upload them to the release.