invalid options persist in fragments

[vfazio]

Currently, OpenMP support is flagged in configs/optionals/*.config

This fragment gets applied to all configurations and is persisted into the release fragment, however, this option is not supported for microblaze and arc targets https://gitlab.com/buildroot.org/buildroot/-/blob/master/package/gcc/Config.in.host#L136

There probably needs to be an additional step to make sure that optionals persist after a make savedefconfig, or if they're really optionals, we need to run make savedefconfig and copy out the result so that we do not incorrectly flag support that buildroot's gen-bootlin-toolchains may accidentally apply.

diff --git a/update_gitlab-ci.sh b/update_gitlab-ci.sh
index 26214b3..288315d 100755
--- a/update_gitlab-ci.sh
+++ b/update_gitlab-ci.sh
@@ -127,6 +127,9 @@ function gen_fragment {
             if [[ $name = $optional_m ]]; then
                optionals="${optionals} ${optional}"
                 cat "${base_dir}/configs/optionals/$optional" >> ${config_file}
+                cp ${config_file} ${br_path}/.config
+                make -C ${br_path} savedefconfig 2>&1 1>/dev/null
+                cp ${br_path}/defconfig ${config_file}
             fi
         done
         mv ${config_file} ${frag_dir}/${release_name}.config

[tpetazzoni]

I'm not sure I see this happening. In preparation for a 2023.08 toolchain release, I did a test build of all toolchains, and the fragment generated for the arcle-hs38 toolchain looks like this:

BR2_arcle=y
BR2_archs38=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://toolchains.bootlin.com/downloads/test-2023.08/toolchains/arcle-hs38/tarballs/arcle-hs38--uclibc--bleeding-edge-2023.08-test1.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_13=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_TOOLCHAIN_EXTERNAL_FORTRAN=y
# BR2_TOOLCHAIN_EXTERNAL_DLANG is not set
# BR2_TOOLCHAIN_EXTERNAL_OPENMP is not set
# BR2_TOOLCHAIN_EXTERNAL_HAS_SSP is not set
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y
# BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC=y

Which correctly as OpenMP flagged as not available.

Could you provide more details? Or maybe I misunderstood the issue?

[vfazio]

I'm not sure I see this happening. In preparation for a 2023.08 toolchain release, I did a test build of all toolchains, and the fragment generated for the arcle-hs38 toolchain looks like this:

BR2_arcle=y
BR2_archs38=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://toolchains.bootlin.com/downloads/test-2023.08/toolchains/arcle-hs38/tarballs/arcle-hs38--uclibc--bleeding-edge-2023.08-test1.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_13=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_TOOLCHAIN_EXTERNAL_FORTRAN=y
# BR2_TOOLCHAIN_EXTERNAL_DLANG is not set
# BR2_TOOLCHAIN_EXTERNAL_OPENMP is not set
# BR2_TOOLCHAIN_EXTERNAL_HAS_SSP is not set
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y
# BR2_TOOLCHAIN_EXTERNAL_INET_RPC is not set
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC=y

Which correctly as OpenMP flagged as not available.

Could you provide more details? Or maybe I misunderstood the issue?

@tpetazzoni Sure, here's what i see on master:

vfazio@Zephyrus:~/development/toolchains-builder$ ./update_gitlab-ci.sh -d -b dd98584f6c304eca7e7742487a68081aa11aaa54 -v stable -l glibc -a microblazeel -n testing_mbl
~/development/toolchains-builder/buildroot ~/development/toolchains-builder
HEAD is now at dd98584f6c package/linux-headers: backport 5.0 patch to 4.19 to fix AArch64 musl build
~/development/toolchains-builder
|                 arch |    libc |        variant |                         extras |                                          optionals | status

error: branch 'builds-microblazeel-2023-08-05--06-31-15' not found.
Switched to a new branch 'builds-microblazeel-2023-08-05--06-31-15'
|         microblazeel |   glibc |         stable |                                |                                           *.config | OK
microblazeel--glibc--stable.config
[builds-microblazeel-2023-08-05--06-31-15 f4de1df] Build bot: trigger new builds
 2 files changed, 20 insertions(+)
 create mode 100644 frags/microblazeel--glibc--stable.config
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
vfazio@Zephyrus:~/development/toolchains-builder$ git show builds-microblazeel-2023-08-05--06-31-15  -- frags/
commit f4de1df155b7fb5a7613ff09a986a2be784ff598 (builds-microblazeel-2023-08-05--06-31-15)
Author: Vincent Fazio <vfazio@gmail.com>
Date:   Sat Aug 5 06:31:16 2023 -0500

    Build bot: trigger new builds

diff --git a/frags/microblazeel--glibc--stable.config b/frags/microblazeel--glibc--stable.config
new file mode 100644
index 0000000..eeb8fa2
--- /dev/null
+++ b/frags/microblazeel--glibc--stable.config
@@ -0,0 +1,15 @@
+BR2_microblazeel=y
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
+BR2_PACKAGE_GLIBC_KERNEL_COMPAT=y
+BR2_KERNEL_HEADERS_4_14=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+BR2_TOOLCHAIN_BUILDROOT_FORTRAN=y
+BR2_INIT_NONE=y
+BR2_SYSTEM_BIN_SH_NONE=y
+# BR2_PACKAGE_BUSYBOX is not set
+# BR2_TARGET_ROOTFS_TAR is not set
+BR2_PACKAGE_HOST_GDB=y
+BR2_PACKAGE_HOST_GDB_TUI=y
+BR2_PACKAGE_HOST_GDB_PYTHON3=y
+BR2_PACKAGE_GDB=y
+BR2_GCC_ENABLE_OPENMP=y

If i add a similar snippet to check for rejects like in check_config:

vfazio@Zephyrus:~/development/toolchains-builder$ ./update_gitlab-ci.sh -d -b dd98584f6c304eca7e7742487a68081aa11aaa54 -v stable -l glibc -a microblazeel -n testing_mbl
~/development/toolchains-builder/buildroot ~/development/toolchains-builder
HEAD is now at dd98584f6c package/linux-headers: backport 5.0 patch to 4.19 to fix AArch64 musl build
~/development/toolchains-builder
|                 arch |    libc |        variant |                         extras |                                          optionals | status

error: branch 'builds-microblazeel-2023-08-05--06-37-12' not found.
Switched to a new branch 'builds-microblazeel-2023-08-05--06-37-12'
|         microblazeel |   glibc |         stable |                                |
REJECTS FOUND
BR2_GCC_ENABLE_OPENMP=y
BR2_PACKAGE_GDB=y
BR2_PACKAGE_HOST_GDB=y
BR2_PACKAGE_HOST_GDB_PYTHON3=y
BR2_PACKAGE_HOST_GDB_TUI=y
                                           *.config | OK
microblazeel--glibc--stable.config  microblazeel-glibc-stable
[builds-microblazeel-2023-08-05--06-37-12 9b803a0] Build bot: trigger new builds
 3 files changed, 35 insertions(+)
 create mode 100644 frags/microblazeel--glibc--stable.config
 create mode 100644 frags/microblazeel-glibc-stable
M       update_gitlab-ci.sh
Switched to branch 'builds-microblazeel-2023-08-05--06-34-11'

We see OPENMP being saved in the fragment but it should not be. Similarly, GDB is not supported per:

config BR2_PACKAGE_GDB_ARCH_SUPPORTS
    bool
    default y
    depends on !((BR2_arm || BR2_armeb) && BR2_BINFMT_FLAT)
    depends on !BR2_microblaze

[vfazio]

savedefconfig is probably not what you want, obviously, but something like it, to prune out rejected symbols

[tpetazzoni]

I see, but that does not cause any problem. Those unsupported features get dropped during the build. The final "build fragment" that is saved for example at https://toolchains.bootlin.com/downloads/releases/toolchains/microblazeel/build_fragments/microblazeel--glibc--bleeding-edge-2023.08-1.defconfig doesn't have those unsupported options.

[vfazio]

Ok, so long as there's no functional issue in the actual released fragments, then i'll close this issue. i just noticed it while reviewing the code for another issue.

sorry for the noise