search

linuxboot / heads

A minimal Linux that runs as a coreboot or LinuxBoot ROM payload to provide a secure, flexible boot environment for laptops, workstations and servers.
https://osresearch.net/
GNU General Public License v2.0
1.42k stars 187 forks source link

Build Error on building heads: heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; x230-flash/util/sconfig/sconfig.tab.o #853

Closed explit7 closed 6 months ago

explit7 commented 4 years ago

Hello colegues. Have some problems to build heads on Debian 10 Maybe this is known? Thanks

The sections containing CBFSes are: COREBOOT
    CC         bootblock/lib/cbfs.o
    CC         bootblock/lib/cbmem_console.o
    CC         bootblock/lib/delay.o
    CC         bootblock/lib/fmap.o
    CC         bootblock/lib/gcc.o
    CC         bootblock/lib/halt.o
    CC         bootblock/lib/hexdump.o
    CC         bootblock/lib/libgcc.o
    CC         bootblock/lib/memchr.o
    CC         bootblock/lib/memcmp.o
    CC         bootblock/lib/prog_loaders.o
    CC         bootblock/lib/prog_ops.o
    CC         bootblock/lib/reset.o
    CC         bootblock/lib/version.o
    HOSTCC     util/sconfig/sconfig (link)
/usr/bin/ld: x230-flash/util/sconfig/main.o:/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; x230-flash/util/sconfig/sconfig.tab.o:/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [util/sconfig/Makefile.inc:43: x230-flash/util/sconfig/sconfig] Error 1
make[2]: Leaving directory '/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1'
make[1]: *** [Makefile:438: /media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/x230-flash/.build] Error 1
make[1]: Leaving directory '/media/sdb1/Git-Projects/heads'
make: *** [Makefile:643: all] Error 2
Thrilleratplay commented 4 years ago

I know I have seen that error before but can't remember the exact reason. You may be missing a dependency, try this and see if anything needs to be installed. If so, install it and try building again.

    apt-get install \
      bc \
      bison \
      build-essential \
      cpio \
      flex \
      libelf-dev \
      libncurses5-dev \
      pkg-config \
      texinfo \
      zlib1g-dev
tlaurion commented 4 years ago

Quick reminder that for a Heads commit ID, dependencies for Debian 10 are currently covered under CircleCI under https://github.com/osresearch/heads/blob/master/.circleci/config.yml#L8-L11 as referred under heads-wiki https://github.com/osresearch/heads-wiki/blob/master/Building.md

If for some reason, there is corrupted artifacts from past builds (some patches were added for the same module dowloaded version after decompression of module archive), patches are only applied once per module version change.

The documentation might need to be clearer on how to create a cleaner build: make BOARD=x230-flash real.clean

While you may want to add: rm -rf ./build/* and redo make BOARD=x230-flash

And adding this to the precedent commands will rebuild also musl-cross-make: rm -rf ./crossgcc make BOARD=x230-flash

explit7 commented 4 years ago

I know I have seen that error before but can't remember the exact reason. You may be missing a dependency, try this and see if anything needs to be installed. If so, install it and try building again.

    apt-get install \
      bc \
      bison \
      build-essential \
      cpio \
      flex \
      libelf-dev \
      libncurses5-dev \
      pkg-config \
      texinfo \
      zlib1g-dev

Thanks for the quick answer! All dependencies are installed...

bc is already the newest version (1.07.1-2+b2).
bison is already the newest version (2:3.7.2+dfsg-1).
build-essential is already the newest version (12.8).
cpio is already the newest version (2.13+dfsg-4).
libelf-dev is already the newest version (0.181-1).
flex is already the newest version (2.6.4-8).
libncurses5-dev is already the newest version (6.2+20200918-1).
pkg-config is already the newest version (0.29.2-1).
texinfo is already the newest version (6.7.0.dfsg.2-5).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2).

Will try to do

 make BOARD=x230-flash real.clean
 rm -rf ./build/*
 rm -rf ./crossgcc
 make BOARD=x230-flash
Thrilleratplay commented 4 years ago

@explit7 Before you do, try installing all of the packages @tlaurion mentioned in the circleci/config.yml. cmake and autoconf may be needed. Try this before removing everything in build and crossgcc, it will save a lot of time.

explit7 commented 4 years ago

@explit7 Before you do, try installing all of the packages @tlaurion mentioned in the circleci/config.yml. cmake and autoconf may be needed. Try this before removing everything in build and crossgcc, it will save a lot of time.

Thanks, already installed, but in debian 10 there are some problems with the package python. as i understand, package python is deprecated, so i installed python2 instead and made symlink to python.

sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20200528-1).
autoconf is already the newest version (2.69-11.1).
bc is already the newest version (1.07.1-2+b2).
bison is already the newest version (2:3.7.2+dfsg-1).
build-essential is already the newest version (12.8).
bzip2 is already the newest version (1.0.8-4).
ccache is already the newest version (3.7.12-1).
cmake is already the newest version (3.18.2-1).
cpio is already the newest version (2.13+dfsg-4).
libelf-dev is already the newest version (0.181-1).
flex is already the newest version (2.6.4-8).
git is already the newest version (1:2.28.0-1).
gnat is already the newest version (9).
gnupg is already the newest version (2.2.20-1).
libusb-1.0-0-dev is already the newest version (2:1.0.23-2).
m4 is already the newest version (1.4.18-4).
nasm is already the newest version (2.15.05-1).
libncurses-dev is already the newest version (6.2+20200918-1).
patch is already the newest version (2.7.6-6).
patch set to manually installed.
pkg-config is already the newest version (0.29.2-1).
texinfo is already the newest version (6.7.0.dfsg.2-5).
uuid-dev is already the newest version (2.36-3+b1).
wget is already the newest version (1.20.3-1+b3).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python : PreDepends: python-minimal (= 2.7.17-2) but it is not going to be installed
          Depends: libpython-stdlib (= 2.7.17-2) but it is not going to be installed
          Depends: python2 (= 2.7.17-2) but 2.7.18-2 is to be installed
E: Unable to correct problems, you have held broken packages.
explit7 commented 4 years ago

unfortunately the same result after 

make BOARD=x230-flash real.clean
 rm -rf ./build/*
 rm -rf ./crossgcc
 make BOARD=x230-flash
tlaurion commented 4 years ago

@explit7 this is weird, since CircleCI builds are based on debian-10. Will check again on local QubesOS debian-10 based AppVM on clean build later on

tlaurion commented 4 years ago

@explit7 @Thrilleratplay Will do with build trace

user@x230-master:~/heads$ git fetch osresearch 
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (7/7), done.
Unpacking objects: 100% (18/18), done.
remote: Total 18 (delta 7), reused 10 (delta 6), pack-reused 5
From https://github.com/osresearch/heads
   89c4577..5614525  master     -> osresearch/master
user@x230-master:~/heads$ git checkout osresearch/master
Previous HEAD position was 89c4577 coreboot configs: remove CONFIG_LOCALVERSION artifacts in conformity of  https://github.com/osresearch/heads/commit/bd7a945bbbf586ceb7c83785ef36d95f0adf7a15
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ git reset --hard
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ grep apt .circleci/config.yml
            apt update
            apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
user@x230-master:~/heads$ sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20181213-1).
autoconf is already the newest version (2.69-11).
automake is already the newest version (1:1.16.1-4).
bc is already the newest version (1.07.1-2+b1).
bison is already the newest version (2:3.3.2.dfsg-1).
build-essential is already the newest version (12.6).
bzip2 is already the newest version (1.0.6-9.2~deb10u1).
ccache is already the newest version (3.6-1).
cmake is already the newest version (3.13.4-1).
cpio is already the newest version (2.12+dfsg-9).
doxygen is already the newest version (1.8.13-10).
libelf-dev is already the newest version (0.176-1.1).
flex is already the newest version (2.6.4-6.2).
git is already the newest version (1:2.20.1-2+deb10u3).
gnat is already the newest version (8.0.1).
gnupg is already the newest version (2.2.12-1+deb10u1).
graphviz is already the newest version (2.40.1-6).
libdigest-sha-perl is already the newest version (6.02-1+b1).
libtool is already the newest version (2.4.6-9).
libusb-1.0-0-dev is already the newest version (2:1.0.22-2).
m4 is already the newest version (1.4.18-2).
nasm is already the newest version (2.14-1).
libncurses-dev is already the newest version (6.1+20181013-2+deb10u2).
patch is already the newest version (2.7.6-3+deb10u1).
pkg-config is already the newest version (0.29-6).
python is already the newest version (2.7.16-1).
rsync is already the newest version (3.1.3-6).
libudev-dev is already the newest version (241-7~deb10u4).
libudev1 is already the newest version (241-7~deb10u4).
udev is already the newest version (241-7~deb10u4).
texinfo is already the newest version (6.5.0.dfsg.1-4+b1).
uuid-dev is already the newest version (2.33.1-0.1).
wget is already the newest version (1.20.1-1.1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
The following packages were automatically installed and are no longer required:
  gconf2 libqt5sensors5 libqt5webkit5 lightning linux-headers-4.19.0-10-amd64 linux-headers-4.19.0-10-common linux-headers-4.19.0-5-amd64
  linux-headers-4.19.0-5-common linux-headers-4.19.0-6-amd64 linux-headers-4.19.0-6-common linux-headers-4.19.0-8-amd64
  linux-headers-4.19.0-8-common linux-headers-4.19.0-9-amd64 linux-headers-4.19.0-9-common linux-image-4.19.0-10-amd64 linux-image-4.19.0-5-amd64
  linux-image-4.19.0-6-amd64 linux-image-4.19.0-8-amd64 linux-image-4.19.0-9-amd64 mpv phantomjs python-dbus python-qubesdb python-xdg
  python3-pyxattr riot-desktop rtmpdump
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 real.clean
2020-11-13 13:05:19-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
for dir in \
    x230 cryptsetup-1.7.3 busybox-1.28.0 coreboot-4.8.1/x230 coreboot-4.8.1/3rdparty/blobs libksba-1.4.0 kexec-tools-2.0.20 pinentry-1.1.0 lvm2.2.02.168 popt-1.16 hotp-verification-c0956cfa085bcfc2500c1085dad350a440ccbe40/hidapi cairo-1.14.12 mbedtls-2.4.2 npth-1.6 libgcrypt-1.8.6 qrencode-3.4.4 pixman-0.34.0 fbwhiptail-0f14a409735b71c219e0b9b3ee63cdae709ba143 pciutils-3.5.4 libassuan-2.5.3 libusb-1.0.21 linux-4.14.62/linux-x230 gnupg-2.2.21 flashtools-40d5170e84a3822552df7a500cd00aa870fdfe76 libpng-1.6.34 tpmtotp-18b860fdcf5a55537c8395b891f2b2a5c24fc00a libusb-compat-0.1.5 libgpg-error-1.37 flashrom-b1f858f65b2abd276542650d8cb9e382da258967 util-linux-2.29.2 zlib-1.2.11 \
     \
; do \
    if [ ! -z "$dir" ]; then \
        rm -rf "build/$dir"; \
    fi; \
done
cd install && rm -rf -- *
user@x230-master:~/heads$ rm -rf crossgcc/ build/*
user@x230-master:~/heads$ tar zxvf archive.tar.gz ./
./packages/libgcrypt-1.8.3.tar.bz2
./packages/libassuan-2.5.1.tar.bz2
./packages/libassuan-2.5.3.tar.bz2
./packages/cryptsetup-1.7.5.tar.xz
./packages/cryptsetup-1.7.3.tar.xz
./packages/musl-1.1.15.tar.gz
./packages/linux-5.4.30.tar.xz
./packages/cairo-1.14.12.tar.xz
./packages/LVM2.2.02.168.tgz
./packages/coreboot-4.12.tar.xz
./packages/gnupg-2.2.20.tar.bz2
./packages/dropbear-2016.74.tar.bz2
./packages/busybox-1.28.0.tar.bz2
./packages/libpng-1.6.34.tar.gz
./packages/libusb-1.0.21.tar.bz2
./packages/gnupg-2.2.21.tar.bz2
./packages/linux-4.19.114.tar.xz
./packages/popt-1.16.tar.gz
./packages/libksba-1.3.5.tar.bz2
./packages/purism-blobs-f53d4074a81c70352d39839884caac20181274d1.tar.gz
./packages/linux-b87b58e1b057a2706d422fbdc76aa34309c6c90b.tar.xz
./packages/coreboot-4.8.1.tar.xz
./packages/mbedtls-2.4.2-gpl.tgz
./packages/libgcrypt-1.8.4.tar.bz2
./packages/coreboot-blobs-4.12.tar.xz
./packages/linux-4.19.63.tar.xz
./packages/libgpg-error-1.39.tar.bz2
./packages/libksba-1.4.0.tar.bz2
./packages/libgcrypt-1.8.6.tar.bz2
./packages/libgcrypt-1.8.5.tar.bz2
./packages/slang-2.3.1a.tar.bz2
./packages/zlib-1.2.11.tar.gz
./packages/libgpg-error-1.32.tar.bz2
./packages/libgpg-error-1.37.tar.bz2
./packages/pixman-0.34.0.tar.gz
./packages/linux-4.14.62.tar.xz
./packages/gnupg-2.2.10.tar.bz2
./packages/coreboot-blobs-4.8.1.tar.xz
./packages/util-linux-2.29.2.tar.xz
./packages/cryptsetup-2.3.3.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/acpica-unix2-20200110.tar.gz
./build/coreboot-4.12/util/crossgcc/tarballs/gmp-6.1.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/binutils-2.33.1.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/gcc-8.3.0.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpfr-4.0.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpc-1.1.0.tar.gz
./build/coreboot-4.8.1/util/crossgcc/tarballs/acpica-unix2-20180531.tar.gz
user@x230-master:~/heads$ 
user@x230-master:~/heads$ 
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 
2020-11-13 13:07:06-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
tar xf "/home/user/heads/packages/gawk-4.2.1.tar.xz" -C "/home/user/heads/build"
touch "/home/user/heads/build/gawk-4.2.1/.extract"
touch "/home/user/heads/build/gawk-4.2.1/.patch"
cd "/home/user/heads/build/gawk-4.2.1/" ; \
./configure 2>&1 \
| tee "/home/user/heads/build/log/gawk.configure.log" \
> /dev/null
touch "/home/user/heads/build/gawk-4.2.1/.configured"
make -C "/home/user/heads/build/gawk-4.2.1/"  \
    2>&1 \
    | tee "/home/user/heads/build/log/gawk.log" \
    > /dev/null

[...]

tlaurion commented 4 years ago

@Thrilleratplay For cache building clarifications on CircleCI:

Then

@Thrilleratplay So unless musl-cross-make git commit changes happened in its module (that module text has changed), that cache is reused. In both caches.

I have resetted the cache and also rebuilding from CI, while changing the environment variable in CI which is used here and there which is inserted in build through the Environement Variables section of CI for future builds, basically pushing a new .Environment.CACHE_VERSION that will be picked up on next build.

But here, at each builds the docker debian:10 container is used, build dependencies and updates are installed, git is cloned and positioned at latest commit, measurements are taken on modules, and the largest cache matching actual modules+patches measurements is reused if key matches on modules measurements.

If i'm doing something wrong here please let me know, otherwise, we are not recreating a docker image here, we are trying to simulate a debian-10 compilation but not for a signel board, but for all boards, reusing the caches to save time.

tlaurion commented 4 years ago

@Thrilleratplay @explit7 clean build following precedent information is happening here

You can see:

explit7 commented 4 years ago

thanks. Will also try to fire up ubuntu vm to build it.... if nothing helps

tlaurion commented 4 years ago

My question is more: what are the differences between docker 10, QubesOS debian-10 Template and native debian-10 installation to cause the issue.

As specified under #571, I think its time to move out of trying to build on all OSes of the world, and go docker, where that docker should probably be guix-buildstack as @daym is currently working on, so that end user instructions are clear, implementable in CIs, limiting impacts of host differences and permitting to replicate the results of today's commit in 5 years from now, pulling the same buildstack revision and permitting to fixate toolstack revisions desired for different coreboot versions, for example.

Thrilleratplay commented 4 years ago

@tlaurion You are not doing anything wrong, the caching is misleading. apt-update updates the repository information. I suspect, and correct me if I am wrong, CircleCI will not pull the original Debian:10 container again and rerun apt-update and apt-install dependencies but instead pull from its cache the steps taken under Install Dependencies as this has not been changed in the configuration script.

This means the package versions used in CI are not necessarily the same as those users try to build Heads locally. A package may be updated twice in a month where once version creates an error and a second one does not. The result being intermittent bugs in the build process. A preconfigured docker image with all of the necessary packages installed hosted on a registry, similar to the coreboot-sdk image, so that CI and users could use the same base would solve these issues. Depending on the direction #571 it would likely resolve itself.

If this explanation is not clear, please let me know.

tlaurion commented 4 years ago

@tlaurion You are not doing anything wrong, the caching is misleading. apt-update updates the repository information. I suspect, and correct me if I am wrong, CircleCI will not pull the original Debian:10 container again and rerun apt-update and apt-install dependencies but instead pull from its cache the steps taken under Install Dependencies as this has not been changed in the configuration script.

@Thrilleratplay : No! It actually pulls the docker image, calls apt update and then install dependencies at each build. (This is not cached!!!) Please click the links here so that we are all at the same page.

This means the package versions used in CI are not necessarily the same as those users try to build Heads locally.

Well. Users should also update their packages and install the dependencies the same way the CI does at each build. I mean. OS is updated, right? On debian-10, this is the only way we have to detect Host releated problems, and why we want to get away of those dependencies.

A package may be updated twice in a month where once version creates an error and a second one does not. The result being intermittent bugs in the build process. A preconfigured docker image with all of the necessary packages installed hosted on a registry, similar to the coreboot-sdk image, so that CI and users could use the same base would solve these issues. Depending on the direction #571 it would likely resolve itself.

I do not find the references here but coreboot-sdk is not reproducible right now. (searching source and discussions)

If this explanation is not clear, please let me know.

Thrilleratplay commented 4 years ago

@tlaurion I looked back through other CircleCI builds and it is updating and installing the packages each build. My apologies. I assumed it would cache this if subsequent layers were also pulling from cache.

Yes, users should be able to update their OS but if the goal is reproducible build, changes in package versions would likely result in change to the checksum of the compiled rom.

It is not that coreboot-sdk would be reproducible if rebuilt, it is the fact that the same build is shared. Anyone can do a docker pull coreboot/coreboot-sdk:6065f616eb and have the exact same build environment. This would be the same environment today or 5 years from now. The same way docker pull coreboot/coreboot-sdk:1.35 creates the same build environment it did 5 years ago.

tlaurion commented 4 years ago

@tlaurion I looked back through other CircleCI builds and it is updating and installing the packages each build. My apologies. I assumed it would cache this if subsequent layers were also pulling from cache.

@Thrilleratplay : Glad we finally have a trace of the actual problem we are trying to resolve. :)

Yes, users should be able to update their OS but if the goal is reproducible build, changes in package versions would likely result in change to the checksum of the compiled rom.

And here lies the problem. Theoretically, we would expect host buildstack versions to be sticked to major versions where impact should be limited. Problems lies in having to support really old versions of coreboot now (4.8.1 because #709 and coreboot 4.11 #867) where problems will definitely arise, unless we can switch buildstack prior of building those board in CI or locally per proper instructions against a docker image permitting that, with CI instructions and doc explaining and applying the process.

It is not that coreboot-sdk would be reproducible if rebuilt, it is the fact that the same build is shared. Anyone can do a docker pull coreboot/coreboot-sdk:6065f616eb and have the exact same build environment. This would be the same environment today or 5 years from now. The same way docker pull coreboot/coreboot-sdk:1.35 creates the same build environment it did 5 years ago.

So maybe solution 1 here is the way to go. While that docker being too recent won't build coreboot 4.8.1 and too old won't build 4.12+. Here lies the problem and hopes into solution 2.

Thrilleratplay commented 4 years ago

@tlaurion Solution #1 or Solution #2 would solve this. As long as an image of the exact same build environment can be used by both the CI and end users, this should provide the ability to create reproducible builds.

tlaurion commented 4 years ago

@Thrilleratplay @explit7 clean build following precedent information is happening here

You can see:

* [Docker image downloaded here from docker hub](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655/parallel-runs/0/steps/0-0)

* [No cache was downloaded nor applied since cache key signature didn't match](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655) since `.Environment.CACHE_VERSION` was changed in CI so no cache signature literals matched.

* Build is [happening](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655) but is expected to take around 3h from now.

@Thrilleratplay @explit7 :

tlaurion commented 4 years ago

@explit7 @Thrilleratplay Will do with build trace

user@x230-master:~/heads$ git fetch osresearch 
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (7/7), done.
Unpacking objects: 100% (18/18), done.
remote: Total 18 (delta 7), reused 10 (delta 6), pack-reused 5
From https://github.com/osresearch/heads
   89c4577..5614525  master     -> osresearch/master
user@x230-master:~/heads$ git checkout osresearch/master
Previous HEAD position was 89c4577 coreboot configs: remove CONFIG_LOCALVERSION artifacts in conformity of  https://github.com/osresearch/heads/commit/bd7a945bbbf586ceb7c83785ef36d95f0adf7a15
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ git reset --hard
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ grep apt .circleci/config.yml
            apt update
            apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
user@x230-master:~/heads$ sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20181213-1).
autoconf is already the newest version (2.69-11).
automake is already the newest version (1:1.16.1-4).
bc is already the newest version (1.07.1-2+b1).
bison is already the newest version (2:3.3.2.dfsg-1).
build-essential is already the newest version (12.6).
bzip2 is already the newest version (1.0.6-9.2~deb10u1).
ccache is already the newest version (3.6-1).
cmake is already the newest version (3.13.4-1).
cpio is already the newest version (2.12+dfsg-9).
doxygen is already the newest version (1.8.13-10).
libelf-dev is already the newest version (0.176-1.1).
flex is already the newest version (2.6.4-6.2).
git is already the newest version (1:2.20.1-2+deb10u3).
gnat is already the newest version (8.0.1).
gnupg is already the newest version (2.2.12-1+deb10u1).
graphviz is already the newest version (2.40.1-6).
libdigest-sha-perl is already the newest version (6.02-1+b1).
libtool is already the newest version (2.4.6-9).
libusb-1.0-0-dev is already the newest version (2:1.0.22-2).
m4 is already the newest version (1.4.18-2).
nasm is already the newest version (2.14-1).
libncurses-dev is already the newest version (6.1+20181013-2+deb10u2).
patch is already the newest version (2.7.6-3+deb10u1).
pkg-config is already the newest version (0.29-6).
python is already the newest version (2.7.16-1).
rsync is already the newest version (3.1.3-6).
libudev-dev is already the newest version (241-7~deb10u4).
libudev1 is already the newest version (241-7~deb10u4).
udev is already the newest version (241-7~deb10u4).
texinfo is already the newest version (6.5.0.dfsg.1-4+b1).
uuid-dev is already the newest version (2.33.1-0.1).
wget is already the newest version (1.20.1-1.1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
The following packages were automatically installed and are no longer required:
  gconf2 libqt5sensors5 libqt5webkit5 lightning linux-headers-4.19.0-10-amd64 linux-headers-4.19.0-10-common linux-headers-4.19.0-5-amd64
  linux-headers-4.19.0-5-common linux-headers-4.19.0-6-amd64 linux-headers-4.19.0-6-common linux-headers-4.19.0-8-amd64
  linux-headers-4.19.0-8-common linux-headers-4.19.0-9-amd64 linux-headers-4.19.0-9-common linux-image-4.19.0-10-amd64 linux-image-4.19.0-5-amd64
  linux-image-4.19.0-6-amd64 linux-image-4.19.0-8-amd64 linux-image-4.19.0-9-amd64 mpv phantomjs python-dbus python-qubesdb python-xdg
  python3-pyxattr riot-desktop rtmpdump
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 real.clean
2020-11-13 13:05:19-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
for dir in \
  x230 cryptsetup-1.7.3 busybox-1.28.0 coreboot-4.8.1/x230 coreboot-4.8.1/3rdparty/blobs libksba-1.4.0 kexec-tools-2.0.20 pinentry-1.1.0 lvm2.2.02.168 popt-1.16 hotp-verification-c0956cfa085bcfc2500c1085dad350a440ccbe40/hidapi cairo-1.14.12 mbedtls-2.4.2 npth-1.6 libgcrypt-1.8.6 qrencode-3.4.4 pixman-0.34.0 fbwhiptail-0f14a409735b71c219e0b9b3ee63cdae709ba143 pciutils-3.5.4 libassuan-2.5.3 libusb-1.0.21 linux-4.14.62/linux-x230 gnupg-2.2.21 flashtools-40d5170e84a3822552df7a500cd00aa870fdfe76 libpng-1.6.34 tpmtotp-18b860fdcf5a55537c8395b891f2b2a5c24fc00a libusb-compat-0.1.5 libgpg-error-1.37 flashrom-b1f858f65b2abd276542650d8cb9e382da258967 util-linux-2.29.2 zlib-1.2.11 \
   \
; do \
  if [ ! -z "$dir" ]; then \
      rm -rf "build/$dir"; \
  fi; \
done
cd install && rm -rf -- *
user@x230-master:~/heads$ rm -rf crossgcc/ build/*
user@x230-master:~/heads$ tar zxvf archive.tar.gz ./
./packages/libgcrypt-1.8.3.tar.bz2
./packages/libassuan-2.5.1.tar.bz2
./packages/libassuan-2.5.3.tar.bz2
./packages/cryptsetup-1.7.5.tar.xz
./packages/cryptsetup-1.7.3.tar.xz
./packages/musl-1.1.15.tar.gz
./packages/linux-5.4.30.tar.xz
./packages/cairo-1.14.12.tar.xz
./packages/LVM2.2.02.168.tgz
./packages/coreboot-4.12.tar.xz
./packages/gnupg-2.2.20.tar.bz2
./packages/dropbear-2016.74.tar.bz2
./packages/busybox-1.28.0.tar.bz2
./packages/libpng-1.6.34.tar.gz
./packages/libusb-1.0.21.tar.bz2
./packages/gnupg-2.2.21.tar.bz2
./packages/linux-4.19.114.tar.xz
./packages/popt-1.16.tar.gz
./packages/libksba-1.3.5.tar.bz2
./packages/purism-blobs-f53d4074a81c70352d39839884caac20181274d1.tar.gz
./packages/linux-b87b58e1b057a2706d422fbdc76aa34309c6c90b.tar.xz
./packages/coreboot-4.8.1.tar.xz
./packages/mbedtls-2.4.2-gpl.tgz
./packages/libgcrypt-1.8.4.tar.bz2
./packages/coreboot-blobs-4.12.tar.xz
./packages/linux-4.19.63.tar.xz
./packages/libgpg-error-1.39.tar.bz2
./packages/libksba-1.4.0.tar.bz2
./packages/libgcrypt-1.8.6.tar.bz2
./packages/libgcrypt-1.8.5.tar.bz2
./packages/slang-2.3.1a.tar.bz2
./packages/zlib-1.2.11.tar.gz
./packages/libgpg-error-1.32.tar.bz2
./packages/libgpg-error-1.37.tar.bz2
./packages/pixman-0.34.0.tar.gz
./packages/linux-4.14.62.tar.xz
./packages/gnupg-2.2.10.tar.bz2
./packages/coreboot-blobs-4.8.1.tar.xz
./packages/util-linux-2.29.2.tar.xz
./packages/cryptsetup-2.3.3.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/acpica-unix2-20200110.tar.gz
./build/coreboot-4.12/util/crossgcc/tarballs/gmp-6.1.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/binutils-2.33.1.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/gcc-8.3.0.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpfr-4.0.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpc-1.1.0.tar.gz
./build/coreboot-4.8.1/util/crossgcc/tarballs/acpica-unix2-20180531.tar.gz
user@x230-master:~/heads$ 
user@x230-master:~/heads$ 
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 
2020-11-13 13:07:06-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
tar xf "/home/user/heads/packages/gawk-4.2.1.tar.xz" -C "/home/user/heads/build"
touch "/home/user/heads/build/gawk-4.2.1/.extract"
touch "/home/user/heads/build/gawk-4.2.1/.patch"
cd "/home/user/heads/build/gawk-4.2.1/" ; \
./configure 2>&1 \
| tee "/home/user/heads/build/log/gawk.configure.log" \
> /dev/null
touch "/home/user/heads/build/gawk-4.2.1/.configured"
make -C "/home/user/heads/build/gawk-4.2.1/"  \
  2>&1 \
  | tee "/home/user/heads/build/log/gawk.log" \
  > /dev/null

[...]

@Thrilleratplay @explit7 :

2020-11-14 23:10:27-05:00 MAKE coreboot
2020-11-14 23:11:03-05:00 DONE coreboot
"/home/user/heads/build/coreboot-4.8.1/x230/cbfstool" "/home/user/heads/build/coreboot-4.8.1/x230/coreboot.rom" print
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85188 none
cpu_microcode_blob.bin         0x14dc0    microcode       25600 none
fallback/ramstage              0x1b240    stage           81881 none
config                         0x2f280    raw               589 none
revision                       0x2f540    raw               581 none
cmos.default                   0x2f7c0    cmos_default      256 none
cmos_layout.bin                0x2f900    cmos_layout      1804 none
fallback/dsdt.aml              0x30080    raw             13646 none
fallback/payload               0x33640    simple elf    6850500 none
(empty)                        0x6bbe40   null           144792 none
bootblock                      0x6df400   bootblock        3000 none
2020-11-14 23:11:03-05:00 INSTALL   build/coreboot-4.8.1/x230/coreboot.rom => build/x230/heads-x230-v0.2.0-949-g5614525.rom
2020-11-14 23:11:03-05:00 UNCHANGED build/coreboot-4.8.1/x230/coreboot.rom
93bcf58c0159439ed7e61d71e2809a1619ef073280408657ae2e0ff9dad6ead1  build/x230/heads-x230-v0.2.0-949-g5614525.rom
4b3be2b552ab6b60df7291bfbd819d9b92f88641662472e0ebd8ca7fc27a4c63  /home/user/heads/build/gawk-4.2.1/gawk
tlaurion commented 4 years ago

My question is more: what are the differences between docker 10, QubesOS debian-10 Template and native debian-10 installation to cause the issue.

As specified under #571, I think its time to move out of trying to build on all OSes of the world, and go docker, where that docker should probably be guix-buildstack as @daym is currently working on, so that end user instructions are clear, implementable in CIs, limiting impacts of host differences and permitting to replicate the results of today's commit in 5 years from now, pulling the same buildstack revision and permitting to fixate toolstack revisions desired for different coreboot versions, for example.

Can't replicate :/

tlaurion commented 3 years ago

Interestingly enough, that was replicated today for the first time under CircleCI Ubuntu 24.04 docker image (with a lot of patches against coreboot 4.8.1 to try to keep some boards alive) over #1038

Happened against qemu-coreboot CircleCI build

tail /root/project/build/log/coreboot.log
-----
cp /root/project/build/coreboot-4.8.1/util/sconfig/sconfig.tab.h_shipped qemu-coreboot/util/sconfig/sconfig.tab.h
mkdir -p qemu-coreboot/util/sconfig/
cp /root/project/build/coreboot-4.8.1/util/sconfig/sconfig.tab.c_shipped qemu-coreboot/util/sconfig/sconfig.tab.c
printf "    HOSTCC     util/sconfig/lex.yy.o\n"
    HOSTCC     util/sconfig/lex.yy.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/lex.yy.o qemu-coreboot/util/sconfig/lex.yy.c
printf "    HOSTCC     util/sconfig/sconfig.tab.o\n"
    HOSTCC     util/sconfig/sconfig.tab.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/sconfig.tab.o qemu-coreboot/util/sconfig/sconfig.tab.c
touch qemu-coreboot/util/sconfig/.generated
printf "    HOSTCC     util/sconfig/main.o\n"
    HOSTCC     util/sconfig/main.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/main.o util/sconfig/main.c
printf "    HOSTCC     util/sconfig/sconfig (link)\n"
    HOSTCC     util/sconfig/sconfig (link)
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -o qemu-coreboot/util/sconfig/sconfig qemu-coreboot/util/sconfig/lex.yy.o qemu-coreboot/util/sconfig/sconfig.tab.o qemu-coreboot/util/sconfig/main.o
/usr/bin/ld: qemu-coreboot/util/sconfig/main.o:/root/project/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; qemu-coreboot/util/sconfig/sconfig.tab.o:/root/project/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [util/sconfig/Makefile.inc:43: qemu-coreboot/util/sconfig/sconfig] Error 1
make[2]: Leaving directory '/root/project/build/coreboot-4.8.1'
make[1]: *** [Makefile:455: /root/project/build/coreboot-4.8.1/qemu-coreboot/.build] Error 1
make[1]: Leaving directory '/root/project'
make: *** [Makefile:668: all] Error 2
pgeorgi commented 3 years ago

Make that line in sconfig.h:80 state extern struct device *head;

pgeorgi commented 3 years ago

I put up the change for 4.8_branch at https://review.coreboot.org/c/coreboot/+/58534

tlaurion commented 3 years ago

Thanks @pgeorgi

tlaurion commented 6 months ago

I think those errors linked to buildstack and all others are now solved with #1661