Build the Windows binaries fails with missing compiler

[zqb-all]

Description

Build the Windows binaries fail when configure libiconv-1.17

Steps to Reproduce

According to "Build the Windows binaries" in https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-MAINTAINER.md

git -C ~/Work/xpacks/openocd-xpack.git pull && \
xpm run install -C ~/Work/xpacks/openocd-xpack.git && \
git -C ~/Work/xpacks/xbb-helper-xpack.git pull && \
xpm link -C ~/Work/xpacks/xbb-helper-xpack.git && \
xpm run link-deps -C ~/Work/xpacks/openocd-xpack.git && \
xpm run deep-clean --config win32-x64 -C ~/Work/xpacks/openocd-xpack.git && \
xpm run docker-prepare --config win32-x64 -C ~/Work/xpacks/openocd-xpack.git && \
xpm run docker-link-deps --config win32-x64 -C ~/Work/xpacks/openocd-xpack.git
xpm run docker-build-develop --config win32-x64 -C ~/Work/xpacks/openocd-xpack.git

fail with log:

[bash /home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure --prefix=/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/x86_64-w64-mingw32/install --libdir=/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/x86_64-w64-mingw32/install/lib --includedir=/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/x86_64-w64-mingw32/install/include --mandir=/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/x86_64-w64-mingw32/install/share/man --build=x86_64-pc-linux-gnu --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --disable-debug --disable-dependency-tracking --disable-silent-rules --disable-nls --enable-static --enable-extra-encodings]
checking for a BSD-compatible install... /usr/bin/install
checking whether build environment is sane... yes
checking for x86_64-w64-mingw32-strip... x86_64-w64-mingw32-strip
checking for a race-free mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make sets $(MAKE)... (cached) yes
checking for x86_64-w64-mingw32-gcc... x86_64-w64-mingw32-gcc
checking whether the C compiler works... no
configure: error: in `/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/x86_64-w64-mingw32/build/libiconv-1.17':
configure: error: C compiler cannot create executables
See `config.log' for more details

Make the build folder writable by all...

[chmod -R a+w /home/mark/Work/xpacks/openocd-xpack.git/build]

Make the xpacks folder writable by all...

[chmod a+w /home/mark/Work/xpacks/openocd-xpack.git/xpacks]
error: running 'bash  scripts/build.sh --target win32-x64 --build-folder build/win32-x64 --windows --develop' failed
error: running 'docker exec --user $(id -un) openocd-0.12.0-1.1-win32-x64 xpm run build-develop --config win32-x64' failed

Versions

• [OpenOCD version] Haven't started compiling openocd yet
• [operating system] windows wsl ubuntu 20.04

[ilg-ul]

That's weird, since the same scripts were used to build the latest release.

It looks like the C compiler fails. What does the config.log say about it?

[zqb-all]

config.log

config.log show that

configure:4600: checking for x86_64-w64-mingw32-gcc
configure:4632: result: x86_64-w64-mingw32-gcc
configure:5030: checking for C compiler version
configure:5039: x86_64-w64-mingw32-gcc --version >&5
/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure: line 5041: x86_64-w64-mingw32-gcc: command not found
configure:5050: $? = 127
configure:5039: x86_64-w64-mingw32-gcc -v >&5
/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure: line 5041: x86_64-w64-mingw32-gcc: command not found
configure:5050: $? = 127
configure:5039: x86_64-w64-mingw32-gcc -V >&5
/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure: line 5041: x86_64-w64-mingw32-gcc: command not found
configure:5050: $? = 127
configure:5039: x86_64-w64-mingw32-gcc -qversion >&5
/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure: line 5041: x86_64-w64-mingw32-gcc: command not found
configure:5050: $? = 127
configure:5039: x86_64-w64-mingw32-gcc -version >&5
/home/mark/Work/xpacks/openocd-xpack.git/build/win32-x64/sources/libiconv-1.17/configure: line 5041: x86_64-w64-mingw32-gcc: command not found
configure:5050: $? = 127

[ilg-ul]

x86_64-w64-mingw32-gcc: command not found

Something was wrong when the toolchain was installed in the docker image.

Run the docker-remove action, add --verbose to the docker exec ... xpm install ... lines, retry and take a closer look at the console in the initial phase, you should see when the toolchain is installed and links are created to each binary file.

[zqb-all]

thanks, I will try install the toolchain manually first, to see if there are any other problems

 apt-get install gcc-mingw-w64-x86-64 g++-mingw-w64-x86-64 wine64

then I will try to remove docker, retry and take a closer look at the console in the initial phase

[ilg-ul]

The commands you used are specific to using docker in a development environment (basically the environment I use for maintaining the packages and making releases).

Although running local builds might be possible, it requires you to satisfy all dependencies manually, which is not trivial, not to mention that the procedure is not documented and there is no support in case something fails.

[zqb-all]

I try to enter the docker, install toolchain by

 apt-get install gcc-mingw-w64-x86-64 g++-mingw-w64-x86-64 wine64

then run build cmd again,

 xpm run docker-build-develop --config win32-x64 -C ~/Work/xpacks/openocd-xpack.git

the build completes successfully

then I remove the docker container, add --verbose in package.json

diff --git a/package.json b/package.json
index 529b27b..7524a97 100644
--- a/package.json
+++ b/package.json
@@ -184,8 +184,8 @@
             "docker exec {{ properties.containerName }} groupadd --gid $(id -g) --force $(id -gn)",
             "docker exec {{ properties.containerName }} useradd --home-dir ${HOME} --uid $(id -u) --gid $(id -g) $(id -un)",
             "docker exec --user $(id -un) {{ properties.containerName }} bash -c 'lsb_release -sd && whoami && pwd && ls -lLA'",
-            "docker exec --user $(id -un) {{ properties.containerName }} xpm install",
-            "docker exec --user $(id -un) {{ properties.containerName }} xpm install --config {{ configuration.name }}"
+            "docker exec --user $(id -un) {{ properties.containerName }} xpm install --verbose",
+            "docker exec --user $(id -un) {{ properties.containerName }} xpm install --verbose --config {{ configuration.name }}"
           ],
           "docker-link-deps": [
             "docker exec --user $(id -un) {{ properties.containerName }} xpm link -C ${HOME}/Work/xpacks/xbb-helper-xpack.git",

retry build, still failed beacuse of missing compiler. I recorded the operation using the script : typescript-xpack-openocd-win64-fail.log

But I can't find the log of install toolchain. Can you tell me what keywords should I check?

[zqb-all]

I check the ~/.local/xPacks dir , there are .content in /home/mark/.local/xPacks/@xpack-dev-tools/gcc/12.2.0-2.1, but no .content in /home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc/12.2.0-1.1

mark@laptop-mark:~/Work$ ll ./xpacks/openocd-xpack.git/build/win32-x64/xpacks/xpack-dev-tools-gcc
lrwxrwxrwx 1 mark mark 56 Mar 23 21:21 ./xpacks/openocd-xpack.git/build/win32-x64/xpacks/xpack-dev-tools-gcc -> '/home/mark/.local/xPacks/@xpack-dev-tools/gcc/12.2.0-2.1'/
mark@laptop-mark:~/Work$ ll /home/mark/.local/xPacks/@xpack-dev-tools/gcc/12.2.0-2.1
total 48
dr-xr-xr-x  3 mark mark  4096 Mar 22 14:58 ./
drwxr-xr-x  3 mark mark  4096 Mar 22 14:49 ../
dr-xr-xr-x 12 mark mark  4096 Mar 22 14:58 .content/
-r--r--r--  1 mark mark  1518 Mar 22 14:49 CHANGELOG.md
-r--r--r--  1 mark mark  1070 Mar 22 14:49 LICENSE
-r--r--r--  1 mark mark 10159 Mar 22 14:49 README.md
-r--r--r--  1 mark mark 16140 Mar 22 14:49 package.json

mark@laptop-mark:~/Work$ ll /home/mark/.local/xPacks/@xpack-dev-tools/gcc/12.2.0-2.1/.content/bin/gcc
-r-xr-xr-x 1 mark mark 940232 Dec 28 03:28 '/home/mark/.local/xPacks/@xpack-dev-tools/gcc/12.2.0-2.1/.content/bin/gcc'*

mark@laptop-mark:~/Work$ ll ./xpacks/openocd-xpack.git/build/win32-x64/xpacks/xpack-dev-tools-mingw-w64-gcc
lrwxrwxrwx 1 mark mark 66 Mar 23 21:21 ./xpacks/openocd-xpack.git/build/win32-x64/xpacks/xpack-dev-tools-mingw-w64-gcc -> '/home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc/12.2.0-1.1'/
mark@laptop-mark:~/Work$ ll /home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc/12.2.0-1.1
total 60
drwxr-xr-x 2 mark mark  4096 Mar 22 14:49 ./
drwxr-xr-x 3 mark mark  4096 Mar 22 14:49 ../
-rw-r--r-- 1 mark mark  4591 Mar 22 14:49 CHANGELOG.md
-rw-r--r-- 1 mark mark  1070 Mar 22 14:49 LICENSE
-rw-r--r-- 1 mark mark 16597 Mar 22 14:49 README.md
-rw-r--r-- 1 mark mark 20442 Mar 22 14:49 package.json

[ilg-ul]

no .content in /home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc/12.2.0-1.1

This looks like an incomplete download.

Remove /home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc and retry.

[ilg-ul]

And be sure you are using the latest xpm on your host, error processing was improved in the latest release.

[zqb-all]

Remove /home/mark/.local/xPacks/@xpack-dev-tools/mingw-w64-gcc and retry. Still fail

typescript-xpack-openocd-win64-fail-v2.log

I'll check the log later to see why the toolchain download fail

[ilg-ul]

request to https://github.com/xpack-dev-tools/mingw-w64-gcc-xpack/releases/download/v12.2.0-1/xpack-mingw-w64-gcc-12.2.0-1-linux-x64.tar.gz failed, reason: Client network socket disconnected before secure TLS connection was established in fetch https://github.com/xpack-dev-tools/mingw-w64-gcc-xpack/releases/download/v12.2.0-1/xpack-mingw-w64-gcc-12.2.0-1-linux-x64.tar.gz
exit(3)
error: download failed, quit
error: running 'docker exec --user $(id -un) openocd-0.12.0-1.1-win32-x64 xpm install --verbose --config win32-x64' failed

Your docker container is not able to download from https URLs. Usually this happens when you use proxies. If this is your case, you have to inform Docker about those proxies.

[zqb-all]

No proxy is configured on my computer. I tried to download toolchain using wget and it worked. Then I tried xpm run docker-prepare again, and it worked properly this time.

Maybe the previous failures were due to unstable access to github in China.

I have no more questions. Thank you very much for your help

[ilg-ul]

Mark @zqb-all, a quick update.

I further investigated the issue and I identified a problem with the permissions for the $HOME/Work/cache folder, on fresh runs this folder did not exist, and Docker created it with root permissions, which prevented the download to complete.

I updated the scripts to always create the folder before starting Docker, and now the builds should pass even on fresh runs.

[ilg-ul]

Mark @zqb-all, another quick update.

If you are still interested in building OpenOCD, there is also a new documentation page with slightly easier to follow instructions:

• https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-DEVELOPER.md#build-the-windows-binaries

Also the XBB prerequisites page was re-worked:

• https://xpack.github.io/xbb/prerequisites/

I would appreciate any feedback on them.