GCC's parallel LTO assumes a non-Windows shell

[amyspark]

Hi @StephanTLavavej,

I was trying to use your distro as an acid test for a GCC issue that was recently fixed, and I found that the -flto option of GCC cannot be used with yours:

[14/16] Linking target deps/libusb/libusb/libusb-1.0-0.dll El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccMp96s0.mk:3: C:\Users\Amalia\AppData\Local\Temp\ccxRPCaW.ltrans0.ltrans.o] Error 1 (ignored) El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccMp96s0.mk:6: C:\Users\Amalia\AppData\Local\Temp\ccxRPCaW.ltrans1.ltrans.o] Error 1 (ignored) [16/16] Linking target src/bmpflash.exe lto1.exe: warning: visibility attribute not supported in this configuration; ignored [-Wattributes] El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccUx7h4p.mk:3: C:\Users\Amalia\AppData\Local\Temp\cc2CzFQU.ltrans0.ltrans.o] Error 1 (ignored) El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccUx7h4p.mk:6: C:\Users\Amalia\AppData\Local\Temp\cc2CzFQU.ltrans1.ltrans.o] Error 1 (ignored) El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccUx7h4p.mk:9: C:\Users\Amalia\AppData\Local\Temp\cc2CzFQU.ltrans2.ltrans.o] Error 1 (ignored) El sistema no puede encontrar la ruta especificada. make: [C:\Users\Amalia\AppData\Local\Temp\ccUx7h4p.mk:12: C:\Users\Amalia\AppData\Local\Temp\cc2CzFQU.ltrans3.ltrans.o] Error 1 (ignored)

A bit of Procmon shows that Make is trying to call up a Bourne shell in the backend:

I'm not sure if it's a bug that's possible to fix on your packaging, or the LTO plugin should be disabled altogether.

[StephanTLavavej]

This seems like a new requirement - it used to work years ago. I'm not aware of any workaround, so I might have to disable LTO.

[amyspark]

@stl Hi again, I've digged into this tonight and I realised it's a GCC bug on Windows, introduced sometime around 9.2. It's 100% reproducible with your toolchain if one uses the compiler flag -flto=auto or -flto=N with N higher than 2.

I found the following references online, but no clue as to whether it was fixed, or why it's not so easily reproducible on MSYS2 GCC:

• https://bugs.launchpad.net/gcc-arm-embedded/+bug/1853451
• https://github.com/official-stockfish/Stockfish/issues/2977#issuecomment-671652515
• https://github.com/msys2/MINGW-packages/issues/11706

[starg2]

The problem is GCC expects touch, mv, and /dev/null to be available even on Windows.

When parallel LTO is enabled, GCC creates a temporary makefile and invokes make to parallelize code generation. (The make command can be overriden by MAKE environment variable btw.) The makefile looks like this:

C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.ltrans.o:
    @D:\msys64\ucrt64\bin\gcc.exe  '-xlto' '-c' '-fno-openmp' '-fno-openacc' '-fpic' '-fcf-protection=none' '-march=haswell' '-mtune=znver2' '-O3' '-O3' '-s' '-shared' '-mtune=generic' '-march=nocona' '-dumpdir' 'out\bin\libvorbisenc.dll.' '-dumpbase' 'out\bin\libvorbisenc.dll.ltrans0.ltrans' '-fltrans' '-o' 'C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.ltrans.o' 'C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.o'
    @-touch -r "C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.o" "C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.o.tem" > /dev/null 2>&1 && mv "C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.o.tem" "C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.o"
.PHONY: all
all: \
    C:\Users\user\AppData\Local\Temp\cce4yGcr.ltrans0.ltrans.o

Since Windows doesn't have touch, mv, or /dev/null, the commands on line 3 will fail.

The solution is to use msys2 shell, which provides the required functionality. Alternatively, you can patch lto-wrapper to fix or suppress the generation of the problematic commands. (Search gcc/lto-wrapper.cc for touch.)

[StephanTLavavej]

Thanks, this should be reported upstream to GCC.

[amyspark]

Bug report: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710

[Peter0x44]

I submitted a patch upstream to fix this: https://gcc.gnu.org/pipermail/gcc-patches/2024-April/650142.html

[Peter0x44]

The patch was applied upstream, but it's too late for 14.1. Hopefully it makes it to gcc 14.2.