Clang doesn't do LTO

[kayhayen]

Hello,

with Nuitka I have LTO working with MinGW64. However, when I enable it with Clang, at link time, it tells me it doesn't understand the file format of the .o files. Looking at it with file it tells me it is bytecode.

But first without, this command line from the command:

python3.9 bin/nuitka --mingw --clang --lto=no --show-scons tests/basics/Asserts.py

"C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\ccache.exe" "C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\clang.exe" -o static_src\MetaPathBasedLoader.o -c -std=c11 -fvisibility=hidden -fwrapv -pipe -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -D_XOPEN_SOURCE -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=0 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -IC:\Python39_64\include -I. -IC:\Users\kayha\repos\Py2C\nuitka\build\include -IC:\Users\kayha\repos\Py2C\nuitka\build\static_src static_src\MetaPathBasedLoader.c

gives me:

file Asserts.build/static_src/MetaPathBasedLoader.o
Asserts.build/static_src/MetaPathBasedLoader.o: Intel amd64 COFF object file, not stripped, 8 sections, symbol offset=0x3fc9, 157 symbols, 1st section name ".text"

And that works, and has been used every since Clang on MinGW64 was made to work with Nuitka. Currently it's regressed a bit, so factory branch would have to be used, to not have to say --lto=no.

However, with yes, I get this:

"C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\ccache.exe" "C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\clang.exe" -o static_src\MetaPathBasedLoader.o -c -flto -std=c11 -fvisibility=hidden -fwrapv -pipe -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -D_XOPEN_SOURCE -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=0 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -IC:\Python39_64\include -I. -IC:\Users\kayha\repos\Py2C\nuitka\build\include -IC:\Users\kayha\repos\Py2C\nuitka\build\static_src static_src\MetaPathBasedLoader.c

file Asserts.build/static_src/MetaPathBasedLoader.o
Asserts.build/static_src/MetaPathBasedLoader.o: LLVM IR bitcode

However, it gets rejected as file not recognized: file format not recognized and that's at link time using this command:

"C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\ccache.exe" "C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\clang.exe" -o static_src\MetaPathBasedLoader.o -c -flto -std=c11 -fvisibility=hidden -fwrapv -pipe -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -D_XOPEN_SOURCE -D_WIN32_WINNT=0x0501 -D__NUITKA_NO_ASSERT__ -DMS_WIN64 -D_NUITKA_CONSTANTS_FROM_RESOURCE -D_NUITKA_FROZEN=0 -D_NUITKA_EXE -D_NUITKA_PLUGIN_MULTIPROCESSING_ENABLED=1 -IC:\Python39_64\include -I. -IC:\Users\kayha\repos\Py2C\nuitka\build\include -IC:\Users\kayha\repos\Py2C\nuitka\build\static_src static_src\MetaPathBasedLoader.c
C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\11.2.0-12.0.1-9.0.0-r1\mingw64\bin\clang.exe -o C:\Users\kayha\repos\Py2C\Asserts.exe -flto -Wl,--exclude-all-symbols -Wl,--out-implib,.\import.lib -municode -O3 -Wl,--stack,9863168 @".\@link_input.txt" -LC:\Python39_64\libs -lm -lpython39

Right now I am assuming that LTO with Clang as distributed by winlibs is just not supported. However, I feel that makes comparing performance between GCC and Clang a bit unfair here.

Yours, Kay

[brechtsanders]

Do you have the same problem without ccache?

[kayhayen]

Yes it does, but good point, didn't think of that.

[brechtsanders]

I never got ccache 4 to work properly on Windows. Version 3 worked better, but I usually build without until they get it working properly....

[kayhayen]

But ccache is not part of the equation, it doesn't work without it either.

[kayhayen]

I think it's really that some LTO support is missing from the clang linker, much like it did for gcc some time ago. The linking side seems to not support the bytecode written by the compiler backend. I assume there is a missing bit somewhere. For what it's worth, I believe that Clang based on MSVC works for LTO, but I need to verify if it's really doing it.