llvm: add support for rebuild and PGO

[Andarwinux]

This reduces the first CI build time by 20 minutes. For cached builds, the build system itself and ccache are the biggest bottlenecks. Local users can also easily try LLVM main branch and update LLVM at any time without waiting hours.

[shinchiro]

This reduces the first CI build time by 20 minutes. For cached builds, the build system itself and ccache are the biggest bottlenecks.

How about compiling with ccache disabled?

[zhongfly]

https://releases.llvm.org/18.1.0/tools/clang/docs/UsersManual.html#profiling-with-instrumentation According to the documentation, my understanding of the steps to enable PGO for llvm is as follows, please correct me if there are any errors:

1. build llvm with -fprofile-generate flag.
2. run llvm with LLVM_PROFILE_FILE environment variable, profile data will written to LLVM_PROFILE_FILE.LLVM_PROFILE_FILE will override -fprofile-generate=<dir> setting, so no need to specify profile data dir in step 1.If we only want to use PGO for llvm, there's no need to build toolchain with -fprofile-generate.
3. use profile data to rebuild llvm.I'm not sure if -fprofile-use and -fprofile-generate can be enabled at the same time, but I'm guessing they can.

[zhongfly]

-fprofile-use and -fprofile-generate are mutually exclusive options.

I haven't found any documentation that says they can't be enabled at the same time, I'll try it later.

[zhongfly]

There does exist another PGO that supports and requires both generate and use, called CSPGO, but that requires three builds, which I haven't tried.

I'm wondering if there's a way to continually optimize iteratively using PGO, currently seems to be possible by continually merging profile data and then building CSPGO? Build LLVM( -fprofile-generate) → Build LLVM(CSPGO) → LLVM source update → Build LLVM(CSPGO), instead of 'Build LLVM( -fprofile-generate) → Build LLVM( -fprofile-use )' every time.

[zhongfly]

How about compiling with ccache disabled?

Build mpv,disable ccache, no cache,disable llvm lto,CLANG_PACKAGES_LTO=ON:

• Without PGO: 1h
• Training LLVM: 1.5h
• With PGO: 40+min

Haven't compared training with build mpv and just build toolchain yet.

[zhongfly]

Is this CSPGO?

No, CSPGO takes too much time and you replied earlier that the difference is not significant.

Using only the training data from building the toolchain only took 1 minute more compared to merging the data from building all the packages. So it looks like we only need to use the toolchain build to collect data.(CLANG_PACKAGES_LTO=ON)

It may take more tries to prove it, but I'm too lazy to do that.

[shinchiro]

Disable thinlto for the CI.

[shinchiro]

As usual untested on my local machine but hopefully it does not give me problem later.

[shinchiro]

Duh, update requirement in readme too