CesiumGS / cesium-native

Apache License 2.0
414 stars 210 forks source link

CI build improvements #815

Closed jherico closed 4 months ago

jherico commented 6 months ago

This PR ...

compiler caching

ccache/sccache are tools that will wrap the call from the build system (such as make or ninja) to the compiler and determine whether the file has already been compiled compared to the version in the cache. There are two kinds of cache hits...

Cache misses are built as normal but then the resulting object file is placed into the cache with the keys allowing the build to potentially skip that file in the future.

This can result in an order of magnitude drop in build times when the bulk of the files are unchanged, or effectively unchanged after preprocessing.

Fixes #814

jherico commented 6 months ago

For comparison, here are the timings for a build where the compiler cache was empty...

image

And here are the timings for a build with the cache populated...

image

There are, generally, very large improvements, with the overall build time for an optimally used cache cut roughly in half. The limiting factor for getting it down even further is whatever seems to be going on here:

image

There's an excessive delay between the last compile line timestamp and the linking step. This delay only shows up in two places out of all the jobs... the RelWithDebInfo configs for both macos and gcc on ubuntu-latest. It doesn't happen with Clang on ubuntu and it doesn't happen on any debug config.

In both cases the compiler cache output looks roughly like this...

image

I'm unclear on what is causing the cache misses since the exact same code is being compiled as the prior execution. Assuming it is part of the compilation, I'm investigating the Catch2 functionality to see if there's any mechanism for fuzzing or randomization at build time of test inputs, which might contribute to defeating the compiler cache.

kring commented 4 months ago

Thanks @jherico, this is a nice improvement. Merging!