Viladoman / CompileScore

Tools for profiling and visualizing C++ build times.
MIT License
464 stars 19 forks source link

Unreal Engine Improvements #20

Closed rlabrecque closed 9 months ago

rlabrecque commented 2 years ago

Hey there, I'm impressed with how well this works with Unreal out of the box! That being said there's a few parts that could be improved.

  1. Running "Build and Profile" on just the game project causes a full rebuild of the entire engine. I suspect this is due to Unreal being 'smart' about which toolchains it's using. We're using AutoSDK and I'm guessing CompileScore only uses the built in toolchain. Not sure if there's any easy answers to this one.
  2. Need to manually disable "Unity" (Fat/Amalgamation/Combined) builds. Currently CompileScore isn't terribly useful just because the results are not accurate due to the unity builds. If CompileScore did this automatically somehow it would definitely cause a full recompile every time. Also not really sure if there's an easy answer to this.

I think Unreal can output various forms of compilation databases, which may be interesting to replay the exact same commands with the exact same compilers Unreal is using.

I think some documentation on these (or other best practices on using CompileScore with Unreal) would probably satisfy me. Worst case at least having a ticket open with some of these gotcha's is hopefully enough for anyone else wanting to try this out 👍

Thanks for making this!

Viladoman commented 2 years ago

Hi! Thanks for all the info and for using the extension.

I don't know how your game solution and projects are organized, but there is already a way to build/rebuild and profile a single project, using the context menu on the project in the solution explorer.

image

Hopefully this can help with your iteration times. If this is not good enough I would like to learn more about it.

I am not familiar with AutoSDK, but I guess eventually it will invoke a C++ compiler ( MSVC, Clang, GCC... ). If it ends up calling MSVC then your should have no problem capturing the events as they compile. The recorder will capture anything that MSCV builds in the machine. This means that it doesn't matter who invokes the compiler, the recorder will capture it.

Still you should consider the following:

In the case of Clang you might need to add the -ftime-trace to the compiler invocation so that it generates the traces. This is more robust to distributed builds because each translation unit creates its own trace data, but you need to make sure the trace information files ( .obj.json ) are brought back from the remote machines.

For more detailed information on how the extraction works you can check the wiki page.

Regarding Unity builds, it should still give you the results and the expensive headers. Although removing some dependencies might have no impact to the build times as that dependency will be then added by some other file in the same unity unit. I totally agree that you can get better insights by decomposing the unity files into single translation units.

Sadly, I am not super familiar with Unreal, but we can definitely create a wiki page with best practices.

rlabrecque commented 2 years ago

I think the core issue I have is using "Build and Profile" like that on the individual game project ends up being the exact same as "Rebuild Solution and Profile". I'll try to dig into it more in the future to try and figure out exactly why this is. Currently it's like ~7 minutes for rebuilding just the project vs 30 minutes for the entire solution on my machine/current project.

Viladoman commented 9 months ago

Regarding this issue there have been 2 different developments that should solve it. This PR fixed some issues with nested projects which might have impacted what you described in here. On top of that there 2 new actions that can be triggered on demand to 'Start Trace' and 'Stop Trace' in the Extensions->Compile Score->Actions. These can be mapped to any key shortcut or custom buttons so you can do a custom trace using the regular build mechanisms.

rlabrecque commented 9 months ago

Lovely, thanks, I'll give this a try again shortly!