search

CesiumGS / cesium-unreal

Bringing the 3D geospatial ecosystem to Unreal Engine
https://cesium.com/platform/cesium-for-unreal/
Apache License 2.0
885 stars 286 forks source link

Plugin artifacts for UE 5.4 are huge (too many objects files) #1410

Closed csciguy8 closed 2 months ago

csciguy8 commented 2 months ago

Artifacts for Unreal 5.4 seem to be twice as big as the last version...

Ex. CesiumForUnreal-54-main -> 2.35 GB CesiumForUnreal-53-main -> 1.12 GB

Refer to this CI build

This was originally noticed during code review, and looks to be that UE 5.4 is generating way more object files than we expect.

kring commented 2 months ago

The problem is that 5.4 changed how UBT decides which files are in the "working set". Working set files are excluded from the adaptive unity build, and so they balloon the size of our plugin.

Specifically, the 5.4 bug (that's what I'd call it) was introduced on this line: https://github.com/EpicGames/UnrealEngine/blob/5.4.1-release/Engine/Source/Programs/UnrealBuildTool/System/SourceFileWorkingSet.cs#L395

As a result of that line, any time we're using an "installed" engine (which we are, because we didn't build it from source), we get a PerforceSourceFileWorkingSet. That working set implementation deems any file that is not read-only to be part of the working set.

As for how to fix it, I'm less sure. Setting all our source files to read-only before doing the build on CI would probably work.

kring commented 2 months ago

What I wrote above is probably not the full story, though, because as far as I can tell UE 5.3 would have used the PerforceSourceFileWorkingSet too, and it would have behaved the same way. So I'm not sure why this is a new problem.

csciguy8 commented 2 months ago

I just realized that the slow builds observed in 5.4 vs 5.3 can probably be attributed to this bug too. Non-unity builds are slower.