godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.13k stars 69 forks source link

Save intermediate files (.o/.obj/.lib/.a etc) to a separate directory (out-of-tree builds) #2973

Open DrRevert opened 3 years ago

DrRevert commented 3 years ago

Describe the project you are working on

Game which requires modifying engine on core level thus requiring to create my own custom build

Describe the problem or limitation you are having in your project

The intermediate object files created by the compiler and linker (.o/.obj/.lib etc.) are saved to the same directory as source code files (.h/.cpp). Which causes some inconveniences (false positives when diffing multiple branches, no easy way to remove old object files, and overall issues when navigating source code directories)

Describe the feature / enhancement and how it helps to overcome the problem or limitation

All intermediate files created by the compiler/linker should be saved to a separate directory called obj or intermediate. This will improve the workflow with making custom builds, especially when multiple platforms are compiled. It will be easier to force rebuild Godot simply by deleting the intermediate directory, without having to filter out *.obj files using scripts or by hand. When performing diffs between two versions of the engine would it will be easier to simply ignore one directory instead of ignoring multiple binary files located next to the source code.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

All intermediate files created by the compiler/linker should be saved to a separate directory called obj or intermediate.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, it's an improvement to the building process.

Is there a reason why this should be core and not an add-on in the asset library?

It's an improvement to the building process.

YuriSizov commented 3 years ago

no easy way to remove old object files

Build artifacts can be removed with either a scons command or with Git (e.g. git clean -fxd) since generated files are ignored.

overall issues when navigating source code directories

Can you elaborate on that?

fire commented 3 years ago

This is a feature of the meson port but that has slowed.

DrRevert commented 3 years ago

no easy way to remove old object files

Build artifacts can be removed with either a scons command or with Git (e.g. git clean -fxd) since generated files are ignored.

That's true, but is arguably less convenient than removing directory.

overall issues when navigating source code directories

Can you elaborate on that?

Sure, it's about the visual clutter similar to this image Every configuration creates one extra file for every cpp, not counting .lib or .a files. If you compile 3 different configurations you will have 3 .objs for every 1 .cpp and that's only one platform.

Calinou commented 3 years ago

As far as I know, SCons does not support out-of-tree builds. Implementing this would require switching to another build system.

PS: Many IDEs/editors such as Visual Studio Code can be configured to hide files with specific extensions in their filesystem docks. **/*.o, **/*.obj, **/*.a, **/*.lib will hide intermediate build files on all platforms.

DrRevert commented 3 years ago

As far as I know, SCons does not support out-of-tree builds. Implementing this would require switching to another build system.

PS: Many IDEs/editors such as Visual Studio Code can be configured to hide files with specific extensions in their filesystem docks.

Actually I found that there is a chapter about separating source and build trees in Scons manual but I'm not sure how easily can it be integrated in the current build flow without causing any bigger issues. In theory it should be as easy as adding extra parameter to every SConscript call. I'll followup when I found some time to experiment with that more.

Xrayez commented 3 years ago

Yes, SCons supports this via VariantDir.

But the problem is that Godot build scripts heavily rely on custom Python path resolution (using os.path rather than File() or Dir()), so it makes it problematic to use at the moment. You'll likely stumble upon build errors if you try to use VariantDir, I've tried this for slightly different use case in the past, like building Godot as part of another SCons project via SConscript, and that failed miserably, see godotengine/godot#27842.

roboboredom commented 4 months ago

You should put a link to this thread on the godot-cpp readme or at least the wiki tbh