[Question] How to build in Linux?

[Tx-Charlie]

[Question] How to build in Linux? It seems scripts like build_all.cmd only available in Windows.

[NicSavichev]

[Question] How to build in Linux? It seems scripts like build_all.cmd only available in Windows.

Most tools (including editors and dabuild) are built only for windows. You can build code for windows/linux/macos/etc. but you have to build/export assets/locations using Windows tools. You can download binaries for samples (that contain built game data) and build samples with just

cd prog/samples/physTest
jam
jam -f jamfile-test-jolt

cd samples/skiesSample/prog
jam

cd samples/testGI/prog
jam

then you should be able to run these samples under linux

[NicSavichev]

Some utils are buildabe for linux (e.g. vromfsPacker or csq), some can be ported (shader compiler for SpirV), maybe even dabuild with export plugins. But AssetViewer and DaEditorX is not portable (imo), at least not easily portable. So treat Dagor Engine as windows-based that can produce cross-platform projects (build on Windows for other platforms)

[AntonYudintsev]

Wine/Proton was capable to launch editor though. So it is possible to make all development solely in Linux, if you are ok with using prebuilt windows editors

[seanballais]

@NicSavichev Mind if I take a stab at porting some of the tools for Linux and contributing them back here? Thinking of doing it in my free time, especially in the hopes that my would-be contributions can help improve the Linux port of War Thunder.

[NicSavichev]

@NicSavichev Mind if I take a stab at porting some of the tools for Linux and contributing them back here? Thinking of doing it in my free time, especially in the hopes that my would-be contributions can help improve the Linux port of War Thunder.

I don't mind, most console tools are either already ported to linux or easily portable to linux. Shader compiler is a bit windows-bound but since we managed to compile dsc-metal under macOS I think it is possible to compile dsc-spirv under linux. dabuild may be ported too (it is partially ported already, at least texExp.cpp plugin since we used it as a service on linux servers). But AssetViewer and daEditorX are tightly bound to windows GUI so they cannot be ported (unless rewritten to some cross-platform GUI lib ike Qt).

[seanballais]

@NicSavichev Mind if I take a stab at porting some of the tools for Linux and contributing them back here? Thinking of doing it in my free time, especially in the hopes that my would-be contributions can help improve the Linux port of War Thunder.

I don't mind, most console tools are either already ported to linux or easily portable to linux. Shader compiler is a bit windows-bound but since we managed to compile dsc-metal under macOS I think it is possible to compile dsc-spirv under linux. dabuild may be ported too (it is partially ported already, at least texExp.cpp plugin since we used it as a service on linux servers). But AssetViewer and daEditorX are tightly bound to windows GUI so they cannot be ported (unless rewritten to some cross-platform GUI lib ike Qt).

Oh cool! Thanks for the pointers. I'll try either the shader compiler or dabuild first.

[pollend]

Would there be any interest in using a different build system? was wondering if that would make compiling more accessible? would be interested in getting everything to compile on linux.

[seanballais]

Would there be any interest in using a different build system? was wondering if that would make compiling more accessible? would be interested in getting everything to compile on linux.

Ideally, in the long term, I would vouch for switching to Ninja or CMake. However, the engine has existed for quite a long time already and is heavily integrated with Jam even for building for consoles (based on the code, at least with how I understand things). Changing the build system may also likely introduce risks to Gaijin from a business perspective. A switch to a different system may likely require retraining in-house devs and re-prioritization of tasks. The company also maintains the build system used, so I don't see them changing systems any time soon.

So, at least in the short term, I wouldn't want the engine to change build systems unless Gaijin would like to or if someone is willing to maintain a fork with a different build system, perhaps while ensuring feature parity with the engine.

[pollend]

Would there be any interest in using a different build system? was wondering if that would make compiling more accessible? would be interested in getting everything to compile on linux.

Ideally, in the long term, I would vouch for switching to Ninja or CMake. However, the engine has existed for quite a long time already and is heavily integrated with Jam even for building for consoles (based on the code, at least with how I understand things). Changing the build system may also likely introduce risks to Gaijin from a business perspective. A switch to a different system may likely require retraining in-house devs and re-prioritization of tasks. The company also maintains the build system used, so I don't see them changing systems any time soon.

So, at least in the short term, I wouldn't want the engine to change build systems unless Gaijin would like to or if someone is willing to maintain a fork with a different build system, perhaps while ensuring feature parity with the engine.

that makes sense, the other thing that can be done is write a wrapper similar to how gradle or repo works. that's probably a lot simpler to do, I guess for the moment I'll just download the deps manually and see what happens when i try to compile it on linux.

[seanballais]

If anyone is curious, as of November 9, 2023 (Philippine Time) I managed to get the shader compiler compiling. There are script errors though, so I'll have those fixed later today (yes, I am writing this at 2 in the morning :sweat_smile:).

I also made a draft PR here for anyone looking to check the progress.

[seanballais]

Hey, @NicSavichev! Just wanted some clarification: Since the README said that Clang is required, should the Linux port (currently porting ShaderCompiler2) also use Clang or stick with GCC since that's how things are seemingly set up at the moment?

[NicSavichev]

Hey, @NicSavichev! Just wanted some clarification: Since the README said that Clang is required, should the Linux port (currently porting ShaderCompiler2) also use Clang or stick with GCC since that's how things are seemingly set up at the moment?

It doesn't matter, we currently use both, GCC and CLANG, when building for linux. It is easier to use clang (since it is used in cross-compilation for many platforms so code likely suits clang) but GCC proved sometimes to generate better code so we still use it for linux-based servers and for WT client.

[seanballais]

Hey, @NicSavichev! Just wanted some clarification: Since the README said that Clang is required, should the Linux port (currently porting ShaderCompiler2) also use Clang or stick with GCC since that's how things are seemingly set up at the moment?

It doesn't matter, we currently use both, GCC and CLANG, when building for linux. It is easier to use clang (since it is used in cross-compilation for many platforms so code likely suits clang) but GCC proved sometimes to generate better code so we still use it for linux-based servers and for WT client.

I see. Alright. I'll try to make the shader compiler work with GCC. Parts of the shader compiler use anonymous PODs inside a union, which GCC does not like. I'll try figuring out a way to make it work (and review some of the more involved parts of C++) under GCC.

I appreciate any pointers too!

[pollend]

I got the skiesSample to build I seem to be missing shaders. I built the jam-8 repo from here: https://github.com/GaijinEntertainment/jam-G8

for ubuntu 23.10 here are the few packages I needed

sudo apt install libfltk1.3-dev libxkbfile-dev  libgtk2.0-dev

I ran a python3 env and installed

pip3 install clang
pip3 install cymbal

CLANG_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/ ~/devtools/jam -a

[NicSavichev]

I got the skiesSample to build I seem to be missing shaders. I built the jam-8 repo from here: https://github.com/GaijinEntertainment/jam-G8

You need to download and unpack samples-prebuilt-game.7z It contains prebuilt gameres and shaders (and EXE for windows, but it is not needed for linux)

[NicSavichev]

I got the skiesSample to build I seem to be missing shaders. I built the jam-8 repo from here: https://github.com/GaijinEntertainment/jam-G8

Yes, I will add instructions how to build under Linux a little bit later (required packages and general build sequence). For now I just checked that everything (samples and some tools) are built on machines with proper environment.

[pollend]

I got the skiesSample to build I seem to be missing shaders. I built the jam-8 repo from here: https://github.com/GaijinEntertainment/jam-G8

Yes, I will add instructions how to build under Linux a little bit later (required packages and general build sequence). For now I just checked that everything (samples and some tools) are built on machines with proper environment.

that's nice, I got the skies sample running. I guess that is a start umm, is there any guidelines if someone wanted to contribute code?

[NicSavichev]

Use https://github.com/GaijinEntertainment/DagorEngine/blob/main/build_all_linux.sh to build all tools (all console tools except for daEditorX and assetViewer) and then shaders and code of samples. Then use https://github.com/GaijinEntertainment/DagorEngine/blob/main/dabuild_all_linux.sh to dabuild assets of sample projects. After that samples are runable under linux (and macOS too when built with relevant shell scripts).

Scripts are not mandatory for regular use, we just use jam, compile_shaders_spirv.sh and ../../tools/dagor3_cdk/bin-linux64/daBuild-dev ../application.blk -jobs:$(getconf _NPROCESSORS_ONLN) -q for regular update actions, but these script will show proper usage and switches used for builds.