Apply DSO sorting patches to glibc in Linux images

[slonopotamus]

See https://patchwork.sourceware.org/project/glibc/list/?q=17645 These patches severely reduce non-monolithic UE startup time.

While they're still not merged into glibc upstream, Arch Linux uses them.

[TBBle]

Are you thinking of switching the base Linux container to arch and using their glibc-dso package?

[slonopotamus]

No-no, that would be too much intrusive. I'm thinking about just applying a patch to Ubuntu package.

[TBBle]

Patch and rebuild glibc? That sounds somewhat out-of-scope for this project, and still pretty intrusive. Perhaps if you create a PPA for this rebuild, then we have upstream build architecture/pipeline, some level of supply-chain assurance, and don't need to mess about too much in ue4-docker.

It still feels pretty intrusive either way, though, particularly to deal with the nvidia cuda variants etc.

Or perhaps I am misunderstanding, this looked like a source change when I was skimming the patches earlier.

[slonopotamus]

I thought once again, possibly you're right. User can simply build new image on top of ue4-minimal/ue4-full and do there modifications to glibc (either via patching and rebuilding or by using PPA). As far as I know, such PPA doesn't exist yet. If I manage to publish patched glibc to PPA, I'll leave a link here.

[adamrehn]

This is something that I've been interested in for a while, since it would speed up cook times for both building the Engine DDC and for packaging projects. The impression I get from reading through the messages on the #linux channel of the Unreal Slackers Discord server is that no one from the Unreal Engine community has managed to get the DSO sorting patch working on Ubuntu yet, possibly due to the fact that current LTS releases use an older version of glibc than Arch uses.

[slonopotamus]

See https://launchpad.net/~slonopotamus/+archive/ubuntu/glibc-dso

[slonopotamus]

For the record:

1. PPA above provides patched glibc for Ubuntu 18.04, 20.04, 21.04 and 21.10
2. Patches DO NOT have noticeable impact on ue4-docker build times
3. Patches DO have noticeable impact on Unreal Engine user usage. For example, editor startup of 5.0.0-early-access-2 on a Ryzen 3700X goes from ~70s down to ~7s. This also affects commandlets (that are often used as part of CI).
4. Installing patch is as easy as sudo apt update -y && sudo apt install -y software-properties-common && sudo add-apt-repository -y ppa:slonopotamus/glibc-dso && sudo apt update -y && sudo apt upgrade -y libc6 && export GLIBC_TUNABLES=glibc.rtld.dynamic_sort=2

[javierfurus]

For the record:

1. PPA above provides patched glibc for Ubuntu 18.04 and 20.04
2. Patches DO NOT have noticeable impact on ue4-docker build times
3. Patches DO have noticeable impact on Unreal Engine user usage. For example, editor startup of 5.0.0-early-access-2 on a Ryzen 3700X goes from ~70s down to ~7s. This also affects commandlets (that are often used as part of CI).
4. Installing patch is as easy as sudo apt update -y && sudo apt install -y software-properties-common && sudo add-apt-repository -y ppa:slonopotamus/glibc-dso && sudo apt update -y && sudo apt upgrade -y libc6 && export GLIBC_TUNABLES=glibc.rtld.dynamic_sort=2

Wonderful! Any chance you could build 2.33 for Ubuntu 21.04? :) I think I could myself, but guides are rather lacking.

[slonopotamus]

@javierfurus Done, same PPA now also provides package for Ubuntu 21.04.

[javierfurus]

Thank you for your amazing work!

[maxjust]

@slonopotamus , hi, can you please add ubuntu 21.10 build to PPA?

[slonopotamus]

@maxjust Done.

[maxjust]

@slonopotamus thanks for your fast reply and rebuild. But still unreal starting around 2 minutes. I have add vars to .bashrc, and try to pass it dircetly via cmd GLIBC_TUNABLES=glibc.rtld.dynamic_sort=2 ./UE4Editor but still no effect. every launch takes 2 minutes. If anybody have an ideas of how to fix it, welcomes. And anyway @slonopotamus thanks for your quick reply and rebuild!

[veritas1]

@slonopotamus Thanks so much for the patch. I am using PopOS (Ryzen 7 2700X, 32gb RAM, GTX 1050 Ti) and compiled UE v5.1.0. The editor launch would take over 4 minutes, but with your patch it is now just a few seconds :open_mouth: :clap:

[slonopotamus]

@maxjust I made an error when built package for Ubuntu 21.10: I disabled x86 package. That caused apt-get to refuse to update glibc. Fixed package was published on 2022-01-28.

[adamrehn]

@slonopotamus ah, so that's what I was hitting the other day when I tried to install it on one of the machines at my office. I wasn't sure if it was something specific to that system, so I was planning on testing in a fresh Ubuntu VM to verify before reporting it to you. Good to know it's fixed, I'll retry on that machine again.

[maxjust]

@slonopotamus thanks for fixing, now all working fine, unreal start ~10 seconds. Thanks for you patch, you saved my every launch time :)

[pierresigwalt]

Hi,

Thank you for the work! Is there any change it will work on Ubuntu 22?

Thank you!

[slonopotamus]

Ubuntu 22.04 ships with fixed version of glibc and doesn't need patching.

[maxjust]

@slonopotamus Hi, ubuntu 21.10 a few days ago release update of libc6 which have newer version but have no dso patches yet. And now packages from you software channel wont be installed. Can you please bump version of you lib for 21.10? Thanks in advance

[maxjust]

for someone else if need a command to downgrade to @slonopotamus libs are: sudo apt install libc6=2.34-0ubuntu3dso3 libc6:i386=2.34-0ubuntu3dso3 libc6:i386=2.34-0ubuntu3dso3 libc6-dbg=2.34-0ubuntu3dso3 libc6-dbg:i386=2.34-0ubuntu3dso3 libc6-dev=2.34-0ubuntu3dso3 libc6-i386=2.34-0ubuntu3dso3 libc6-x32=2.34-0ubuntu3dso3 libc-dev-bin=2.34-0ubuntu3dso3

sometimes before need dpkg -r glibc-doc

[slonopotamus]

I'll upload updated package for 21.10 in several days.

[ImmortalEmperor]

heads up focal just bumped version: libc6: Installed: 2.31-0ubuntu9.7 Candidate: 2.31-0ubuntu9.7 Version table: *** 2.31-0ubuntu9.7 500 500 http://au.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages 100 /var/lib/dpkg/status 2.31-0ubuntu9.2dso2 500 500 http://ppa.launchpad.net/slonopotamus/glibc-dso/ubuntu focal/main amd64 Packages 2.31-0ubuntu9 500 500 http://au.archive.ubuntu.com/ubuntu focal/main amd64 Packages

[slonopotamus]

2.31-0ubuntu9.7dso1 is building...

UPD: And done.

[mika314]

OMG... it just works. 4 min 6 sec down to 14 seconds.

[RicardoEPRodrigues]

Heads up focal just bumped version: 2.31-0ubuntu9.9

[slonopotamus]

Heads up focal just bumped version: 2.31-0ubuntu9.9

Updated build is in process: https://launchpad.net/~slonopotamus/+archive/ubuntu/glibc-dso/+sourcepub/13624185/+listing-archive-extra

[poboy975]

Hello, question. I've running Linux Mint 20.3, and I've built Unreal Engine Version: 5.0.1-0+++UE5+Release-5.0. I currently have (Ubuntu GLIBC 2.31-0ubuntu9.9) 2.31 installed. Will adding this to my system increase the Editor load times? Will it break my system?

[slonopotamus]

I have no idea about Linux Mint compatibility with Ubuntu. Please refer to your distro documentation.

[mika314]

@poboy975 I used this patch on Pop OS! 20.04 and currently using it on 22.04. I do not have any issues. I assume it should be fine on Mint as well.

[poboy975]

@mika314 Ok, thanks. I'll give it a try.

[Kendarr443]

Hey, it works on mint 20.3!, my god it went down from like 6 minutes to 10 seconds I guess, Haven't timed it, but it's a lot faster!. Thank you, @slonopotamus

[RicardoEPRodrigues]

@poboy975 I used this patch on Pop OS! 20.04 and currently using it on 22.04. I do not have any issues. I assume it should be fine on Mint as well.

You don't need the patch on 22.04. It already has a newer version with the improvements.

[siuleongyu]

@slonopotamus Thanks a lot for your magic fix. The launching time of UnrealEngine editor reduces from a few minutes to less than a minute on my Ubuntu 20.04. However, when using VSCode for debugging, the launching of UE editor goes back very slow. Any suggestions are welcome.

[gugananadev]

@slonopotamus Thanks a lot for your magic fix. The launching time of UnrealEngine editor reduces from a few minutes to less than a minute on my Ubuntu 20.04. However, when using VSCode for debugging, the launching of UE editor goes back very slow. Any suggestions are welcome.

Hey @siuleongyu - if you're still wondering you can tweak the launch.json parameters on VSCode (for your project) and add the environment variable there for your preferred launch config. Should look something like this:

`{

"configurations": [
    {
        "name": "Launch Unreal",
        "type": "cppdbg",
        "request": "launch",
        "program": "<pathToUnreal>",
        "args": ["<pathToProject>"],
        "environment": [{ "name": "GLIBC_TUNABLES", "value": "glibc.rtld.dynamic_sort=2" }],
        "miDebuggerPath": "/usr/bin/gdb",
        "cwd": "${workspaceFolder}"
      }
]

}`

Keep in mind that running GenerateProjectFiles.sh will likely reset the changes you do to launch.json. Hope this works for you has it has worked for me!

[DevtekDev]

Applied this a few weeks ago on Mint 20.3 - it fixed the Unreal launch which is awesome but I got a black screen on bootup so had to roll back with timeshift.

Wondering if it's a driver compatibility issue because I have dual xeon server motherboard with an nVidia 2060 stuffed into it so have to use nVidia server drivers.

Attempting it again now I have a bit more time to fxck about, I'll keep you posted.

[trung-nws]

focal bumped version: 2.31-0ubuntu9.12 can you rebuild? @slonopotamus

[slonopotamus]

Possibly next week. But you should strongly consider moving to ubuntu 22.04 where patching is no longer needed)

[LinnChan]

If use newer version glibc on ubuntu20.04, likeme installed libc6 2.31-0ubuntu9.14, just downgrade to @slonopotamus built version 2.31-0ubuntu9.9dso1. It's working fine for me. Thanks you @slonopotamus for your solution.

sudo apt update -y &&\
sudo apt install -y software-properties-common &&\
sudo add-apt-repository -y ppa:slonopotamus/glibc-dso &&\
sudo apt update -y && \
sudo apt install libc6=2.31-0ubuntu9.9dso1  libc6:i386=2.31-0ubuntu9.9dso1  libc6:i386=2.31-0ubuntu9.9dso1  libc6-dbg=2.31-0ubuntu9.9dso1  libc6-dbg:i386=2.31-0ubuntu9.9dso1  libc6-dev=2.31-0ubuntu9.9dso1  libc6-i386=2.31-0ubuntu9.9dso1  libc6-x32=2.31-0ubuntu9.9dso1  libc-dev-bin=2.31-0ubuntu9.9dso1 && \
export GLIBC_TUNABLES=glibc.rtld.dynamic_sort=2 

[ryansburgoyne]

Thank you @LinnChan! I am stuck on 20.04 for now and this helped me fix the slow editor startup! (And thank you @slonopotamus for the original patch!)