Random pausing/hanging of CS2

System information

System info: https://gist.github.com/dancrn/f1a6a0d5b3e890ab5131470f0ac013c4 system up to date. using @kisak-valve's mesa PPA. output from vulkaninfo:

VkPhysicalDeviceVulkan12Properties:
-----------------------------------
    driverID                                             = DRIVER_ID_MESA_RADV
    driverName                                           = radv
    driverInfo                                           = Mesa 23.2.1 - kisak-mesa PPA

et al.

The issue

There are numerous >30s temporary hangs when opening CS2, loading a map, viewing the inventory, interacting with some menus, etc. To be clear, the pauses don't happen randomly, it's in fact 100% reliable where they occur, but it's at seemingly random areas of the game.

Principally this is only annoying - the game always becomes responsive eventually, however, all servers will drop connections to my client - presumably because of a lack of timely response.

During this time, I can see that one thread essentially spins at 100%, filling up a bunch of RAM - up to 40gb (!!) - before freeing it all up and then continuing to behave as normal.

Playing offline is no different (it takes around 4 minutes to load a map), but once this has passed, the game runs perfectly normally, at framerates >300fps without issue. No pauses or slowdowns occur in game.

I don't usually run in a window, but I've attached a few screenshots to explain what I mean;

Hanging after loading.

If you look on the right side of the capture, you'll see one thread at 100%, then beneath that, the memory usage graph, which shows a sawtooth pattern in usage. The spikes are all very similarly sizes. The left most one is from where I'd started the game before but killed it before it got to the menu.

Once in the menu;

Screenshot from 2023-10-13 01-00-52

The game hits the 120fps limiter without issue. Once it's done it's pausing thing, we're fine.

Loading a map.

Screenshot from 2023-10-13 01-02-48

At this point, the game is unresponsive again. You can see two memory usage spikes. The first occurred during the initial map load, the second shortly after team selection.

In game

Screenshot from 2023-10-13 01-07-20

Again, once in game, it's totally fine. I haven't been able to cause it to hang during normal play. The memory load spikes are far on the left of the graph because I hadn't actually explored inferno yet..

It's almost as if there is code like this in a handful of places:

{
  std::vector<uint32_t> x;
  for (uint32_t i=0; i<UINT32_MAX; i++) {
    x.push_back(i);
  }
}

// continue loading game..

Things I've tried

If I don't write an outcome, it's because there was no change.

ran with -vulkan
ran at different resolutions
ran with -nojoy (not even sure this does anything any more..)
ran with a stock linux kernel (I'm running 6.5.0, with the low-latency scheduler)
ran cs2 under Wayland (as opposed to X)
used kisak's mesa apt PPA (was Ubuntu 20.04 before)
updated to Ubuntu 22.04 (which is terrible btw)

I'd had some success with running the closed beta with proton (and one of GloriousEggroll's wine builds), of course with out online play. This didn't have the pausing issue that the native build does.

Steps for reproducing this issue:

Open CS2
After opening video (which seems to have no delays), a delay of around 30 seconds to 2 minutes happens before the main menu is loaded
Opening inventory, loading a map, joining a server all cause multiple delays. (and in the latter case, the server kicks off the client eventually for, presumably, lack of timely response, before game can be played).

@kisak-valve any updates w.r.t this? i still have this issue :(

I can see high memory usage with anv driver and Mesa 23.2.1 on Fedora 39 when 8GB RAM and 9GB swap in zram are almost fully used.

@jaccc i'm not sure if that is related. my high memory usage is transient.

i'm a bit sad there's not much feedback on this. i know there's a trillion linux combinations and what not, but in my experience, issues like these tend to indicate that something may be wrong generally, and this is just the first symptom of it..! i'm happy to help too, if you tell me where to look.

i'll note that the behavior has changed a little since first opening this issue. the main menu is usable much quicker now (although still ~20 seconds). there's still a thread spinning away doing something, but i can see that memory usage is noticeably down, so whatever is taking its time during start up seems to have been optimised a bit :upside_down_face:

it looks like someone has added in some.. "current state" readouts at the bottom left of the game window. at least i hope so. here's what i see when opening the game initially:

it's worth noting that the inventory is now accessible instantly. that's cool! this used to have a similar delay, but that appears to have gone away entirely.

unfortunately, the game otherwise still has the same issues as before. here's me attempting to load up a deathmatch game:

also worth noting that the renderer, as before, has frozen here (and in all the subsequent screenshots). the audio system is still working: i can hear the background mirage ambience (i think?), but the next main menu music track didn't play/wasn't queued once the first one had finished.

once that freeze passes (counting two memory use spikes), we begin loading the map:

which incurs another 2 memory use spikes.

then we have a handful of frames of interactivity on the team selection menu, before another freeze for one memory spike.

then we're kicked back to the main menu, for taking to long to reply to the server.

here's my tiny memory usage chart for this period:

the launch of the game is off to the left of the chart, which can no longer be seen. the stable memory usage on the right is the main menu (which is fully interactive). note that the usage is a fraction of 128gb, so.. that's a lot of bytes.

this issue still exists, exactly the same as described.

good news, it now crashes, as opposed to becoming eventually playable.

here's the output from the terminal when this crash happens. perhaps a coincidence, but it looks like there are a bunch of 0-length allocations being done.

$ ./cs2.sh 
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libengine2.so, got 0x55bd072e3960
Using breakpad crash handler
[S_API] SteamAPI_Init(): Loaded '/home/dan/.local/share/Steam/linux64/steamclient.so' OK.
Setting breakpad minidump AppID = 730
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
05/25 14:56:46 minidumps folder is set to /tmp/dumps
05/25 14:56:46 Init: Installing breakpad exception handler for appid(730)/version(8926243)/tid(838663)
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
SteamInternal_SetMinidumpSteamID:  Caching Steam ID:  76561197970284625 [API loaded yes]
SteamInternal_SetMinidumpSteamID:  Setting Steam ID:  76561197970284625
Setting breakpad minidump AppID = 2347771
Loaded libSDL3.so.0, got 0x55bd07423960
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libtier0.so, got 0x55bd072c50e0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libfilesystem_stdio.so, got 0x55bd07484d60
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libinputsystem.so, got 0x55bd074972c0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/liblocalize.so, got 0x55bd074d2080
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/librendersystemvulkan.so, got 0x55bd074c9390
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libresourcesystem.so, got 0x55bd074ebfc0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libschemasystem.so, got 0x55bd074cba90
Fossilize INFO: Overriding serialization path: "/data/dan/SteamLibrary/steamapps/shadercache/730/fozpipelinesv6/steamapprun_pipeline_cache".
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libmaterialsystem2.so, got 0x55bd0b543620
Loaded libpangoft2-1.0.so, got (nil)
 failed to dlopen "libpangoft2-1.0.so" error=/lib/x86_64-linux-gnu/libpangoft2-1.0.so: undefined symbol: pango_coverage_get_type
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libpanorama_text_pango.so, got 0x55bd0d6ed140
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 39: unknown element "fontpattern"
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 40: unknown element "fontpattern"
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 41: unknown element "fontpattern"
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 42: unknown element "fontpattern"
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 43: unknown element "fontpattern"
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/fonts.conf", line 86: saw string, expected range
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/../../../core/panorama/fonts/conf.d/41-repl-os-win.conf", line 148: Having multiple values in <test> isn't supported and may not work as expected
Fontconfig warning: "/data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/panorama/fonts/../../../core/panorama/fonts/conf.d/41-repl-os-win.conf", line 160: Having multiple values in <test> isn't supported and may not work as expected
Loaded libfreetype.so.6, got 0x55bd0b2199b0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libpanorama_text_pango.so, got 0x55bd0d6ed140
Loaded libfontconfig.so.1, got 0x55bd0b0b2650
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libmeshsystem.so, got 0x55bd0acf0da0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libworldrenderer.so, got 0x55bd0e8fb870
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libpulse_system.so, got 0x55bd0b031310
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libvscript.so, got 0x55bd0a19b4f0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libnetworksystem.so, got 0x55bd0d67c660
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libanimationsystem.so, got 0x55bd0b1a3db0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libvphysics2.so, got 0x55bd0f580420
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libsoundsystem.so, got 0x55bd0e7046e0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libsteamaudio.so, got 0x55bd0bbcff20
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libscenesystem.so, got 0x55bd0ddf8ff0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libparticles.so, got 0x55bd0b755d60
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libpanoramauiclient.so, got 0x55bd0aa9e880
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libv8system.so, got 0x55bd0b454520
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libpanorama.so, got 0x55bd0b932fb0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/bin/linuxsteamrt64/libserver.so, got 0x55bd0f98e0c0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/bin/linuxsteamrt64/libclient.so, got 0x55bd0a457da0
Loaded libmpg123.so.0, got 0x55bd0baf9d20
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libcairo.so, got 0x55bd100c88c0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libengine2.so, got 0x55bd072e3960
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/bin/linuxsteamrt64/libhost.so, got 0x55bd0f699450
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/bin/linuxsteamrt64/libscenefilecache.so, got 0x55bd09f04eb0
Loaded /data/dan/SteamLibrary/steamapps/common/Counter-Strike Global Offensive/game/csgo/bin/linuxsteamrt64/libmatchmaking.so, got 0x55bd09dba8f0
***** OUT OF MEMORY! attempted allocation size: 0 ****

this has been an issue for me since october last year. if this isn't being worked on/investigated, that's fine, but it's quite frustrating not knowing whether my efforts to help are just me pissing into the wind or not.

Screenshot from 2024-06-12 21-35-46

latest build's failure. at least we're doing dialogues now.

the real question is why would so many descriptor pools get allocated..

more error information: Screenshot from 2024-07-07 12-10-45

also worth noting that the game was actually in the process of transitioning to the menu - you can see it poking through the transparency of the CS2 splash:

Screenshot from 2024-07-07 12-11-10

the long spin at the start up is still present.

Have exactly the same behavior, but only as of last week on Debian stable, with the packaged 525 drivers running Wayland. Up until that point, CS2 worked without issue. Tried on Arch as well with the packaged 555 drivers, same symptoms. 30+ second hangs in the menus. If I do get into a game, no more hangs until either alt-tab or just tab menu. Eventually game server will disconnect, I suppose due to a timeout. NB it doesn't crash with OOM for me, just hangs - no memory leaks or CPU spike.

AMD 5950X, 32GB Ram, 4090

Arch. AMD 5900X, 64GB RAM, AMD 6900 XT. Similar symptoms.

ok, so i have another machine i can play CS2 on now. finally, after 10 months i can actually play the game. :/

it's a bit lower spec'd than my main machine, but honestly at this point i don't care. it's still linux, still ubuntu.

CS2 works
it has the same issue!

specifically, the hanging and pausing on this machine is the same as on my main machine. the difference being is that it doesn't last long enough (most of the time) to get me disconnected, or run out of.. memory.

but hey, i have the console output now, and there's even some mini profiles. yay. lets see:

[VProf]                                                       All frames         Active frames   
[VProf]                                                         Avg    P99        N    Avg    P99
[VProf] ---------------------------------------------------- ------ ------   ------ ------ ------
[VProf]                                           FrameTotal   7.45  12.12    35136   7.45  12.12
[VProf]                                     Client Rendering   4.16   6.32    35136   4.16   6.32
[VProf]                                       Frame Boundary   2.89   4.85    35136   2.89   4.85
[VProf]                                    Client Simulation   2.04   3.65    35136   2.04   3.65
[VProf]                                           Prediction   0.62   2.30    35136   0.62   2.30
[VProf]                                           PanoramaUI   0.52   1.38    35136   0.52   1.38
[VProf]                                                  HUD   0.43   1.37    35136   0.43   1.37
[VProf]                                          Unaccounted   0.28   1.21    35136   0.28   1.21
[VProf]                                     Client_Animation   0.27   0.77    16644   0.57   0.80
[VProf]                                           Networking   0.27   1.67    16650   0.57   2.28
[VProf]                                        Interpolation   0.26   0.45    35136   0.26   0.45
[VProf]                                       Client Physics   0.18   0.70    35136   0.18   0.70
[VProf] LatchAndSaveLastSimulationValuesForInterpolationList   0.14   0.75    16644   0.29   1.23
[VProf]                                  Client Input/Output   0.12   0.27    35136   0.12   0.27
[VProf]                                           FileSystem   0.04   0.00       26  54.681043.73
[VProf]                                           Javascript   0.01   0.04      735   0.31   1.01
[VProf]                                                Magic   0.00   0.17      892   0.12   0.57
[VProf]                                     Econ Description   0.00   0.00       26   0.10   0.52

wait, hang on, the enormous p99 in a profile of the renderer - the real time renderer - is in the wha-

Excessive frame time of 1398.06ms clamped.  Acting as if the frame only took 100.00ms
1389.04ms Client Simulation
1385.35ms FileSystem
   5.09ms Client Rendering
   4.86ms Frame Boundary
   1.66ms PanoramaUI
   1.43ms Prediction

:facepalm: ugh. you have to be kidding me.

which might be explained by one difference between these two machines: this linux set up is using ext4, whereas my other set up uses BTRFS. i mean, just why the file system even on this profile is a question i genuinely don't want to know the answer to, but i'm sure that was an.. interesting code review.

look, @kisak-valve, i understand you have a thankless job, dealing with angry linux nerd rage comments from all over the steam catalogue, but i've been trying to help as best i can here. i work in low-latency/realtime software myself, and i truly get how linux may be.. more challenging in some areas, to provide good support for. well, compared to what is basically a mono-platform with windows. i get that there's "secret sauce" or whatever in some parts of the engine, and that valve wouldn't want to inadvertently leak some details that perhaps rely on being kept secret. that said, this radio silence is as needless as it is infuriating.

almost anything as a response would have been better. even if it was the response of "we don't have the time or resources to look into this right now, sorry".

i think this will be my last update.

This seems to be largely resolved in the recent CS2 patch, @dancrn

ValveSoftware / csgo-osx-linux