TF2: Huge, lasting drop in framerate

[jorgenpt]

Issue transferred from ValveSoftware/steam-for-linux#140 @FliPPeh posted at 2012-12-20T17:08:11Z:

Occasionally, sometimes instantly on round start / game join, sometimes halfway into a round, my framerate drops from mostly (see issue #138) playable levels into unplayable 10 - 15 frames per second.

The game will stay that way until I force a reinitialization of the graphics by changing any advanced graphics setting (e.g. Motion Blur, Filtering) or completely restart the game (stuttering persists even in main menu resulting in delayed button clicks and hover states).

One thing I now know is that it's not an out of memory condition, TF2's memory consumption stayed the same pre- and post bug and was overall well below my total RAM (2.4 GiB usage, 8 GiB avail). It also is not connected to the 100%-CPU bug that was fixed yesterday, because it since happened again.

This bug makes the game pretty much unplayable for me, sadly.

Processor Information:
    Vendor:  GenuineIntel
    Speed: 3301 Mhz
    4 logical processors
    4 physical processors
    HyperThreading:  Unsupported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Unsupported
    SSE41:  Supported
    SSE42:  Supported

Network Information:
    Network Speed:  

Operating System Version:
    Ubuntu 12.10 (64 bit) [Unity]
    Kernel Name:  Linux
    Kernel Version:  3.5.0-21-generic
    X Server vendor:  The X.Org Foundation
    X Server release:  11300000

Video Card:
    Driver:  NVIDIA Corporation GeForce GTX 560 Ti/PCIe/SSE2

    Driver Version:  4.3.0 NVIDIA 310.14
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0x1200
    Number of Monitors:  2
    Number of Logical Video Cards:  1
    Primary Display Resolution:  1920 x 1080
    Desktop Resolution: 3600 x 1080
    Primary Display Size: 20,51" x 11,54"  (23,50" diag)
                                            52,1cm x 29,3cm  (59,7cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 1023 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
    Audio device: 20K2

Memory:
    RAM:  7956 Mb

Miscellaneous:
    UI Language:  English
    LANG:  de_DE.UTF-8
    Microphone:  Not set
    Total Hard Disk Space Available:  679553 Mb
    Largest Free Hard Disk Block:  184560 Mb

Installed software:

Recent Failure Reports:

[benaryorg]

I can confirm this bug, but only on map-change here.

Specs:

Processor Information:
    CPU Vendor:  GenuineIntel
    CPU Family:  0x6
    CPU Model:  0x5e
    CPU Stepping:  0x3
    CPU Type:  0x0
    Speed:  2601 Mhz
    8 logical processors
    4 physical processors
    HyperThreading:  Supported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Unsupported
    SSE41:  Supported
    SSE42:  Supported
    AES:  Supported
    AVX:  Supported
    CMPXCHG16B:  Supported
    LAHF/SAHF:  Supported
    PrefetchW:  Unsupported

Operating System Version:
    "Gentoo Base System release 2.2" (64 bit)
    Kernel Name:  Linux
    Kernel Version:  4.7.2-gentoo-benaryorg
    X Server Vendor:  The X.Org Foundation
    X Server Release:  11804000
    X Window Manager:  Enlightenment
    Steam Runtime Version:  steam-runtime-beta-release_2016-06-15

Video Card:
    Driver:  NVIDIA Corporation GeForce GTX 970M/PCIe/SSE2

    Driver Version:  4.5.0 NVIDIA 370.23
    OpenGL Version: 4.5
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0x13d8
    Revision Not Detected
    Number of Monitors:  1
    Number of Video Cards Not Detected
    Primary Display Resolution:  1920 x 1080
    Desktop Resolution: 1920 x 1080
    Primary Display Size: 20.04" x 11.26"  (22.95" diag)
                                            50.9cm x 28.6cm  (58.3cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 3072 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Memory:
    RAM:  16000 Mb

[kisak-valve]

Hello, Is anyone is still experiencing this issue with the current TF2 client?

[ZeXx86]

Hello,

since I have switched to a new GPU with a bigger memory there are zero issues. But with NVidia 8800GTS it was happening. I guess that others with the same issue are running new computers as well so although the issue can be still there, it is not a problem.

[db47h]

Still experiencing this as well. Ubuntu 16.04, Nvidia GeForce GTX 560 TI, latest proprietary drivers. It happened only once while in game, but it happens quite often right from game start. The only way to fix it reliably is to restart the X server.

[volca02]

Hi, I believe I am experiencing this bug. I seem to be able to reproduce it easily by doing:

map jump_sketchy2_rc1
map plr_hightower

Commands in console, second one right after the first is loaded.

I see a very heavy HDD usage after the second command, and after it finally ends using HDD, the FPS stays around 10 till I exit the game entirelly. At the game's exit the game segfaults, I see the segfaults in the coredumpctl list output, but the coredump can't be found, sadly.

           PID: 23771 (hl2_linux)
           UID: 1000 (volca)
           GID: 1000 (volca)
        Signal: 11 (SEGV)
     Timestamp: Wed 2016-12-28 15:34:57 CET (2min 39s ago)
  Command Line: /home/volca/.local/share/Steam/SteamApps/common/Team Fortress 2/hl2_linux -game tf -steam -novid
    Executable: /home/volca/.local/share/Steam/SteamApps/common/Team Fortress 2/hl2_linux
 Control Group: /user.slice/user-1000.slice/session-c1.scope
          Unit: session-c1.scope
         Slice: user-1000.slice
       Session: c1
     Owner UID: 1000 (volca)
       Boot ID: f6119d1c8f5f49dbbb5b7d3f3878ea08
    Machine ID: c9f579222d2d594e1fc55b9e000005c5
      Hostname: volca
       Storage: none
       Message: Process 23771 (hl2_linux) of user 1000 dumped core.

                Stack trace of thread 23771:
                #0  0x00000000ffffffff n/a (n/a)

Coredump entry has no core attached (neither internally in the journal nor externally on disk).

Also, using perf I see MatQueue0 process/thread being on top of the CPU load, but with mere 14-15% of CPU. (when I use mat_queue_mode 2).

[volca02]

@jorgenpt I can confirm that changing Advanced Graphics Settings (I changed AA for the test) mitigates the issue when it started happening.

[LongJohnPiano]

@jorgenpt I can confirm that changing Advanced Graphics Settings (I changed AA for the test) removes the issue.

I wonder if there's a way to run apitrace¹ and compare traces with this problem, and without this problem.

¹ http://apitrace.github.io Trace OpenGL API calls

[volca02]

I have a strong feeling it is related in some way to the mat_queue:

mat_queue_report 1
con_filter_text_out "CMaterialSystem::Unlock"

After first map command:

CMaterialSystem::ThreadRelease: 9.84ms = Release:9.62ms + Acquire:0.22ms
*CMaterialSystem::Lock: 9.89ms
CMaterialSystem::ThreadRelease: 1.78ms = Release:0.12ms + Acquire:1.66ms
*CMaterialSystem::Lock: 1.83ms
CMaterialSystem::ThreadAcquire: 1.53ms
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
CMaterialSystem::ThreadRelease: 3.62ms = Release:0.37ms + Acquire:3.25ms
*CMaterialSystem::Lock: 3.68ms
CMaterialSystem::ThreadAcquire: 1.13ms
CMaterialSystem::ThreadAcquire: 2.26ms
CMaterialSystem::ThreadAcquire: 1.93ms
CMaterialSystem::ThreadAcquire: 2.17ms

After second map command:

CMaterialSystem::ThreadRelease: 160.30ms = Release:79.25ms + Acquire:81.06ms
*CMaterialSystem::Lock: 221.25ms

So there clearly is a change in the behavior of that system.

Next I do the advanced graphics settings change, it will release some parts of that system and reinitialize, most probably:

CMaterialSystem::ThreadRelease: 163.98ms = Release:81.86ms + Acquire:82.12ms
Changing resolutions from (1920, 1200) -> (1920, 1200)
Unable to remove /home/volca/.local/share/Steam/SteamApps/common/Team Fortress 2/tf/textwindow_temp.html!
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
--- Missing Vgui material vgui/..\vgui\maps\menu_thumb_Missing
CMaterialSystem::ThreadRelease: 24.08ms = Release:0.46ms + Acquire:23.62ms
*CMaterialSystem::Lock: 24.16ms
CMaterialSystem::ThreadRelease: 14.49ms = Release:0.35ms + Acquire:14.13ms
*CMaterialSystem::Lock: 14.53ms

[volca02]

Another interesting thing:

mat_shadercount

After first map load: Num Pixel Shaders = 8043 Vertex Shaders=3088

After second map load: Num Pixel Shaders = 0 Vertex Shaders=0

I'll see if I can use apitrace to see if it is related to OpenGL

[kisak-valve]

The majority of video cards reported in this issue report are nVidia cards with around 1GB of vram. I suspect that some VRAM in the hotpath for rendering is getting allocated to system ram.

While the game is suffering from reduced framerate, can someone open nvidia-settings and check the PCIe Bandwidth Utilization. In most systems this does not exceed about 10% unless something has gone awry or too much is being asked of the hardware. High utilization is confirmation that VRAM has been allocated to system ram.

[volca02]

@kisak-valve I can confirm your theory. I have 2 GB GPU (GTX 770) and I see exactly the kind of behavior you describe. The VRAM fills up during the second map loading operation, it frees up later after it is done loading, but the 100% PCIe Bandwith Utilization stays during the gameplay.

Edit: Screenshot: http://imgur.com/a/EfSsP

[raku-cat]

I also have this issue on arch linux

Computer Information:
    Manufacturer:  Unknown
    Model:  Unknown
    Form Factor: Desktop
    No Touch Input Detected

Processor Information:
    CPU Vendor:  AuthenticAMD
    CPU Brand:  AMD FX(tm)-6300 Six-Core Processor             
    CPU Family:  0x15
    CPU Model:  0x2
    CPU Stepping:  0x0
    CPU Type:  0x0
    Speed:  3500 Mhz
    6 logical processors
    6 physical processors
    HyperThreading:  Unsupported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Supported
    SSE41:  Supported
    SSE42:  Supported
    AES:  Supported
    AVX:  Supported
    CMPXCHG16B:  Supported
    LAHF/SAHF:  Supported
    PrefetchW:  Unsupported

Network Information:
    Network Speed:  Don't Know

Operating System Version:
    "Arch Linux" (64 bit)
    Kernel Name:  Linux
    Kernel Version:  4.9.11-1-lqx
    X Server Vendor:  The X.Org Foundation
    X Server Release:  11902000
    X Window Manager:  Xfwm4
    Steam Runtime Version:  <Runtime disabled>

Video Card:
    Driver:  NVIDIA Corporation GeForce GTX 950/PCIe/SSE2

    Driver Version:  4.5.0 NVIDIA 378.13
    OpenGL Version: 4.5
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0x1402
    Revision Not Detected
    Number of Monitors:  2
    Number of Logical Video Cards:  1
    Primary Display Resolution:  1920 x 1080
    Desktop Resolution: 3520 x 1080
    Primary Display Size: 18.90" x 10.63"  (21.65" diag)
                                            48.0cm x 27.0cm  (55.0cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 2048 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
    Audio device: Realtek ALC1150

Memory:
    RAM:  7932 Mb

Miscellaneous:
    UI Language:  English
    LANG:  en_US.utf8
    Microphone:  Don't know
    Steam Controller Cable and Base:  No Receiver
    Total Hard Disk Space Available:  938770 Mb
    Largest Free Hard Disk Block:  450463 Mb
    VR Headset: None detected

Recent Failure Reports:
    Wed Mar  8 21:18:44 2017 GMT: file ''/tmp/dumps/crash_20170308151842_24.dmp'', upload yes: ''CrashID=bp-75e06494-e0e8-4881-b7ff-fdefd2170308''
    Thu Mar  9 21:15:22 2017 GMT: file ''/tmp/dumps/crash_20170309151520_25.dmp'', upload yes: ''CrashID=bp-a2fd4942-a2c1-45e0-a5cb-a66122170309''
    Fri Mar 10 21:11:01 2017 GMT: file ''/tmp/dumps/assert_20170310151058_23.dmp'', upload yes: ''CrashID=bp-6de12bb0-5c8c-4bb1-b291-eaeb82170310''
    Sat Mar 11 06:23:00 2017 GMT: file ''/tmp/dumps/assert_20170310151049_1.dmp'', upload yes: ''CrashID=bp-40390b48-5efc-4593-9970-1ad012170310''
    Sat Mar 11 06:23:03 2017 GMT: file ''/tmp/dumps/crash_20170311002302_2.dmp'', upload yes: ''Discarded=1''
    Sat Mar 11 06:23:47 2017 GMT: file ''/tmp/dumps/assert_20170311002316_1.dmp'', upload yes: ''CrashID=bp-262172c0-2529-48c8-ba6d-ebb282170310''

This happens when the map changes on a server, or if I play one map on a server, disconnect, and then go on another server.

[raku-cat]

The game has recently started becoming randomly laggy in general, random framedrops and high resource usage, I can't confirm if it's the vram issue because I hardlock, when everything is back tf2 is crashed and usually steam, as well as other unrelated applications such as discord and firefox.

[ElectricTriangle]

@raku-cat That's not related to this bug in particular, but I have experienced it as well. It's related to the recent toolchain update. If you want it addressed you should open a new bug report if it hasn't been reported already.

[jayhendren]

I have this issue on Linux as well. GTX 950 here. I can also confirm @kisak-valve's hypothesis of high PCIe bandwidth utilization. The workaround of reloading video options (by changing some settings and hitting "apply") seems to work for me.

I would also like to add that for some reason I almost always see this issue when playing valve servers and then switching to community servers. For some reason I don't see this issue if I only play community or only play valve servers during one session.

[Tiagoquix]

@kisak-valve I think this can be closed considering OS and game updates (especially the Vulkan render for Linux now).

[Nocty1501]

I still have this problem on a GTX 970, running the Linux client on Arch. It happens most commonly when tabbing out of and back into the game while connecting to a server.