CSteam3Client::RunFrame peaks causing lag, and peaks even more when re-connecting the internet yielding the entire game. Something is wrong with PersonaStateChange SteamAPI callback in Portal 2

[karl-police]

Update: This issue was analyzed. At the bottom of this post you will find more detail that points very close to the exact cause, maybe even the exact cause.

 

I was recommended to come here to Source 1 issues by the Steam Support. The issue here doesn't really have a exact step to reproduce, like other bug reports.

OS: Windows 10 64Bit - Version 1803 (Build 17134.950) Game the issue occurs: From all games I know together with any Source game, is Portal 2 the only game that does that behaviour. Graphics card: NVIDIA GeForce GTX 960M Driver version: (The issue existed doesn't matter which one I updated to, not sure about older versions) Current version is, 431.60 GeForce Game Ready Driver RAM: 16 GB and (from Steam systeminfo RAM: 16282 MB) Audio: Audiogerät: Lautsprecher (Conexant SmartAudio) CPU: Intel i7-6700HQ

Sadly, I'm not really sure which graphic card Portal 2 is using, if there would be a console command for that, it would be cool. Nvidia control panel is not enough for me. From Steam sysinfo thing, it gets recognized as a Direct-X card.

Description of issue:

A random second where the "sv" goes on red, causing the game to lag for like 1 second, during that second any sound or something could turn into a sound crackling at the end or something. Also it causes inaccurate aim during that frame drop, kinda.

 

Research description:

So what I did to find traces of "what the issue could be". I've done the following. Well, actually I tested with cl_showfps 1, there I've noticed, "Hey, it goes down for a short one second". (During that second a kinda lag appears and should there be sound it could be that a small sound crackling appears. On the showfps I noticed that it seems to go up.

Then I decided to disable vsync or try each one, together with multicore setting, that didn't work either.

After that I used net_graph 4 and noticed that the "sv" goes on red. (I was not able to get a proper picture of it gonig down to red so I just set fps_max to 2, to show what I mean with it going red. That empty gap btw. is me alt tabbing before or something.

It happens in any map, even the story mode maps. And main menu too. The sound crackling is like a "pkk" or something, like audio spitting or something or a stone, not sure.

I can be open for any video requests, might update this with one, but better would be one by request. But it can cause inaccurate results because a recording program can impact the performance of the game then, but I think that shouldn't be a deal I don't know.

 

Also setting fpx_max to 60 didn't fix the issue. When I disabled vsync I might got like 200 or 150FPS but the same amount of FPS drop still exist.

Maybe it is related with the GPU Temperature Limit but I don't know. I've closed Google Chrome that didn't fix it either, so the issue is not depended on RAM or CPU probably, not sure what exactly it is. All I know is that it is different to all other games.

Switching between motion blur on and off didn't work either. The issue occurs more when I move the player together with the mouse than just standing. Verifing game files didn't work, don't think even reinstalling will work, the issue has to be somewhere else.

More can be found in the comments

  Where does it occur?

Basically anything related with a map. Even the main menu. Not sure if after a several amount of time it seems to better, but if it does sound crackling or atleast a small lag can still occur. That lag can be disturbing and cause inaccurate movement.

[karl-police]

For a clear description I tried to fix it again. If I use cl_ignorepackets the red "sv" doesn't drop. It seems to be a packet issue. -nojoy didn't fix it. The packet issue is only ingame or in the main menu and since the main menu is also loading a map, it could be only when in a map. Even if I lower the resolution it still appears.

[RedSkittleFox]

I have the issue when I play on my main account, it dissapears when I switch accounts. It may have something to do with the config saved in steam cloud.

[karl-police]

Another thing to note is that, this issue happens when playing singleplayer.

So playing coop atleast when you're p-body this issue doesn't really happen.

[karl-police]

I went into Airplane mode which disables all internet and for some reason it fixed the issue. But only when disconnected from the Internet.

Means something when I'm connected to the internet is causing this issue. Weirdly it is only causing it in Portal 2 and not any other application.

Another thing is that if connected

host_ShowIPCCallCount: 1480 IPC calls in the past [49 frames, 56 ticks] Avg: [30.20/frame, 26.43/tick]

I can get this and not sure but maybe each time it calls I lag, unsure. But if I'm disconnected it's always on 0.

As soon as I connected back to the internet the IPC call was over 11'000, unsure why but the game froze a while.

-noip does not fix the issue either

[karl-police]

Each time when the "sv" is about to drop

Steam is on full bar

[karl-police]

Can we get a convar to disable this?

[UGC] File maps\workshop\776231328129491909\1568492038.bsp already in sync on client. (Duration: 67.231285 seconds)

I found this in workshop_log.txt

@kisak-valve do you see updated I do to the thread?

[karl-police]

I've heard that it is going to get fixed, once the team has time for it.

[Yetoo1]

@karl-police Where did you get this information from? I'm interested in any official activity regarding this issue tracker.

[karl-police]

@karl-police Where did you get this information from? I'm interested in any official activity regarding this issue tracker.

I asked them.

[karl-police]

Issue still persists, I guess. Was the last update ment to fix it? Does a file have to be deleted maybe?

[karl-police]

Issue seems still to persist, after new update.

[karl-police]

It was never fixed, not even on another device. The issue isn't related to GPU Temperature Limit though.

[karl-police]

Each time when the "sv" is about to drop

Steam is on full bar

this command was +showbudget btw.

[karl-police]

For a clear description I tried to fix it again. If I use cl_ignorepackets the red "sv" doesn't drop. It seems to be a packet issue. -nojoy didn't fix it. The packet issue is only ingame or in the main menu and since the main menu is also loading a map, it could be only when in a map. Even if I lower the resolution it still appears.

I think cl_ignorepackets just freezed it.

but turning off the internet, doesn't cause this from happening, it is only if you're connected at the internet

[karl-police]

"An error occurred while attempting to download a file from the UGC server!"

I wonder which UGC and if it is the one causing the issue. I've gone through the Portal 2 subscribed workshop addons, where I have found some content that was removed or isn't just available.

Not sure if Steam is attempting to constantly redownload those, but failing since it doesn't even exist anymore.

However, removing all the non-existing content anymore, hasn't fixed the issue either.

[karl-police]

"An error occurred while attempting to download a file from the UGC server!"

I wonder which UGC and if it is the one causing the issue. I've gone through the Portal 2 subscribed workshop addons, where I have found some content that was removed or isn't just available.

Not sure if Steam is attempting to constantly redownload those, but failing since it doesn't even exist anymore.

However, removing all the non-existing content anymore, hasn't fixed the issue either.

there used to be that theory, that it may is trying to download something that doesn't exist, though I don't know what but there's maybe a way to figure it out with Wireshark, the problem is, it's not directly in plaintext.

[karl-police]

Instead of having to disable the internet. I figured out that net_fake_state NoSteam in the console from Steam, would also prevent those lag spikes. However, your Steam gets disconnected...

[karl-police]

@kisak-valve I did the super debugging.

This is VPROF. +showvprof

We're looking at the peak by CSteam3Client::RunFrame the issue is CSteam3Client::RunFrame. I will re-name this bug properly.

This peak issue, is related to the following issue as well: When you disconnect your internet (e.g. going in Airplane mode) and reconnect your internet. CSteam3Client::RunFrame will peak. That's why the number on the image is 112028.58p.

Don't know why. Maybe the callbacks clog up, and then SteamAPI_RunCallbacks tries to run it all. And then it just peaks a lot. So the question is if it's still clogging up or re-registering the callbacks.

CSteam3Client::RunFrame function apparently, is present in engine.dll. Disabling the function does prevent the issue, but is not a real solution, as it breaks Co-op and anything else that would require Steam in Portal 2.

CSteam3Client::RunFrame is a function apparently, to process callbacks from SteamAPI_RunCallbacks.

SteamAPI_RunCallbacks is part of Steamworks https://partner.steamgames.com/doc/api/steam_api

This makes this issue, a Steam, Steamworks and Portal 2 related issue.

[karl-police]

Additionally this is not only present in Portal 2. But any Portal 2 related mod, like Aperture Tag.

I will be trying if I can dive deeper into all of this, and figure out what kind of callbacks are causing this issue. There's one callback that is probably messing this all up.

[karl-police]

I was dumping some callbacks. 333, 738 and 1701 are callback IDs that I have no info about yet.

The game seems to register these callbacks in int:

331, 304, 1101, 1102, 703

[karl-police]

I have tired blocking all callbacks. At least I guess so.

When I did that, I'd not get that huge peak when re-connecting. And the console is logging this everytime. This is very suspicious, it might be the thing that is causing those lag spikes.

[karl-police]

Acquired more info

PersonaStateChange = 304
GameOverlayActivated = 331
GameLobbyJoinRequested = 333

SteamAPICallCompleted = 703
FloatingGamepadTextInputDismissed = 738

UserStatsReceived = 1101
UserStatsStored = 1102

 SteamGameCoordinator = 1701

I blocked 1101 and 1102 though the thing is still there, for the inbetween lag spikes. So, unsure.

When simulating NoSteam and then reconnecting Steam. Friend function stuff has a lot of calls

[karl-police]

I blocked callback 304. PersonaStateChange. Blocking this callback solved the major of those peaks that caused lag, and also solves the major peak when re-connecting.

IClientFriends::GetFriendPersonaState for some reason Portal 2 is always trying to somehow keep this updated, even though the only time you see the friendlist on Portal 2, is when you go to that Co-op menu.

[karl-police]

@kisak-valve I have more or less re-production steps for this issue now. Not sure how I should update them without ruining. The old steps. (there weren't any)

But it's related on Steam Friends. So the more friends someone has, the worser this issue is, apparently. This makes a lot of sense to @RedSkittleFox comment, that they get this only on a certain account.

[karl-police]

Created a plugin, so that I have a fix and others can get a fix for it as well.

https://github.com/karl-police/portal2_steamAPI_peakFix