Model rendering issues when using NPC models

[impulsh]

When using NPC models as player models and overriding some animation code to translate activities properly, they will sometimes cause various issues when the model is rendered.

The most reliable way that I've found to make it happen is to use a map that has some PVS optimization (my test map is https://steamcommunity.com/sharedfiles/filedetails/?id=1532126505) and have two players be a fair distance apart and in different visleaves. After waiting a couple of minutes, bring the players back together and there's about a 1 in 3 chance that they will have some sort of bone issue. It's more likely to happen for yourself if you are tabbed out of the game.

Forcing a full update for your client will resolve this issue temporarily, and many players have resorted to having to use record 1; stop in console to do so.

There are a couple different kinds of bugs that will happen, including incorrect bone rotation: https://i.imgur.com/dRiHaXs.jpg

Models not rendering at all: https://streamable.com/c2y7z

And models twitching left and right rapidly, which I couldn't get a video of at the moment.

I've isolated this issue with a test gamemode that includes a stripped down version of the animation code that we use. No other addons/lua files have been ran except for the gamemode code and base GMod stuff. You can check out the gamemode here https://gist.github.com/impulsh/b6de45e03ecf5b2c5d722a1bea07ee0e

[impulsh]

The issues that are happening seems to be similar to #3010. Using the fix from https://github.com/ValveSoftware/source-sdk-2013/issues/404#issuecomment-405042224 might solve the problem

[alexgrist]

Bone positions become completely invalid which is the point at which the player becomes completely invisible.

] lua_run_cl print(LocalPlayer():GetEyeTraceNoCursor().Entity:GetBonePosition(1))
-nan(ind) -nan(ind) -nan(ind)    nan nan 0.000

[its-johnny]

@willox @robotboy655

[impulsh]

This also seems to happen for regular player models, although not nearly as frequent as NPC models it seems

[robotboy655]

Merged the potential fix from https://github.com/ValveSoftware/source-sdk-2013/issues/404#issuecomment-405042224 Let me know if that fixes it.

[impulsh]

When would this be pushed to the dev branch?

[robotboy655]

All changes are pushed to Dev branch within minutes of them being committed automatically.

[alexgrist]

The patch has fixed it partially, players no longer go invisible at all and the spine twisting/twitching seems much less frequent.

[alexgrist]

Not sure if this was reverted in the latest patch but it seems to have started occurring frequently since.

[robotboy655]

It wasn't.

[MuckerMayhem]

I'm seeing this a lot now on citizen based NPC models, the spine is turned roughly 90 degrees to the right if I'm not mistaken. Happening frequently to those with higher FPS.

[WolffeUS]

Been starting to see this more and more often now that I'm entering gamemodes with higher FPS, honestly a weird occurrence and I've seen this occur on both Citizen and Combine based models

[neico]

I could only speculate that it's becoming more of an issue with the x64 branch if that's where you're testing it.

And yes, the higher the FPS the higher the chance of it happening (max_fps 120 on the client should prevent it from happening as a workaround), but really, it wasn't a problem anymore with the Lerp_Hermite specialisation, so I'm at a loss as to where else it could come from, someone would need to do some deep digging if it still has to do with the pose params going out of bounds.

[Kouch1]

The only workaround I've found is starting to record a demo with 'record demo' in console, then following it up with 'stop'.

[alexgrist]

Yeah, that forces a full client update which would usually be done with cl_fullupdate but that command requires cheats.

[impulsh]

@robotboy655 Could this get reopened? This issue seems to still be persisting even through the lerp fix

[impulsh]

We've found a pretty reproducible test case for triggering the issue. It isn't as extreme as we've seen in the wild, but it's as good as it's going to get in terms of reliably reproducing the issue.

We simply switched between the male_01 and police models a couple times and it would happen almost every time as captured in this video: https://streamable.com/mebf8. It shows the model being switched 6 times and with the final switch back to the male_01 model, you can see the spine being distorted. Shortly after, cl_fullupdate was used to reset the entity and fix the issue.

The issue also shows up in demos, so I've uploaded one at https://www.dropbox.com/s/mw4a27ucacgcieo/animtest.dem if it's at all useful.

The test code that was used is below. The gamemode used is the one linked in the OP (https://gist.github.com/impulsh/b6de45e03ecf5b2c5d722a1bea07ee0e).

concommand.Add("animmodel", function(client)
    for i = 1, 3 do
        timer.Simple(i * 0.1, function()
            local newModel = client:GetModel() == "models/humans/group01/male_01.mdl" and "models/police.mdl" or "models/humans/group01/male_01.mdl"
            client:SetModel(newModel)
        end)
    end
end)

[willox]

It appears as if changing the model results in the pose parameters being copied over, but the models have different pose parameters (or pose parameters are stored in a different order.)

I can't say for sure it is related to your first post, but I can try to fix it. I would assume this only causes any issues when the players' model is changed.

[willox]

I pushed a fix for that to the dev branch. I'm still not sure if it'll fix the actual issue.

[alexgrist]

It appears as if changing the model results in the pose parameters being copied over, but the models have different pose parameters (or pose parameters are stored in a different order.)

I can't say for sure it is related to your first post, but I can try to fix it. I would assume this only causes any issues when the players' model is changed.

I'll give it a test. This happens on it's own over time (5-10 minutes) without any model changes but changing the model a couple times is the most reproducable trigger of the issue.

[Jaydawg8888]

I pushed a fix for that to the dev branch. I'm still not sure if it'll fix the actual issue.

It's happening less frequently now, however, it is still noticeable in a couple of players over time 10minutes & People still continue to go invisible which in combat screws over the person they're fighting against.

[TIMONz1535]

I can confirm that it is still reproduced when I change the player model to the NPCs model and back. Some pose parameters goes up to 300k or even inf/nan by engine.

ClearPoseParameters right after the model change does not help. but I can just set player:SetPoseParameter("body_yaw", 0) for example in UpdateAnimation and forget.

Or maybe the calculation code of "body_yaw" is completely broken.

[robotboy655]

I have made some changes that may or may not help with this.

I could never reproduce this the way described above, but I did cause similar issues by manually setting cycle to infinity in the engine.

[robotboy655]

I am considering this solved for now, unless someone can disprove it.

[TIMONz1535]

Weeeeeeeeeeell I found it 💀

Be prepared, there is several things that lead to the main issue.

models.zip

1) Player:EyeAngles client-side is not normalized

Let's take a simple pose params code of some old Prone mod. It uses DoD's shader anims for models.

hook.Add(
    "UpdateAnimation",
    "Debug3642",
    function(ply)
        local EyeAngP = ply:EyeAngles().p
        if EyeAngP < 89 then
            ply:SetPoseParameter("body_pitch", math.Clamp(-EyeAngP, -10, 50))
        end
        -- debug
        if ply == Entity(1) then
            print(EyeAngP, EyeAngP < 89, math.Clamp(-EyeAngP, -10, 50))
        end
    end
)

https://github.com/Facepunch/garrysmod-issues/assets/7351599/3b1fa202-b028-462e-933e-cd073e08e1e3

Now you can see how your angle pitch value becomes 359 when you stop moving the camera above the horizon. Model is shaking - we're on the right way.

Expected behavior: Player:EyeAngles should be normalized.

2) Player:SetPoseParameter prediction breaks

If we normalize the angle, model is no longer shaking, but it's not a solution - the angle was still clamped, so why is it breaking pose params? Based on condition when the camera stops, we accidentally stopped setting pose param. Let's simplify it.

hook.Add(
    "UpdateAnimation",
    "Debug3642",
    function(ply)
        local EyeAngP = math.NormalizeAngle(ply:EyeAngles().p)
        if EyeAngP > -40 then
            ply:SetPoseParameter("body_pitch", math.Clamp(-EyeAngP, -10, 50))
        end
        -- debug
        if ply == Entity(1) then
            print(EyeAngP, EyeAngP > -40, ply:GetPoseParameter("body_pitch"))
        end
    end
)

Because of this, the prediction starts to go crazy. Set -tickrate 20 instead of 33 and you will be happy.

https://github.com/Facepunch/garrysmod-issues/assets/7351599/785bb685-20a5-47a9-8018-e7f5a586842b

Expected behavior: If you stop touching the pose param, it doesn't go crazy.

3) Untracked pose params goes NaN and break rendering (derivative issue)

Why don't we see this on the player models? The fact is that almost all player models pose parameters are controlled by the engine/base Lua - they set the value every frame. But on NPC models, there are a lot of untracked pose parameters, so prediction goes crazy. On the player model we can use for example vehicle_steer or vertical_velocity outside of vehicle.

hook.Add(
    "UpdateAnimation",
    "Debug3642",
    function(ply)
        local EyeAngP = math.NormalizeAngle(ply:EyeAngles().p)
        if EyeAngP > -40 then
            ply:SetPoseParameter("vehicle_steer", math.Remap(EyeAngP, -30, 30, 0, 1))
        end
        -- debug
        if ply == Entity(1) then
            print(EyeAngP, EyeAngP > -40, ply:GetPoseParameter("vehicle_steer"))
        end
    end
)

hook.Add(
    "CalcMainActivity",
    "Debug3642",
    function(ply)
        if ply:Crouching() then
            return ACT_MP_STAND_IDLE, ply:LookupSequence("drive_jeep")
        end
    end
)

https://github.com/Facepunch/garrysmod-issues/assets/7351599/8e935672-49f1-4652-9c9f-ae38a8d98450

Well, only with animation that uses our pose param, when breaking the prediction we finally get a NaN and an invisible model as result.

Expected behavior: Ugh... I think this is the result of the 2nd case. So you need to solve the 2nd case.

4) Model switching inherit prediction break (the actual issue)

Now I just changed the model from DoD's to NPC's. I'm not touching anything, all pose params are controlled by the engine. Some pose params are just copied from the old model to new with the same ID, and of course the prediction breaks - the engine no longer sets these pose params, because these are different names!!!

if SERVER then
    hook.Add(
        "KeyPress",
        "Debug3642",
        function(ply, key)
            if (key == IN_USE) then
                if ply:GetModel() ~= "models/humans/group03/male_03.mdl" then
                    ply:SetModel("models/humans/group03/male_03.mdl")
                    ply:SetColor(Color(255, 255, 255))
                else
                    -- ply:SetModel("models/player/kleiner.mdl")
                    ply:SetModel("models/player/p_kleiner.mdl")
                    ply:SetColor(Color(0, 255, 0))
                end
            end
        end
    )
end

https://github.com/Facepunch/garrysmod-issues/assets/7351599/2936311b-d5c9-4aae-b091-c59949430c9a

Expected behavior: Do not inherit pose parameter values and reset pose parameters prediction when model changed.

Thank you for spending your life on this

[robotboy655]

What are your test conditions? Dedicated server? What gamemode or what mod are you using to set the animations? What beta for the server and client? Are you able to reproduce this in Sandbox, without any mods?

I am struggling to make anything happen on 16tick dedi server using your code examples.

[TIMONz1535]

I use dedicated server, main branch, Sandbox gamemode. Client is also main branch.

Protocol version 24
Exe version 2023.06.28 (garrysmod)
Exe build: 17:00:46 May 24 2024 (9280) (4000)
GMod version 2024.05.24, branch: unknown
Windows 32bit Dedicated Server

Protocol version 24
Exe version 2023.06.28 (garrysmod)
Exe build: 19:00:21 Mar 21 2024 (9246) (4000)
GMod version 2024.03.21, branch: unknown, multicore: 1
Windows 32bit

I started the local server from the game client without addons -noworkshop -tickrate 20, and created an autorun lua file with case 3, then I slowly raise the camera up and it looks shaky as well. So -tickrate 20 and probably gmod_mcore_test 1 is important for crazy extrapolation. I use default kleiner model.

[robotboy655]

Default kleiner model? Not the one you attached?

[TIMONz1535]

Yes, in example 3, I use the default kleiner gmod player model (normal color) to reproduce the NaN extrapolation as a proof that the issue is not in the "bad" model.

In other examples 1 2 4, I use my model p_kleiner (green color), which has a body_pitch pose parameter. But you can still reproduce 1 and 2 without my model, just change pose parameter to "untracked by engine" one, like in example 3... I mean the example 3 is actually 2 with a little changes, I wrote why.

Only for example 4 you will need my model p_kleiner, because it has pose parameter IDs that intersect with the NPCs model IDs. The default gmod player model has other IDs - no fatal intersection.

[robotboy655]

Are you absolutely certain you have nothing else influencing the poseparameters on the player? I cannot reproduce any oscillations you showcase in your videos, not even slightly, on a clean 20tick dedicated server with multicore enabled clientside.

cl_threaded_bone_setup 1, fps_max 999, mat_queue_mode 2, what am I missing?

How easily can you reproduce the issue?

[TIMONz1535]

Ugh ><

I recorded a video where I took another computer with different hardware, with newly installed Steam with another account, with a completely clean newly installed last release of main branch of gmod, without any other games, and there is no mounted games in gmod.

I do FactoryReset-GMod.bat, then I do read-lock empty cfg/config.cfg because of Steam Cloud. Then I run first game -noworkshop -tickrate 20 for the local server and second one -noworkshop -multirun for actual client.

By default cvars are mat_queue_mode -1, cl_threaded_bone_setup 0, fps_max 150, gmod_mcore_test 0, default graphics settings. I entered developer 1 for detailed log, but this does not affect the bug.

https://www.dropbox.com/scl/fi/lqbgzidcmdql91qbouvz8/woooaw_bug.mp4?rlkey=xvpp4y27gdddoedjsf0dujee6&st=9wjdpd36&dl=0 its pc

AMD Ryzen 7 3700X
Radeon RX 580 Series
Win10 22H2 19045.4412

previous pc

AMD Ryzen 7 7800X3D
RTX 3090
Win11 23H2 22631.3593

[robotboy655]

I see the discrepancy. I was under the impression you were testing the animations using a camera on yourself, not on another player, thanks.