Muted sound when shotgun fires?

[ghost]

Hi! I'm debugging a weird issue with my mod, and hoping maybe someone can point me in the right direction...

We have a shotgun that's almost identical to the one from Half-Life. Originally, primary fire was 4 pellets and secondary was 8. Then I made it 16 for primary, 32 for secondary..and we started seeing this issue. If lots of pellets hit a player at once, the entire shot seems to be 'silent' (no gunshot sound, no flesh hit/takedamage sound, nothing).

Now I'm testing a build with 80 pellets for primary, 160 for secondary. It's very easy to recreate this if more than about a quarter of the pellets hit a player. You can gradually aim closer and closer to someone, and eventually the sound of pellets hitting the wall gets replaced by silence (as lots of them hit the player).

It seems like a queue or something that holds sounds is being flushed/overwritten..?

So far what I've determined is:

1. it's not caused by null.wav playing (if I replace null.wav with something that has sound, I don't hear that)
2. it doesn't seem to be caused by most of the obvious clientside sounds (bullets hitting textures, shotgun firing, decals tracing etc)

Anyone have ideas about how to debug this / what could be causing it? Any idea how to make snd_show 1 work? (it says it can't be used in multiplayer when I try to enable it)

[tschumann]

Does what you're shooting actually take damage when this happens?

[ghost]

Yep, everything except sound is working correctly. All my traces hit and deal damage, make blood spurts etc.

[tschumann]

Is it heard by both the shooter and the victim? Or one or the other?

[ghost]

It's silent for both (with both using the same client binary for testing)

Wrong! Doing some more testing now, and it's only silent for the person being shot.

[ghost]

I found out that I can do 'deathmatch 1; maxplayers 1' and run a local game with snd_show 1....but I can't use snd_show when viewing a demo (as it becomes multiplayer). Reproducing this in a useful setting might be tricky..

Oh, and I just made a correction up above: it's only silent for the person being shot.

[ghost]

Another update: it's caused by the player code that spawns blood sprites (as tempents). Not sure how the server architecture works (since it's closed source!!), but basically the stuff to play sounds gets dropped when too many pellets are making someone bleed at once. Now to make a cheeky fix...

[ghost]

I fixed this by making blood spawn way less often (every 10th shotgun pellet), but a better fix would be to raise the temp entity limit above 500 or whatever it is now. Too bad all the code for that is inaccessible and hidden behind the engine!

[tschumann]

Out of curiosity, how did you work out what the issue was?

[ghost]

First, I tried looking at all the stuff involving sounds on the client side..

• null.wav
• sounds that happen when the client gets a shotgun fire event
• decal tracing sounds
• texture hit sounds

It wasn't any of these. I messed around a bit, then started looking at the server side again on a whim. The server code for my shotgun is basically like this (the important part):

for ( ULONG iShot = 0; iShot <= cShots; iShot++ )
    {
        x = ...
        y = ...
        Vector vecDir = vecDirShooting + (x * vecRight) + (y * vecUp);
        Vector vecEnd = vecSrc + vecDir * flDistance;
        UTIL_TraceLine(vecSrc, vecEnd, dont_ignore_monsters, ENT(pev), &tr);

        if (tr.flFraction != 1.0)
        {

            pEntity->TraceAttack(...)
        }

        // make bullet trails
        UTIL_BubbleTrail( vecSrc, tr.vecEndPos, (flDistance * tr.flFraction) / 64.0 );
    }

I already knew it's caused by more pellets hitting a player (and less of them missing), so I commented out the TraceAttack. With that taken out, it didn't happen at all (of course, the shotgun couldn't hit either). So then I knew it was below that function...

int CBasePlayer::TraceAttack( ... )
{
    ... stuff about hitgroups in here ...

        // only spawn blood if this player can be hurt
        if ( ShouldSpawnBlood( pevAttacker ) )
        {
            SpawnBlood(ptr->vecEndPos, BloodColor(), flDamage, this);
            TraceBleed( flDamage, vecDir, ptr, bitsDamageType );
        }

        AddMultiDamage( pevInflictor, pevAttacker, this, flDamage, bitsDamageType );
    }

    return iHitgroup;
}

It was down to the 'spawn blood' stuff or the 'addmultidamage' stuff. Changing addmultidamage didn't affect it, so I tried cutting out the blood spawning. My shotgun could hit, deal damage and make sounds (with no blood).

This is where it gets annoying: if you trace down far enough, blood and tracers both use tempents. If I shoot 100 pellets at someone and they all hit, I make more than 200 tempents (including sounds etc). Considering that the server has a limit of 500 tempents, and my game has 10ish players in the server at the same time, this is quite bad! I'm not sure what the rules are about making them, but they seem to flush out old ones if you make enough at one time (so the tempents playing sounds vanish).

So for now the fix is to draw blood every 10th pellet (using my own DMG_SHOTGUN to massage the spawnblood part of TraceAttack). But I'm not happy with this as a solution, and it would be really great if we could have more than 500 tempents. Or I could rearchitect all the blood/damage code to just draw one big splotch for each shotgun attack.

For what it's worth, the reason I'm digging myself deeper and deeper with this is to try and make a shotgun where the damage scales across ranges properly. If there's some way I can just trace a cone around the crosshair, that would be super useful...

also I should mention that having a bot to test with in a live game was really valuable. I couldn't play back a demo under different client builds (because it still makes the same tempents that it did when the demo was written), and it's annoying to bug people to test with me over and over. Since the silence only happens for the person being shot, it's easy to let a bot shoot me over and over while I try different changes.

[ghost]

It's a good thing I never closed this. I'm still getting various crashes and things caused by this. The latest one seems to be some kind of general memory exhaustion on the server, or on clients...right now I literally have a build where the server segfaults inside the Vector + operation.

The best part is that I got to this one by cleaning up some of our spectator code that harmlessly puts things on the stack and has no logical errors in it (which is why it takes 10 minutes of 30 bots firing 80 pellet shotguns to reveal this).

I guess video games just weren't meant to have more than 8 shotgun pellets at a time. Must be one of those million dollar hard computer science problems..

[SamVanheer]

I checked the engine source code (i have access, SC dev, can't reveal specifics); sounds are put into the unreliable buffer. So too much (reliable) data being sent will cause it to just ignore the sound.

You could use pfnEmitSound to send it reliably, but that won't solve your problems, since it could cause a reliable channel overflow. You'll have to use an approach similar to tracers and only play sounds/show decals once for a spread of pellets.

Unfortunately, GoldSource just isn't capable of handling that volume of events happening in 1 frame.

[ghost]

Unfortunately, GoldSource just isn't capable of handling that volume of events happening in 1 frame.

Do you think it would be if the engine code was open sourced and people could change a handful of #defines for things like buffer sizes ?

I spent ages debugging this thing and ended up gutting half the code that happens under TakeDamage (which included sending broadcasts for the spectator director every time a pellet hit someone, among other ridiculous things). It's all working now and we have what I think is probably the most consistent shotgun in any FPS game, but it basically broke my will to keep being ambitious with my mod. Oh, and I had to break spectating completely... which I don't mind since it shouldn't be coupled to the damage code in the first place.

Sure wish I was part of the inner circle with engine access.

Even if I had access to fix the engine, testing this revealed my favourite bug so far: if I leave a server full of bots running until they get around 2000 total kills, the game crashes inside the Steam overlay binary. It happens even if I tell Steam to not run the overlay with Half-Life too! Whatever this game does with memory behind the scenes must be really...interesting.

[SamVanheer]

I doubt it would ever be open sourced, since there are anti-cheat measures in it, and it hasn't happened yet, so it likely never will.

Increasing a few limits won't do, unfortunately. What you need is code that can be run on the client side, responding to state changes from the server.

We've also noticed crashes related to the Steam overlay. One particular case was that we compiled parts of the engine with Visual Studio 2015, causing some code to break due to version issues, and code that doesn't function correctly when targeting Windows XP. If you compile your mod with VS2015 and target XP, try using an older version instead. Note that there is some old code in the engine meant to interact with the overlay. I doubt it's actually in use, but it looked pretty risky from what i could tell.

Regarding memory usage, it's pretty terrible. Various different allocators are used, and access to memory isn't always as secure as you'd hope.

We've removed HLTV (spectators) completely from our version of the engine. It simplifies a lot of code, but breaks any mods that try to target spectators explicitly. Essentially, all of the networking code for HLTV is separate from the rest of the game, so if your code breaks there, it could be due to some ancient bug that was never found.

[ghost]

Hmm..so the engine is closed source because it contains anti-cheat stuff... but the built-in consistency checking of the client DLL is disabled ?

The engine and hlds are so full of issues that people are flat-out replacing them and bragging about it on here. We even have issues going untouched where people ask for help making cheats!

It sorta looks like everyone outside of Sven Coop is hopeless. At least when it comes to modding this 19 year old game that isn't making money anymore. What's the point of open sourcing 1/3 of the codebase, and not documenting the rest at all?

[SamVanheer]

Even in the SC team not much is happening on the engine side. I was pushing for bug fixes and stability updates, but nothing's being done. Seems like nobody has any interest in fixing anything.

[ghost]

Another example just cropped up here

Apparently the rehlds people are finding and fixing issues in the engine code that isn't public anywhere. It's supposed to be through reversing, but they don't explain their method anywhere.

Didn't someone famously go to jail for leaking the HL2 source code?

[SamVanheer]

They might just be reverse engineering the engine using a program like IDA. You can get a lot of information from it if you use the game dll interfaces and Quake's engine source as a starting point.