search

multitheftauto / mtasa-blue

Multi Theft Auto is a game engine that incorporates an extendable network play element into a proprietary commercial single-player game.
https://multitheftauto.com
GNU General Public License v3.0
1.38k stars 424 forks source link

Endless Game Freeze during Gameplay #1718

Closed haron4igg closed 3 years ago

haron4igg commented 3 years ago

Got pretty much reports from players about game getting frozen during gameplay. No crash after, PC is unresponsible, but player stays connected. Only hard-reset helps in this case.

MTA Diags, from affected PCs: https://pastebin.mtasa.com/770694034 https://pastebin.mtasa.com/802091107 https://pastebin.mtasa.com/684407507 https://pastebin.mtasa.com/475454060

(waiting for crash dumps from these people)

Dutchman101 commented 3 years ago

I just want to note that during the past year, you have presented us a lot of weird, vague and unreproducable issues (both on here and discord) which turned out to be caused by bad scripting practises/mistakes on your end, or were left not cleared up with you not recognizing how you may be predisposing players to said issues. I am sorry for not stating the latter in relevant places and including explanation, but i didn't feel like it.

What im getting to: it's totally possible for a server to cause client freezes. So what MTA devs can do about this issue might be limited.

But anyways, without further speculation, let's first have someone look on the thread of the dumps to see what it was processing at the time of freeze. Even in the case of abuse by scripts it can give a hint.

haron4igg commented 3 years ago

i agree that in most of cases its my fault. I'm just using MTA API, trying to keep-up with players needs.... and when some minority report issues, i cant just stand away and ignore them. I didn't knew that CEF may be such unstable thing, in such a stable thing like MTA so invested a lot of time adopting it. And now its lead to all those Issues i reported before... i felt very exhausted having so much sleepless nights for last 4 months fighting it, so we got what we got now :(

but anyway i'm very thankful for all your help and very sorry for bothering. I definitely should learn how to investigate crashdumps on my own.

haron4igg commented 3 years ago

Player who had issue with freeze on loading, who reported this crashdump above, just reported that his HDD had damaged cells. After doing cell-remap and reinstalling windows - his issue is gone.

Will look forward for crashdumps from players who experience issue during gameplay

qaisjp commented 3 years ago

Nice find!

ghost commented 3 years ago

Btw, I did analyze one of the crash dumps you provided. I'm not sure if I got it form this issue, but it was a recent one from 20668 build. I checked the main thread, it was stuck on a random function in ntdll.dll. I couldn't get anything useful out of it. Once you provide us with a few more dumps, I might be able tell where it crashed.

haron4igg commented 3 years ago

I got one more, but it is private one (public doesn't creates for this player, probably due to cyrilic symbols in path, file name looks damaged as well). And its similar, but not exacly the same as described issue above. Player told that game stuck here, for 10 seconds (with enabled Lua Trace) but crashed on its own, he wasn't using lctrl + rctrl combo.

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Fri Oct 16 13:23:29 2020 Module = D:\Àëüîøà\GTA\mta\core.dll Code = 0xC0000005 Offset = 0x000661C2

EAX=00000004 EBX=2A63D4D8 ECX=019BB340 EDX=00000000 ESI=019BB340 EDI=60010E68 EBP=0022F258 ESP=0022F248 EIP=5FE961C2 FLG=00010246 CS=001B DS=0023 SS=0023 ES=0023 FS=003B GS=0000

-- End of unhandled exception --

crashdump 20668.0.zip

Few more people reported same issue, i hope i will get good sample soon.

qaisjp commented 3 years ago

Btw, I didn't mention earlier, but the the Lua Trace debug setting resets after MTA restarts, so if they can't get LCtrl+RCtrl to work, it could be because of that

ghost commented 3 years ago

I got one more, but it is private one (public doesn't creates for this player, probably due to cyrilic symbols in path, file name looks damaged as well). And its similar, but not exacly the same as described issue above. Player told that game stuck here, for 10 seconds (with enabled Lua Trace) but crashed on its own, he wasn't using lctrl + rctrl combo.

crash stack trace ![image](https://user-images.githubusercontent.com/10183157/96265049-649b7c80-0fde-11eb-8673-89d727e2deb4.png)

The crash has something to do with binds (S key). Here, szResource points to an invalid address, so it crashes.

haron4igg commented 3 years ago

So, this shouldn't cause a game freeze? looks like a different issue... but anyway i went throue all my binds. i have an option for players to define their own keys for some features. but this feature is limited to only specific keys, there is no dirrect input, but onClientKey to capture user input and 'S' is not in the list. the only bind i have, bindKey("backwards", "down", testRunHack) which is bound to function, which triggers timer, and timer handler does button state check, to detect players, who abuse 'w+a+block+aim+sprint' combination and plays fall animation.
we checked all the binds he have, and only default 'backwards' bound there.

This szResource, is it a lua resource, or something different? If so, how could it be invalid, if only, this isn't memory corruption?

Dutchman101 commented 3 years ago

the only bind i have, bindKey("backwards", "down", testRunHack) which is bound to function, which triggers timer, and timer handler does button state check, to detect players, who abuse 'w+a+block+aim+sprint' combination and plays fall animation. we checked all the binds he have, and only defual 'backwards' bound there.

The crash happened when that player pressed the "s" key (which is the control backwards), which means it's relevant as he froze and crashed during the execution of your function testRunHack - so please provide us the full function code

It's not unlikely that it is unoptimized and that not everyone's PC can handle it, or that it introduces a memory issue risk.

haron4igg commented 3 years ago

This code works in production for more then 4 weeks, but original issue with game-freeze, i first received few days ago.

function doKeyTest()    
    if  ( getPedControlState(localPlayer, "forwards") or getPedControlState(localPlayer, "backwards") ) and 
        ( getPedControlState(localPlayer, "left") or getPedControlState(localPlayer, "right") ) and
        ( getPedControlState(localPlayer, "sprint") and getPedControlState(localPlayer, "jump") and getPedControlState(localPlayer, "aim_weapon" ) ) then

        setPedControlState(localPlayer, "left", false)
        setPedControlState(localPlayer, "right", false)
        setPedControlState(localPlayer, "sprint", false)
        setPedControlState(localPlayer, "jump", false)

        setPedAnimation(localPlayer, "ped", "FALL_collapse", -1, false, true, false, false)
    end
end

function testRunHack(key)
    setTimer(doKeyTest, 10, 1)
end
bindKey("left", "down", testRunHack)
bindKey("right", "down", testRunHack)
bindKey("forwards", "down", testRunHack)
bindKey("backwards", "down", testRunHack)
bindKey("sprint", "down", testRunHack)
bindKey("jump", "down", testRunHack)
bindKey("aim_weapon", "down", testRunHack)
ghost commented 3 years ago

I forgot to mention that the crash occurs after the user clicks the "OK" button in MTA settings. It seems that looping through the binds list in settings causes the crash.

This szResource, is it a lua resource, or something different? If so, how could it be invalid, if only, this isn't memory corruption?

It's the name of the resource, but the name points to an incorrect pointer. Just for the record, anything can corrupt the value of a variable. The malfunction can happen anywhere in code, but it's usually close to where the crash happened.

Can you ask the player if he modified something in the settings before the crash happened? Please ask the one who sent you this crash dump file.

haron4igg commented 3 years ago

Well, now he added new details.. that crash happen when he went to settings, to check is 'Lua Trace' enabled, touched it and game crashed... I feel very sorry about this. its hard to expect something to be done right from non-technial people :(

haron4igg commented 3 years ago

One of players captured this, with Lua Trace force crash.

client_1.5.8-release-20668.0.000_core_0002086a_5_CPxMb_5974DE53_55F3_56E_49905_20201016_1956.rsa.zip

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Fri Oct 16 19:56:41 2020 Module = C:\Program Files (x86)\MTA San Andreas 1.5\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=77601CBC EDX=00000000 ESI=01BE0790 EDI=764CAE00 EBP=045AFEFC ESP=045AFE48 EIP=7092086A FLG=00010246 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

-- End of unhandled exception --

Last frame from recorded video. He was reloading weapon, and then game froozen. He wait for 3-4 secs, and pressed ctrl's. so looks like it is legit freeze, and after 2-3 seconds he got kicked with crash. image

p.s. i also got this freeze once, but Lua Trace didn't worked. for me it happen when i were flying aroung in freecam and chased some car, saw it lod model, and then got freeze. it was v1.5.8-release-20668. Then i installed 1.5.7 and tried with it, after 25 minutes - no freezes. Now trying again with 1.5.8, but no freezes as well so far :(

haron4igg commented 3 years ago

Few insights: Got reports about freeze from 55 players with Windows 10 and 16 players with windows 7 Today got report from one of my testers, about this freeze on our 'technical' server, which wasn't updated since 17.03.2020 (both server and resources) This issue was noticed when i released update to my public servers, simultaneosly with 1.5.8 mta release, ~ at 12.10-14.10.2020 Right now have 5 people hunting for this freeze with 'Process Hacker' instructions. Got one more report of this freeze, with same offset like in dump above, when player used 'Lua Trace' feature. (https://github.com/multitheftauto/mtasa-blue/issues/1718#issuecomment-710260573)

qaisjp commented 3 years ago

It seems that looping through the binds list in settings causes the crash.

@saml1er Could be caused by https://github.com/multitheftauto/mtasa-blue/commit/062aa3fe92242a9e30e3177871a9467a1293ea6d (I haven't looked into the dump)

haron4igg commented 3 years ago

It seems that looping through the binds list in settings causes the crash.

@saml1er Could be caused by 062aa3f

just in case, i think it is 2 separate issues, this https://github.com/multitheftauto/mtasa-blue/issues/1718#issuecomment-710000977 crash related to binds and the game freeze. One player just reported by mistake this crashdump, related to settings issue, like game freeze, because mta stuck for ~10 seconds, before its actually crashed. The freeze issue is never ends with crash, according to my observations so far

haron4igg commented 3 years ago

I'm trying to suggest people to donwgrade to 1.5.7, because we haven't got any reports back about freeze from players on 1.5.7 so far. But players who made a downgrade, reports back this freeze and its finds out mta silently updated to 1.5.8.

Also, no one who got instructions about capturing this issue with Process Hacker, haven't found it... some people with 1.5.8 tells that issue happens 1-2 times and then its gone.

Same happen to me. Since i got this freeze 1st time, i'we spent ~3 days with connected client, spectating different players in different situations and i haven't got this issue anymore. But i'm keep geting reports from new people every day.

was there anything in https://github.com/multitheftauto/mtasa-blue/issues/1718#issuecomment-710260573 dump?

Dutchman101 commented 3 years ago

1.5.8 downgrading

At this point, i don't think its smart to doubt MTA's stability, because of the aforementioned track record of blaming MTA (and for each issue, pinpointing a certain build that you figured is when it started) for issues that later turned out to be a result of something on your server. So, i don't mean it negatively, just saying - unless we start finding out more, it's not yet warranted to start reverting random stuff or doing anything to the auto-update.

But i'm keep geting reports from new people every day

Note that we ourselves haven't received any reports from players, so that indicates the issue may be limited to players on your server.

Also, so far you have only provided a single dump that is definately related (a freeze dump i mean, captured with Lua trace or debug.txt in MTA installation folder, then pressing both CTRL's).. it would be nice if you can get such dumps from more players. The one related to binds can very well be unrelated and random.

In fact, that's a crash that affects 10 individual users out of over a million users on each major release, judging by crash stats. It has done so for a while.

haron4igg commented 3 years ago

This is the second report made with lua trace, i got from player, who sent 'binds' issue before.

15 people in total who had this issue before and got instructions about making dumps, can't get this issue so far. But i still get reports from new people and trying to 'recruit' some of them.

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Sat Oct 17 11:01:35 2020 Module = D:\Àëüîøà\GTA\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=0399FE44 EDX=76E16B94 ESI=01A6BD90 EDI=7553A091 EBP=0399FF14 ESP=0399FE60 EIP=5EFE086A FLG=00010246 CS=001B DS=0023 SS=0023 ES=0023 FS=003B GS=0000

-- End of unhandled exception --

dump from private folder. public is missing. client_1.5.8-release-20668.0.zip

I'm not telling that my resources are innocent. I'm sure that some of my resources causing this issue in conjunction with the 1.5.8 update. Taking into account the facts:

And probably, you can't see it by crash-stats, because it is not a crash and half of responders used to restart PC, others - killing MTA with System Monitor.

qaisjp commented 3 years ago

Thanks for collating and sharing this information @haron4igg. It's better to share this information with us instead of leaving us in the dark.

haron4igg commented 3 years ago

horay! got this on my pc :D MTA still having some IO activity, but windows show that gta_sa.exe unresponsive: image

Made dump with process hacker, its quite big: https://upload.mtasa.com/u/429205061/MultiTheftAuto.zip_

however a bit tentavie about the real reason, i just left game working, attached freecam to some players, mta was windowed. I left pc and when i got back, windows went in power saving mode by dimming displays (but usual, mta keep working normally after such cases). And accoring to game situation when mta got frozen - it appeared after few minutes when i left, when windows was still active.

I've been thinking that CEF could be also related to this problem, but when i had this issue last time on my PC, i've explicitly disabled all CEF features before test and when i got this freeze, there was only 2 subtasks image

Dutchman101 commented 3 years ago

Made dump with process hacker, its quite big: https://upload.mtasa.com/u/429205061/MultiTheftAuto.zip_

Nice.. but now, please create a file in your MTA installation directory (next to Multi Theft Auto.exe) called "debug.txt", so you can force a regular MTA "freeze crash dump" next time it happens as well, by holding down CTRL L + CTRL R simultaneously. It's easier than enabling "Lua trace" every time.. or did you get such a dump already, when it happened to you? Then please upload it as well.

I've been thinking that CEF could be also related to this problem

It's not impossible.. but it wasn't smart to develop your entire gamemode around CEF to begin with. I mean in general. A lot of players have a PC that finds it difficult to handle CEF, either due to performance reasons or some sort of corruption on their PC (a lot of external software and viruses target, or interfere with, Chromium/CEF as well). So, many users can easily get memory issues when a lot of your gamemode is based on CEF, and in the past you mentioned that you didn't know this before so for that i gotta admit that we need better documentation to warn of this. To just do a few things in CEF, but not everything. Memory issues can definately be tied to game freezes.

I will write better documentation for it soon

haron4igg commented 3 years ago

Didn't knew about this debug.txt feature, will do it in future. Accoring to my observations over players who helped me, this lctr+rctrl feature works only until windows detects mta stalled, if mta become 'unresponsive', then it doesn't work anymore. This time i found it 'unresponsive' already :(

CEF Offtop This mode was developed 6 years ago, around ceGUI. Just got tired of it basic look and feel, then added cef control panel for player accounts, settings and team managemet just recently, because like you said, didn't knew. But, since i released it to mass public, it looks pretty stable. (0.4% of crashed sessions, when accessed this panel). Just minority with ~4 GB ram experince extremely slow loading. i just got 'lucky' with one tester on development phase, who raised so many issues related to CEF, which i reported :D ![image](https://user-images.githubusercontent.com/4466653/96503661-4b076880-125c-11eb-9e42-90637fb65907.png) ![image](https://user-images.githubusercontent.com/4466653/96504797-e4834a00-125d-11eb-8722-185704624ce2.png) ![image](https://user-images.githubusercontent.com/4466653/96504883-07adf980-125e-11eb-9b09-baaeb550c248.png)
haron4igg commented 3 years ago

One more dump from player, forced with ctrls. Player downgraded to 1.5.7, played ~3 hours, then quit and mta did silent update to 1.5.8. later he been jumping between servers ~10 times with short 10-15 minute sessions in game and at last session he got freeze, while he were driving car around the map). He can't remeber was there any client restarts, except one, when mta updated. But as for me, i got this freeze on 1st game session at Friday, and on ~3rd session today.

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Mon Oct 19 22:29:15 2020 Module = C:\Program Files (x86)\MTA San Andreas 1.5\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=77D61CBC EDX=00000000 ESI=01BBD400 EDI=77B7AE00 EBP=048FFEFC ESP=048FFE48 EIP=7117086A FLG=00010246 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

-- End of unhandled exception --

client_1.5.8-release-20668.0.000_core_0002086a_5_CPxMb_5974DE53_55F3_7B8_49905_20201019_2229.rsa.zip

Dutchman101 commented 3 years ago

this lctr+rctrl feature works only until windows detects mta stalled, if mta become 'unresponsive', then it doesn't work anymore.

It should always work, even more so, the entire feature is designed to help debugging freezes.. so those players who report it didn't forcibly cause a crash, probably did something wrong.

Anyways, nice.. without further speculation, let's wait for someone to look on the stack of the 2 new freeze dumps.

haron4igg commented 3 years ago

Got one more dump, maden with process-hacker, by one of the players. https://upload.mtasa.com/u/615493695/MultiTheftAuto-freeze2.zip_ MTADiag (please let me know if this kind of reports isn't helpful, so i will provide these hunters with 'lua-trace' instructions)

this morning got report from player, about this issue, experineced on other server. So i made one more poll. so far 65 people responded out of 422 people who seen it. 10 of 27, who also plays on other servers, reported experincing this issue out there.

I'm not using public available resouces, except of some default mta resources, like admin panel or scoreboard. The server which had similar issue, had similar gamemode idea, so they are using same approach, like model replacing, shaders, sounds.

I also asked some players who participated in poll, to keep an eye on this issue with lua-trace instructions, when they playing on other servers, where they seen this problem.

haron4igg commented 3 years ago

Got one more interesting case, this player told that had this freeze 3 times today and was restarting PC. But last time, during drive-out around the map, he got freeze and after ~30 seconds mta crashed on its own:

===============================================

Version = 1.5.8-release-20668.0.000 Time = Tue Oct 20 20:18:56 2020 Module = F:\Games1\Grand Theft Auto San Andreas\proxy_sa.exe Code = 0xC0000005 Offset = 0x00354B1C

EAX=891ADAD3 EBX=000000A6 ECX=0177FBFF EDX=FFA3A4A6 ESI=00000000 EDI=00000016 EBP=00000004 ESP=0177FADC EIP=00754B1C FLG=00010286 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

===============================================

client_1.5.8-release-20668.0.000_proxysa_00354b1c_5_FGM_5774DE53_55F3_6A5_BAA25_20201020_2018.rsa.zip

haron4igg commented 3 years ago

This night improved my GA tracking integration, to report mta versions and also made a simple 'crash tracker', which saves file on users pc when user connects and then removes it during notmal disconnect. if file was found when resource started - it reports a 'crash'. However, its can't detect was it actually freeze, crash or player forcely stopped mta with Task Manager.

Since 21.10.2020 03:00 (GMT+3): image

In total 284 (16%) sessions ended with timeout so far.

if its could be useful - i can provide serials of this users, so you may compare it with your crash reports. (not sure is it allowed to share serials in public).

ghost commented 3 years ago

https://upload.mtasa.com/u/615493695/MultiTheftAuto-freeze2.zip_ MTADiag

Multi Theft Auto.exe only contains the loader code. You need to dump proxy_sa.exe/gta_sa.exe.

One more dump from player, forced with ctrls. client_1.5.8-release-20668.0.000_core_0002086a_5_CPxMb_5974DE53_55F3_7B8_49905_20201019_2229.rsa.zip

bass.dll (audio library) was reading a file from disk when the crash occured. image

Got one more interesting case, this player told that had this freeze 3 times today and was restarting PC. But last time, during drive-out around the map, he got freeze and after ~30 seconds mta crashed on its own: client_1.5.8-release-20668.0.000_proxysa_00354b1c_5_FGM_5774DE53_55F3_6A5_BAA25_20201020_2018.rsa.zip

This is a random crash.

haron4igg commented 3 years ago

I made a test resource, which is playing all the sounds i have (500 sounds) and made a stress test, running multiple 'threads' of playback of whole sound list. At some moment, after ~10-15 minutes of looping, when i've been looking away, mta got frozen. but neither ctrls helped, neither process hacker. When i tried to make a dump of gta_sa.exe, its just tells 'Unable to create minidump: attached to system device isn't working' (i made debug.txt file, but when MTA is grayed out by windows, showing it is unresponsive, ctrls isn't working anymore. Need to catch this freeze when MTA is on screen probably)

Now i trying to replay this test, again and again. Few hours passed but it's just don't appear again :( I've found few damaged ogg files from npc sound bank, but if i focus on it with stress test, nothing happens as well. Also did a focus on last files, which been logged before gta got frozen. I even made so hard stress test, like 50 ms timer, spawning 10 sounds, so MTA just stalled for few minutes, but was showing 'spinner' at bottom and finnaly made it.

All files optimized according to recommendation on https://wiki.multitheftauto.com/wiki/PlaySound3D This damaged files btw, got damaged when i did this optimisation with batch tool, more then 4 years ago. In last update, there were few files included, but they wasn't played yet, its part of event i made, which is waiting this problem to be solved :D

Is it exactly file from disc? could be related to streamed audio? (i have few features which using TTS engine and have some radio-stations streaming music, but neather of these were playing when i got freeze.

wasn't there any other useful hints? may be its possible to see played file, so i can track the part of script which been around.

haron4igg commented 3 years ago

Got it after 1.5 hours of continuos auido stress test on local server. Not sure is it actually related to playBack of this sounds, looks like something else is messing around. mta was in focus this time, so ctrls worked.

will try one more test on completely clean server.

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Thu Oct 22 21:46:55 2020 Module = C:\Games\NS-DayZ\Multi Theft Auto1.5\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=77B61CBC EDX=00000000 ESI=01C6ED08 EDI=761BAE00 EBP=0489FEFC ESP=0489FE48 EIP=689A086A FLG=00010246 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

-- End of unhandled exception --

client_1.5.8-release-20668.0.000_core_0002086a_5_CGNM_BB00A8C0_55F3_FFF_FEE27_20201022_2146.rsa.zip

2 more crash dums from players, captured with Lua Trace:

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Thu Oct 22 20:16:46 2020 Module = D:\Program Files (x86)\MTA San Andreas 1.5\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=77191CBC EDX=00000000 ESI=01C05D60 EDI=74F5AE00 EBP=0470FEFC ESP=0470FE48 EIP=7423086A FLG=00010246 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

-- End of unhandled exception --

-- Unhandled exception --

Version = 1.5.8-release-20668.0.000 Time = Thu Oct 22 20:34:10 2020 Module = D:\Program Files (x86)\MTA San Andreas 1.5\mta\core.dll Code = 0xC0000005 Offset = 0x0002086A

EAX=00000000 EBX=00000001 ECX=77191CBC EDX=00000000 ESI=01BB67F8 EDI=74F5AE00 EBP=0498FEFC ESP=0498FE48 EIP=7423086A FLG=00010246 CS=0023 DS=002B SS=002B ES=002B FS=0053 GS=002B

-- End of unhandled exception --

client_1_5_8-release-20668_0_000_core_0002086a_5_DPxMb.zip

Dutchman101 commented 3 years ago

Well, we recently updated BASS.dll, so im already in contact with Un4seen studio's regarding this freeze. Not to already say that it's the culprit, but they are going to tell us what's at that address which hung (59de63e4) in the last dump Saml1er checked, and what they figure may be behind it (even if it's not on them or MTA). Those guys are really good, so if they find that it's their library at fault, they will most definately quickly provide a fix.

I expect that we know more within 24 hours

Dutchman101 commented 3 years ago

Here is their reply:

Untitled

So @haron4igg, can you provide that file?

haron4igg commented 3 years ago

Yes, this is exactly the damaged file, which i have found, when tested all sounds. which was damaged for a years :( Weird thing is when i been focusing my stress test on this file, like creaing hundreds of playback each second, nothing actually happen to my client. chase_5.zip

Dutchman101 commented 3 years ago

Thanks.. and yeah, it's clearly damaged and doesn't even play sound.

But we'll await the results of Un4seen studio's, to find out whether or not their recent changes made BASS.dll handling of such corrupted audio files, poorer.. as in, that it would lead to a freeze, while it doesn't have to. He's analyzing it now.

@haron4igg For now, i advise you to remove the corrupted file from your resources and see if that makes the freezes for your players stop.

haron4igg commented 3 years ago

this is from NPC sound bank, i did batch conversion with AVS Audio Converter long time ago, to make this files lighter and remove extra channels. there was ~300 of them all together, few of them got damaged, but nobody mentioned that 'zombie' lacking few sounds out of 30, so its been there.

Based on some players reports, who mentioned same freeze on other servers, there could be or same damaged ogg files this servers have, or could it be, that player had a session on my server and then left with this file 'trying to open' and at some moment its caused the freeze?

Also, noticed few things, when this files are open in MTA, it returns an element, but there is no 'onClientSoundStopped' events triggered for it, they just live in memory. Also no warnings, neither in 1.5.7, neither in 1.5.8. When i stop the resource, spawned this files, the sounds remain in element tree ('getElementsByType("sound")') It looks like sounds are not owned by resourse who played it, in general. (not sure is it expected behaviour) destroyElement() is able to remove it, but have no idea what happens with sound under the hood.

haron4igg commented 3 years ago

The crash rate went down at half! (still some players joing with yesterday crash-records). There is still alot of crashes due to minimized MTA, which is don't want to maximize, and players used to kill it with system monitor. I will try to investigate this.

But so far, no more new reports about the freeze during the gameplay, looks like problem is solved! Thank you so much guys, for your help!

forkerer commented 3 years ago

Crashes when trying to maximize mta are common in general, happens often on our server, happened multiple times on my own pc as well, it's directx issue that is known afaik. If i recall correctly, in my case the process often can't be killed normally, and only debug combination that generates dump is able to force it down

Dutchman101 commented 3 years ago

PR #1761 will fix the corrupted .ogg freeze:

un4seen

Dutchman101 commented 3 years ago

Fixed in ce9de5d