multitheftauto / mtasa-blue

Multi Theft Auto is a game engine that incorporates an extendable network play element into a proprietary commercial single-player game.
https://multitheftauto.com
GNU General Public License v3.0
1.41k stars 436 forks source link

Strange behavior of the Skimmer (e.g. crashes on approach) #1151

Closed Rob3681 closed 4 years ago

Rob3681 commented 4 years ago

Describe the bug

Version Multi Theft Auto v1.5.7-release-20344

saesrpg commented 4 years ago

I'm glad you reported this, our server has been experiencing client crashes lately and couldn't put our finger on it, so we just removed all skimmers on the server with runcode and now, no one is crashing.

Server and clientmin both Multi Theft Auto v1.5.7-release-20344

ccw808 commented 4 years ago

Please try with latest client nightly

Rob3681 commented 4 years ago

The problem still occurs in v1.5.7-release-20347.

ccw808 commented 4 years ago

Please try 20348

Rob3681 commented 4 years ago

It seems that it is now fixed.

saesrpg commented 4 years ago

Can confirm that it is also fixed on our server with the 20348 build

Dutchman101 commented 4 years ago

This is the result of undefined behavior somewhere, to which vs2019 is less tolerant due to stricter conformance demands (which is why vs2019 was reverted for now in f22306b)

So far we have only been able to confirm Skimmers always being invisible, and then when you press enter near it (which would make you enter it instantly without anim, due to specialised Skimmer entering handling in MTA) the game will glitch out by screen becoming white and recursing contrast.

(yes, the thing that's also considered as a very rare GTA bug:

sbx320 1:06 AM screen turning white is usually caused by some NaN / Inf crap propagating through everything

dutchman101 1:07 AM the same thing as can happen randomly (but is very rare) like when you're flying a hydra out of bounds, with the game tabbed out an infinite loop..

sbx320 1:08 AM well the issue is related to something going to NaN and NaN + anything is NaN, same with the other operations so it just propagates through the entire code that does anything with floating point numbers

So it would seem that undefined behavior causes this NaN propagation loop (or it's a side effect of something else failing at an earlier stage). We discussed ways to find the problem, but didn't come up with a good static code analysis tool that will simply reveal where undefined behavior is located in the MTA codebase. It could be that more of such issues will pop up in the near future (to hold back vs2019 becoming stable for us), but currently this Skimmer bug is the only known one, as we also already fixed a bunch of others, therefore being the only one on the project card (meant to track when vs2019 is stable) that qaisjp just created.

It would help if other devs/contributors went to look for undefined behavior in the MTA codebase, especially where custom handling of the Skimmer is involved (or called from in any way). If anyone can suggest a reliable static analysis tool that would also be great. If it's done on the entire codebase however, it may help to avoid similarly caused issues in the future.

*Note: For the unconfirmed parts, we are aware of (and have a history of supporting SAES:RPG) with the crashes they posted about in this issue, which they structurally get for unrelated reasons. So that could be a coincidence in their timing of removing all Skimmers, as they also took advised steps that would usually relieve it.

qaisjp commented 4 years ago

Skimmer falls through ground on v1.5.7-release-20422, reported by Sarrum.

See https://i.imgur.com/LLsScka.mp4.

ccw808 commented 4 years ago

Skimmer bug is random for me when self compiling release build in VS2019. All spawned skimmers work fine for one run, then restart MTA and all spawned skimmers fall through the ground.

qaisjp commented 4 years ago

Can't get the above trick to work using a self-compiled Release build in VS2019 (linker version 14.24, which matches linker version of nightly 20422).

I have a hunch that it goes invisible because of culling bugs -- flying in a certain way through certain objects makes it go invisible or visible.

See Saml1er's reproduction of this here: msg link, video link

ghost commented 4 years ago

@sbx320 suggested that I can take dll files from 20422 nightly build and use them to reproduce the skimmer bug with custom mta build. After doing some testing, I found out that the bug comes from either core.dll or netc.dll. And I'm sure that the culprit is core.dll.

You can see in the video how I reproduced the bug on my custom MTA build: https://www.youtube.com/watch?v=wqiYKrk-2SQ&feature=youtu.be

To join a standard MTA server with a custom build without getting kicked, edit disableac in mtaserver.conf as follows:

<disableac>1,2,4,5,6,7,8,11,13,17,21,26</disableac>

20422 nightly build download link: https://drive.google.com/open?id=1SQwQVx8xQRjzO0UVa2v14zh2mMgq_huk

qaisjp commented 4 years ago

It's likely that the vs2019 upgrade is going to be reverted again.

We still need a way to generate nightly builds not on master, otherwise we're just going to keep trying and reverting every so often, and ultimately lose momentum in fixing this bug.

What will be our approach in fixing this? Squash all warnings in core.dll (and other modules if that doesn't fix it), and push through with a fix for #944?

qaisjp commented 4 years ago

According to this message on Discord, the problem is not related to https://github.com/multitheftauto/mtasa-blue/commit/8a4c6e9db83aa6ddcd8bd3de3debd2f6076b20c4 — reverting that commit does not make the problem go away.

qaisjp commented 4 years ago

Fixed in 74189be2a2129c2b2595cf698cd6f3a6286ffc25, r20430

Rob3681 commented 4 years ago

Since version 20627 the problem appears again.

Again this nightly version will be installed automatically when starting the MTA client, even if "Default" was set as the auto updater build type in the MTA settings. I had already mentioned this in another issue. Was it intended that the last nightly would be distributed to everyone via auto update or is it again a configuration mistake?

qaisjp commented 4 years ago

Re-opening as a precaution. Someone please verify.

ccw808 commented 4 years ago

Again this nightly version will be installed automatically when starting the MTA client, even if "Default" was set as the auto updater build type in the MTA settings.

Players with 20582 will not be auto-updated. Because of various bugs players with anything higher than 20582 will get updated

ImSkully commented 4 years ago

Can confirm that this exists in 20627. Tested on stable 20621 and it was fine, however after switching to 20627 the skimmer becomes invisible when approaching it and is only visible from a distance, see this for result.

This is the only behaviour from this previously reported bug that I could reproduce however, no game crashes or screen flickering.

qaisjp commented 4 years ago

On buildinfo: r20627, r20621 (hmm buildinfo could do with range support)

ccw808 commented 4 years ago

Please test 20626

Rob3681 commented 4 years ago

The problems don't occur in 20626.

ccw808 commented 4 years ago

Please test 20629

Dutchman101 commented 4 years ago

It's fixed, not reproducible anymore