search

Source-Python-Dev-Team / Source.Python

This plugin aims to use boost::python and create an easily accessible wrapper around the Source Engine API for scripter use.
http://forums.sourcepython.com
GNU General Public License v3.0
165 stars 33 forks source link

CS:S - OnServerOutput Crash #404

Closed Frag1337 closed 2 years ago

Frag1337 commented 3 years ago

EDIT: Please check https://github.com/Source-Python-Dev-Team/Source.Python/issues/404#issuecomment-904157804

Hello,

I'm currently having an issue where Source.Python crashes when two player's are inside the same trigger_push entity.

This is the console log:

2021-08-22 06:44:52 - sp    -    EXCEPTION    
[SP] Caught an Exception:
Traceback (most recent call last):
  File "..\addons\source-python\plugins\SurfTricks\libs\misc\hooks.py", line 101, in on_touch_pre
    entity = make_object(Entity, args[1])
  File "..\addons\source-python\packages\source-python\entities\_base.py", line 293, in _obj
    return cls(index_from_pointer(ptr))

ValueError: Conversion from "Pointer" (<_memory.Pointer object at 0x011F5C50>) to "Index" failed.

<=> CRASH <=>

Code:

@EntityPreHook(EntityCondition.is_human_player, "touch")
def on_touch_pre(args):
    player = make_object(Entity, args[0])
    entity = make_object(Entity, args[1])

sp info:

sp info

IMPORTANT: Please copy the full output.
--------------------------------------------------------
Checksum      : 5e3156dad6f48fe59901274827251ec8
Date          : 2021-08-23 00:36:57.404125
OS            : Windows-10-10.0.19041
Game          : css
SP version    : 705
Github commit : 38e4fe48579f9445b878624a9439ce10e28aa7e6
Server plugins:
   00: Metamod:Source 1.12.0-dev+1145
   01: Source.Python, (C) 2012-2020, Source.Python Team.
   02: Tickrate_Enabler 0.4, Didrole
SP plugins:
   00: SurfTricks, 1.6, http://trick.surf
--------------------------------------------------------
jordanbriere commented 3 years ago

It's very unlikely that the exception itself is related to the crash you are encountering and probably just means the touching entity isn't networked. Try using BaseEntity instead or catch the exception and exit the call. That said, my first guess as far as the crash is concerned would be a clash between SP's and SM's hook. Try with only SP installed as well as only a minimal reproducible plugin loaded and see if you are still crashing.

Frag1337 commented 3 years ago

Thank you for your reply. You are right, it isn't actually the cause, it was just find through the error log.

So I've tried to debug the exact cause why this crash was happening, since it was reproducable. I've started with disabling Metamod + Sourcemod, it was still crashing, disabled 100 Tick Enabler, it was still crashing, disabled everything related to hooking in my Source.Python plugin, and it was still crashing.. Hmm, I went further ahead and finally found the cause for it, which I would never thought of.

sp info

sp info

IMPORTANT: Please copy the full output.
--------------------------------------------------------
Checksum      : 9ef93fdfb37c25f134642a2fd0231d68
Date          : 2021-08-23 21:45:24.462681
OS            : Windows-10-10.0.19041
Game          : css
SP version    : 705
Github commit : 38e4fe48579f9445b878624a9439ce10e28aa7e6
Server plugins:
   00: Source.Python, (C) 2012-2020, Source.Python Team.
SP plugins:
   00: test
--------------------------------------------------------

test/test.py

from listeners import OnServerOutput
from core import OutputReturn

@OnServerOutput
def on_server_output(severity, message):
    if message.startswith("test123456"):
        return OutputReturn.BLOCK

I have no idea why OnServerOutput is crashing the server, when two players enter a trigger_push, but atleast I found the cause.

I can upload video of proof in some minutes.

https://user-images.githubusercontent.com/9024822/130525331-514bf7c6-8c9e-4851-aa8d-650c91b0470e.mp4

EDIT: While trying to find the crashing function, I've only had this one-time print after the crash, no idea if this might be related to here or anything sort of:

TEST

jordanbriere commented 3 years ago

Similar crashes related to the output listener were reported into #400. But since you provided more context I guess we can continue the discussion here until we know more. For now, two questions come to mind;

Frag1337 commented 3 years ago

I will try to give you answers on your questions tomorrow, thanks for your fast reply.

Frag1337 commented 3 years ago 
from listeners import OnServerOutput
from core import OutputReturn

@OnServerOutput
def on_server_output(severity, message):
    return OutputReturn.CONTINUE

This is crashing aswell (With the same conditions as in the video), this should answer both of your questions.

jordanbriere commented 3 years ago

Hmmm. What about this: core.zip?

Frag1337 commented 3 years ago

I've replaced it, and it still crashes instantly, without any error. It just crashes.

I can also confirm that the same issue persists on Linux.

Edit:

When not loading any plugin, but doing the same condition where it usually crashes, I get the following output:

image

jordanbriere commented 3 years ago

What map/coordinates?

Frag1337 commented 3 years ago

Map: http://51.38.124.29/fastdl/cstrike/maps/surf_buck-wild_b02.bsp.bz2

Teleport inside trigger_push

setpos -1558.640869 1562.878906 -1974.968750;setang 2.930659 121.932831 0.000000

I'm pretty sure this crash is not related to the map, or this specific trigger_push, but it's the most easiest to reproduce with two players. It has to be something regarding the console output when being inside a trigger_push:

DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.

You could probably reproduce the same output alone, if you spawn a bot into the same coordinates without it being stuck.

Edit: @jordanbriere I've made you a reproducable plugin. Just type test_teleport and it should teleport you & a bot into a trigger_push to reproduce the clamping spam.

test/test.py

from commands.server import ServerCommand
from core import OutputReturn
from cvars import ConVar
from filters.players import PlayerIter
from listeners import OnServerOutput
from mathlib import Vector

# surf_buck-wild_b02
coords1 = Vector(-1632, 1632, -2000)
coords2 = Vector(-1463, 1800, -2000)

def load():
    bot_quota = ConVar("bot_quota")
    bot_quota.set_int(1)

@ServerCommand("test_teleport")
def command_test_teleport(command):
    for player in PlayerIter():
        if player.dead:
            player.spawn()

        print("=> Teleporting {}".format(player.name))

        if player.is_bot():
            player.teleport(coords1, None, None)

        else:
            player.teleport(coords2, None, None)

@OnServerOutput
def on_server_output(severity, message):
    return OutputReturn.CONTINUE

Comment @OnServerOutput and both OutputReturn/OnServerOutput imports to reproduce your assertion failed message.

jordanbriere commented 3 years ago

Thanks. However, I'm unable to reproduce a crash. What is your current sp info, as well as your logging config?

Frag1337 commented 3 years ago

Thats very weird, it's working every time for me.

Steps to reproduce for me: Type sp plugin load test and test_teleport into console, and its crashing for me instantly.

sp info

IMPORTANT: Please copy the full output.
--------------------------------------------------------
Checksum      : 7fa16f4b1046d1ef16fda2d600604871
Date          : 2021-08-25 20:49:23.166773
OS            : Windows-10-10.0.19041
Game          : css
SP version    : 705
Github commit : 38e4fe48579f9445b878624a9439ce10e28aa7e6
Server plugins:
   00: Source.Python, (C) 2012-2021, Source.Python Team.
SP plugins:
--------------------------------------------------------
[LOG_SETTINGS]
# Set to the logging level to use for Source.Python.

# To show all messages, set to level 5.
# To show debugging messages, set to at least level 4.
# To show informational messages, set to at least level 3.
# To show warning messages, set to at least level 2.
# To show exception messages, set to at least level 1.
# To show only critical and base messages, set to 0.
level = 2

# Set to the areas to log (console, main log, sp log, script log):
#   Console    = 1
#   Main log   = 2
#   SP log     = 4
#   Script log = 8
# Add each of the areas up and set the value to that number.
# Example:
#   To have the messages print to the console and be added to SP's log:
#       Console + SP log = 1 + 4 = 5
areas = 5
jordanbriere commented 3 years ago

What is the output of find log?

Frag1337 commented 3 years ago 
find log
"sp_notify_on_update" = "1" min. 0.000000 max. 1.000000
 - Log a warning when a Source.Python update is available. Requires sp_check_for_update to be set to 1.
"sp_logging_areas" = "5" ( def. "1" )
 - The Source.Python base logging areas
"sp_logging_level" = "2" ( def. "0" )
 - The Source.Python base logging level
"sv_debug_player_use" = "0"
 game replicated
 - Visualizes +use logic. Green cross=trace success, Red cross=trace too far, Green box=radius success
"dbghist_dump"
 game
 - Dump the debug history to the console. Format: <category id>
    Categories:
     0: Entity I/O
     1: AI Decisions
     2: Scene Print
     3: Alyx Blind
     4: Log of damage done to player
"log_verbose_enable" = "0"
 game
 - Set to 1 to enable verbose server log on the server.
"log_verbose_interval" = "3.0"
 game
 - Determines the interval (in seconds) for the verbose server log.
"mp_logdetail" = "0" min. 0.000000 max. 3.000000
 game
 - Logs attacks.  Values are: 0=off, 1=enemy, 2=teammate, 3=both)
"sv_netspike_output" = "1"
 - Where the netspike data be written?  Sum of the following values: 1=netspike.txt, 2=ordinary server log
"cl_logofile" = "materials/decals/spraylogo.vtf"
 archive
 - Spraypoint logo decal.
"con_timestamp" = "0"
 - Prefix console.log entries with timestamps
"con_logfile" = "console.log" ( def. "" )
 - Console output gets written to this file
"sv_logblocks" = "0"
 - If true when log when a query is blocked (can cause very large log files)
"sv_logsdir" = "logs"
 archive
 - Folder in the game directory where server logs will be stored.
"sv_logfile" = "1"
 archive
 - Log server information in the log file.
"sv_logflush" = "0"
 archive
 - Flush the log file to disk on each write (slow).
"sv_logecho" = "1"
 archive
 - Echo log information to the console.
"sv_log_onefile" = "0"
 archive
 - Log server information to only one file.
"sv_logbans" = "0"
 archive
 - Log server bans in the server logs.
"sv_logsecret" = "0"
 - If set then include this secret when doing UDP logging (will use 0x53 as packet type, not usual 0x52)
"sv_logfilename_format" = ""
 archive
 - Log filename format. See strftime for formatting codes.
"sv_logfilecompress" = "0"
 archive
 - Gzip compress logfile and rename to logfilename.log.gz on close.
"log"
 - Enables logging to file, console, and udp < on | off >.
"logaddress_add"
 - Set address and port for remote host <ip:port>.
"logaddress_delall"
 - Remove all udp addresses being logged to
"logaddress_del"
 - Remove address and port for remote host <ip:port>.
"logaddress_list"
 - List all addresses currently being used by logaddress.
"sv_rcon_log" = "1"
 - Enable/disable rcon logging.
"sv_setsteamaccount"
 - token
Set game server account token to use for logging in to a persistent game server account
"sv_logdownloadlist" = "1"
"sv_lowedict_action" = "0" min. 0.000000 max. 5.000000
 - 0 - no action, 1 - warn to log file, 2 - attempt to restart the game, if applicable, 3 - restart the map, 4 - go to the next map in the map cycle, 5 - spew all edicts.
"spew_consolelog_to_debugstring" = "0"
 - Send console log to PLAT_DebugString()
"joy_axisbutton_threshold" = "0.3"
 archive
 - Analog axis range before a button press is registered.

image

jordanbriere commented 3 years ago

No matter what I try, I can't get it to crash:

... 
test;test_teleport
[SP] Unloading plugin 'testing'...
[SP] Successfully unloaded plugin 'testing'.
[SP] Loading plugin 'testing'...
[SP] Successfully loaded plugin 'testing'.
=> Teleporting Tom
=> Teleporting L'In20Cible
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
DataTable warning: player: Out-of-range value (2350.000000/1000.000000) in SendPropFloat 'm_vecBaseVelocity', clamping.
sp info

IMPORTANT: Please copy the full output.
--------------------------------------------------------
Checksum      : 04dee6431c4f0586eb0c615240651786
Date          : 2021-08-25 21:13:41.375390
OS            : Windows-10-10.0.19041
Game          : css
SP version    : 705
Github commit : 38e4fe48579f9445b878624a9439ce10e28aa7e6
Server plugins:
   00: Source.Python, (C) 2012-2021, Source.Python Team.
SP plugins:
   00: testing
--------------------------------------------------------

Game will not start until both teams have players.
find log
"sp_notify_on_update" = "1" min. 0.000000 max. 1.000000
 - Log a warning when a Source.Python update is available. Requires sp_check_for_update to be set to 1.
"sp_logging_areas" = "5" ( def. "1" )
 - The Source.Python base logging areas
"sp_logging_level" = "2" ( def. "0" )
 - The Source.Python base logging level
"sv_debug_player_use" = "0"
 game replicated
 - Visualizes +use logic. Green cross=trace success, Red cross=trace too far, Green box=radius success
"dbghist_dump"
 game
 - Dump the debug history to the console. Format: <category id>
    Categories:
     0: Entity I/O
     1: AI Decisions
     2: Scene Print
     3: Alyx Blind
     4: Log of damage done to player
"log_verbose_enable" = "0"
 game
 - Set to 1 to enable verbose server log on the server.
"log_verbose_interval" = "3.0"
 game
 - Determines the interval (in seconds) for the verbose server log.
"mp_logdetail" = "0" min. 0.000000 max. 3.000000
 game
 - Logs attacks.  Values are: 0=off, 1=enemy, 2=teammate, 3=both)
"sv_netspike_output" = "1"
 - Where the netspike data be written?  Sum of the following values: 1=netspike.txt, 2=ordinary server log
"cl_logofile" = "materials/decals/spraylogo.vtf"
 archive
 - Spraypoint logo decal.
"con_timestamp" = "0"
 - Prefix console.log entries with timestamps
"con_logfile" = "console.log" ( def. "" )
 - Console output gets written to this file
"sv_logblocks" = "0"
 - If true when log when a query is blocked (can cause very large log files)
"sv_logsdir" = "logs"
 archive
 - Folder in the game directory where server logs will be stored.
"sv_logfile" = "1"
 archive
 - Log server information in the log file.
"sv_logflush" = "0"
 archive
 - Flush the log file to disk on each write (slow).
"sv_logecho" = "1"
 archive
 - Echo log information to the console.
"sv_log_onefile" = "0"
 archive
 - Log server information to only one file.
"sv_logbans" = "0"
 archive
 - Log server bans in the server logs.
"sv_logsecret" = "0"
 - If set then include this secret when doing UDP logging (will use 0x53 as packet type, not usual 0x52)
"sv_logfilename_format" = ""
 archive
 - Log filename format. See strftime for formatting codes.
"sv_logfilecompress" = "0"
 archive
 - Gzip compress logfile and rename to logfilename.log.gz on close.
"log"
 - Enables logging to file, console, and udp < on | off >.
"logaddress_add"
 - Set address and port for remote host <ip:port>.
"logaddress_delall"
 - Remove all udp addresses being logged to
"logaddress_del"
 - Remove address and port for remote host <ip:port>.
"logaddress_list"
 - List all addresses currently being used by logaddress.
"sv_rcon_log" = "1"
 - Enable/disable rcon logging.
"sv_setsteamaccount"
 - token
Set game server account token to use for logging in to a persistent game server account
"sv_logdownloadlist" = "1"
"sv_lowedict_action" = "0" min. 0.000000 max. 5.000000
 - 0 - no action, 1 - warn to log file, 2 - attempt to restart the game, if applicable, 3 - restart the map, 4 - go to the next map in the map cycle, 5 - spew all edicts.
"spew_consolelog_to_debugstring" = "0"
 - Send console log to PLAT_DebugString()
"joy_axisbutton_threshold" = "0.3"
 archive
 - Analog axis range before a button press is registered.

Game will not start until both teams have players.

Frag1337 commented 3 years ago

I have no idea then..

It's not even only on my CSS/WINDOWS server, it's the same crash condition on my linux ubuntu machine..

Removing OnServerOutput works everything as intended, which is realy weird.

jordanbriere commented 3 years ago

I was able to reproduce it.

jordanbriere commented 3 years ago

So, I've tried a few things without any success. I don't think this is something we can straightforwardly fix on our side and would likely requires some hacky approaches such as storing the outputs and handling them later. This seems to be another of many issues caused by the engine's parallelization. For the same reason you will crash if you hook a send prop's proxy unless you have sv_parallel_packentities 0 in your autoexec.cfg.

CookStar commented 3 years ago

It's not a solution to the underlying problem, but if you want to stop this particular crash or just stop the DataTable warning, I think blocking DataTable_Warning is the easiest way.

https://github.com/CookStar/SmallLib/commit/75ba73e68182487adc34b2287669efc2d40182fb

Patcher is a library for patching binaries without having to worry about plugins bothering each other. https://github.com/CookStar/SmallLib/blob/master/addons/source-python/packages/custom/memorytools/patcher.py

Copy the entire contents of SmallLib and enter sp plugin load dt_warning_blocker. Tested in CS:S/Windows.

Frag1337 commented 2 years ago

Thank you for your effort you two!

Always very appreciated. Should I close this? @jordanbriere

jordanbriere commented 2 years ago

Should I close this? @jordanbriere

We can keep it open until what to do with it is decided. I'm personally not too keen on just knowingly leaving a crash if it can be prevented without server owners having to change their configuration.

NosferatuJoe commented 2 years ago

Hey everyone. Quite interesting to see the same crash issue that I was having. I was just curious how ya'll fixed the "startswith" server crash? Is using DT_warning_blocker together with the patcher.py enough? Not sure what the latter one does though..

NosferatuJoe commented 2 years ago

So, I've tried a few things without any success. I don't think this is something we can straightforwardly fix on our side and would likely requires some hacky approaches such as storing the outputs and handling them later. This seems to be another of many issues caused by the engine's parallelization. For the same reason you will crash if you hook a send prop's proxy unless you have sv_parallel_packentities 0 in your autoexec.cfg.

Sorry for being off-topic on this one but I'm just wondering: Is this why "es playerset model" using a prop crashes all clients on player death?

jordanbriere commented 2 years ago

Hey everyone. Quite interesting to see the same crash issue that I was having. I was just curious how ya'll fixed the "startswith" server crash? Is using DT_warning_blocker together with the patcher.py enough? Not sure what the latter one does though..

CookStar's plugin will prevent the specific context discussed in this thread. However, different contexts may also cause a crash if our logging listener is called from the parallel thread with different outputs. An universal way that would cover all cases would be for you to set sv_parallel_packentities 0 in your autoexec.cfg at the cost of networking optimization losses, unfortunately.

Sorry for being off-topic on this one but I'm just wondering: Is this why "es playerset model" using a prop crashes all clients on player death?

Unrelated. Clients are likely crashing because the model you did set cannot be ragdollified.

CookStar commented 2 years ago

I was just curious how ya'll fixed the "startswith" server crash?

As jordanbriere wrote, dt_warning_blocker only prevents the crash in this particular case. When the DataTable value exceeds the Max or Min value, DataTable_Warning function is called and warning will be output, however, dt_warning_blocker overrides DataTable_Warning function and prevents the execution. As the result, it will not output a warning and OnServerOutput listener will not be dispatched. But be aware this is not suitable if you want to handle the processing at the Python side.

Is using DT_warning_blocker together with the patcher.py enough?

If you have a ServerOutput that can't prevent crashes with dt_warning_blocker, and can reproduce the crashes precisely, please open an issue at https://github.com/CookStar/SmallLib. If possible, I will create a plugin to prevent crashes.

jordanbriere commented 2 years ago

Pushed a fix into: https://github.com/Source-Python-Dev-Team/Source.Python/commit/109ccc6acce6c803e22b9b8f8cf36ae19ad60f6b. Only tested on Windows/CS:S/CS:GO. Here is a CS:S build that no longer crash with the context posted above: core.zip, @Frag1337. If you guys can get it compiled/tested on other games/platforms please let me know how it goes. Thanks!

NosferatuJoe commented 2 years ago

I would love to test it on linux and CSGO but I have no clue how to compile SP. I've already tried with ubuntu 20.4, but I have no linux knowledge and there is no guide as how to. Whenever there's a compiled linux version I'd gladly test it out.

CookStar commented 2 years ago

If you guys can get it compiled/tested on other games/platforms please let me know how it goes.

It compiled and worked fine on CS:S/Windows, but what about Linux? Why not use CThreadMutex?

I would love to test it on linux and CSGO but I have no clue how to compile SP. I've already tried with ubuntu 20.4, but I have no linux knowledge and there is no guide as how to. Whenever there's a compiled linux version I'd gladly test it out.

I don't think this fix is intended for Linux, so we can't try it on Linux.

jordanbriere commented 2 years ago

It compiled and worked fine on CS:S/Windows, but what about Linux? Why not use CThreadMutex?

Pushed that 2 hours ago. 😄

→ https://github.com/Source-Python-Dev-Team/Source.Python/commit/ffef7006f27f2cdb16c41ecee47cdc057f443db7

CookStar commented 2 years ago

Oh, bad timing.

However, the return values of Msg and printf are different, so it cannot be compiled with GCC.

../srcds/Source.Python/src/core/utilities/sp_util.h:55:48: error:
second operand to the conditional operator is of type ‘void’,
but the third operand is neither a throw-expression nor of type ‘void’
   bLogged ? Msg("%s", pMsg) : printf("%s", pMsg);
                                                ^
jordanbriere commented 2 years ago

Oh, bad timing.

However, the return values of Msg and printf are different, so it cannot be compiled with GCC.

../srcds/Source.Python/src/core/utilities/sp_util.h:55:48: error:
second operand to the conditional operator is of type ‘void’,
but the third operand is neither a throw-expression nor of type ‘void’
   bLogged ? Msg("%s", pMsg) : printf("%s", pMsg);
                                                ^

Thanks, should be good now.

Frag1337 commented 2 years ago

@jordanbriere I'm very sorry for the late response.

Its not crashing anymore on CS:S with your updated core.dll (https://github.com/Source-Python-Dev-Team/Source.Python/files/7111413/core.zip). Thank you!

I've used the same test plugin which previously crashed on using test_teleport. But are you sure this error isn't also related to linux? I think I had crashes on Linux aswell, but I will try to test it on Linux aswell once Source.Python is back up again. Sadly I cannot update my servers right now using sp update.

CookStar commented 2 years ago

But are you sure this error isn't also related to linux?

With the new fix, I don't think it will crash on linux. However, I can't say for sure because I have not been able to reproduce the crash on Linux in the first place.

I think I had crashes on Linux aswell, but I will try to test it on Linux aswell once Source.Python is back up again. Sadly I cannot update my servers right now using sp update.

If you want to test server_output on Linux as well, you can try my binaries(core.zip), but if you don't want to do that, you'll have to compile it yourself or wait for @jordanbriere.

P.S. Now I see what you mean.

jordanbriere commented 2 years ago

But are you sure this error isn't also related to linux?

While yes, that platform is also affected, it may or may not crash in this specific context because the behaviours are basically undefined. Based on my minimal testings I did at the time, the problem appeared to be the string being interned by the interpreter and the object being shared for both calls being freed before one or the other had properly incref'ed it resulting into a crash when accessed and/or freed for the second time.

NosferatuJoe commented 2 years ago

But are you sure this error isn't also related to linux?

With the new fix, I don't think it will crash on linux. However, I can't say for sure because I have not been able to reproduce the crash on Linux in the first place.

I think I had crashes on Linux aswell, but I will try to test it on Linux aswell once Source.Python is back up again. Sadly I cannot update my servers right now using sp update.

If you want to test server_output on Linux as well, you can try my binaries(core.zip), but if you don't want to do that, you'll have to compile it yourself or wait for @jordanbriere.

P.S. Now I see what you mean.

Thank you for the files. I'll be testing it tomorrow on my linux csgo server.

NosferatuJoe commented 2 years ago

I can confirm that @CookStar 's binaries for Linux on CSGO (Debian) fixes the issue. My server runs fine without any crashes. I even reverted the changes to the standard .so & bin files to make sure, and it did crash immediately.

So thanks a bunch for this fix, really appreciate the effort <3

Frag1337 commented 2 years ago

Awesome! Also thanks from my side.

Hopefully we can get a new official build with this fix soon!

NosferatuJoe commented 2 years ago

UPDATE:

According to a player report my server crashed multiple times a day and even multiple times in a row. I can confirm that this might've been the case as my server is currently offline due to 10 consecutive crashes in a row. So the fix might not be 100% on Linux, I will investigate this further and provide as much details as I can on this subject.

End of the console.log after last crash before automatically turning off the server:

1RecordSteamInterfaceCreation (PID 8958): SteamGameStats001 / 
terminate called after throwing an instance of 'boost::python::error_already_set'
PreMinidumpCallback: updating dump comment
crash_20210925202156_1.dmp[8994]: Uploading dump (out-of-process)
/tmp/dumps/crash_20210925202156_1.dmp
crash_20210925202156_1.dmp[8994]: Finished uploading minidump (out-of-process): success = yes
crash_20210925202156_1.dmp[8994]: response: Discarded=1
crash_20210925202156_1.dmp[8994]: file ''/tmp/dumps/crash_20210925202156_1.dmp'', upload yes: ''Discarded=1''
Sat 25 Sep 2021 08:36:00 PM CEST - Server has stopped (status: OK)
Sat 25 Sep 2021 08:36:00 PM CEST - Dump file found in /tmp/dumps/. Copying 'dumps' do log path.
Sat 25 Sep 2021 08:36:00 PM CEST - 10 auto-reboot in less than 30 minutes ... ouch. We exit now.
Sat 25 Sep 2021 08:36:00 PM CEST - You can start the service again using the panel.
NosferatuJoe commented 2 years ago

I can also upload a crashdump.dmp but I need a trustworthy fileshare service. If needed recommend me one and I'll upload it right away.

jordanbriere commented 2 years ago

Hopefully we can get a new official build with this fix soon!

The latest build includes https://github.com/Source-Python-Dev-Team/Source.Python/commit/e6920e13b969d10d2112770af9c3fe64ab8a3922 which addresses the crash caused by this issue. You won't be able to filter outputs that are sent from a parallel thread anymore, but it won't crash as they are simply ignored.

I can also upload a crashdump.dmp but I need a trustworthy fileshare service. If needed recommend me one and I'll upload it right away.

Your crash doesn't seem to be related. The issue discussed in that thread doesn't throw a boost::python::error_already_set error whatsoever. Moreover, this error should not really cause the process to terminate as it is usually picked up and forwarded to the Python interpreter translated to an exception. The only scenario I know this would happens, is if you are running SP along a specific SM extension (#321). As for your dumps, just zip them and attach the zip directly to your post here.

NosferatuJoe commented 2 years ago

Here the latest crash dump that should be linked to the console.log snippet I sent: crash_20210925202156_1.zip

Very weird since I'm not running accelerator. I just installed the latest available SP build and then added Cookstar's binaries on top.

jordanbriere commented 2 years ago

I just installed the latest available SP build and then added Cookstar's binaries on top.

It doesn't really matter this time since there was no critical changes between both branches (besides some fixes on the wiki) in that time frame but for future reference; this is something you should never do. If you use binaries compiled from a specific branch, then you should use everything else because some changes on the Python side might be incompatible with some on the C++ side or vice-versa.

That said, it's very unlikely that your crash loop is caused by this issue. Easy to confirm, though. Add the following to your autoexec.cfg: sv_parallel_packentities 0 and if you still crash it means it has nothing to do with it. You should then track the issue by elimination; disable SM, test, disable all your SP plugins, test, enable one plugin, test, another one, test, etc. until the crashes come back so you know where to start looking.

NosferatuJoe commented 2 years ago

I just installed the latest available SP build and then added Cookstar's binaries on top.

It doesn't really matter this time since there was no critical changes between both branches (besides some fixes on the wiki) in that time frame but for future reference; this is something you should never do. If you use binaries compiled from a specific branch, then you should use everything else because some changes on the Python side might be incompatible with some on the C++ side or vice-versa.

That said, it's very unlikely that your crash loop is caused by this issue. Easy to confirm, though. Add the following to your autoexec.cfg: sv_parallel_packentities 0 and if you still crash it means it has nothing to do with it. You should then track the issue by elimination; disable SM, test, disable all your SP plugins, test, enable one plugin, test, another one, test, etc. until the crashes come back so you know where to start looking.

So that means, that the fixes are already applied on the 21 september build? I have that one running now and will let you know if it still crashes after I use the onserveroutput code

NosferatuJoe commented 2 years ago

I can confirm that this piece of code in the WCS mod does not crash the server anymore on linux CSGO after using latest SP build:

# Is ESC supported?
if IS_ESC_SUPPORT_ENABLED:
    #  Used to clean up loading/unloading of wcs
    @OnServerOutput
    def on_server_output(severity, msg):
        if msg.startswith(('[EventScripts] Loaded wcs/modules/races/', 'Unloading wcs/modules/races/')) or (msg.startswith('wcs/modules/races/') and msg.endswith(' has been unloaded\n')):
            return OutputReturn.BLOCK
        if msg.startswith(('[EventScripts] Loaded wcs/modules/items/', 'Unloading wcs/modules/items/')) or (msg.startswith('wcs/modules/items/') and msg.endswith(' has been unloaded\n')):
            return OutputReturn.BLOCK

        return OutputReturn.CONTINUE
IMPORTANT: Please copy the full output.
--------------------------------------------------------
Checksum      : b9ee5c091e712e73e470b2e014ea2225
Date          : 2021-09-26 12:39:19.538046
OS            : Linux-5.9.0-1-amd64-x86_64-with-debian-10.10
Game          : csgo
SP version    : 708
Github commit : 70e71fe6531e655fd164d40bea7c838c447199c0
Server plugins:
   00: Source.Python, (C) 2012-2021, Source.Python Team.
   01: Metamod:Source 1.11.0-dev+1145
SP plugins:
   00: adaptive_radar
   01: es_emulator, https://github.com/Ayuto/EventScripts-Emulator
   02: wcs, 2021.03.30-1, https://forums.sourcepython.com/viewtopic.php?f=7&t=1925
   03: commandsx
--------------------------------------------------------
jordanbriere commented 2 years ago

So that means, that the fixes are already applied on the 21 september build?

The official build includes a patch that prevent the crash by no longer processing outputs that are not from the main thread while the server_output branch process all outputs but ensure there is no crash by locking the shared resources (the interned PyObject of the output itself) so that simultaneous calls don't release memory until the others are done with it. So yes, both should no longer crash when multiple threads send the exact same output at the exact same time.