Closed czarintax closed 3 years ago
Try implement all you need from engine with physics interface or directly. sv/svs stucts are not stable and will break between builds, and will not work if plugin not developed specially for xash3d. SV_DropClient may be implemented with "kick" command Cvar_DirectSet by operating cvar_t struct
I don't think it's good idea to follow signatures from GoldSrc.
If we will ever have better compatibility with AMXModX, it should be done with ReHLDS API.
Try implement all you need from engine with physics interface or directly. sv/svs stucts are not stable and will break between builds, and will not work if plugin not developed specially for xash3d. SV_DropClient may be implemented with "kick" command Cvar_DirectSet by operating cvar_t struct
Thanks for the signatures. I found the rest of the signatures and now it seems to work without any errors.
BTW it is always possible to find required fields using dwarf/pdb data or sh symtabs in unstripped executable
BTW it is always possible to find required fields using dwarf/pdb data or sh symtabs in unstripped executable
I'm sorry, but I'm not too familiar with this kind of things. I mean, because i'm a little new to this, i don't even get the names you've defined. But i'll try to research about it. I really want to get to understand.
Meanwhile, pardon me but i have another more question. What's the trick to make HLStatsX:CE get to work? Because although it is properly configured, it doesn't show any information(0/0 players). I'm not sure if it's engine related or something else but once before i saw a server used to run it smoothly.
2020-05-10 09:07:03: - MYSQL: Connecting to MySQL database 'hlstatsx' on 'localhost' as user 'root' ... connected ok 2020-05-10 09:07:03: - CONFIG: Reading database config... 2020-05-10 09:07:03: - CONFIG: I have found the following server configs in database: 2020-05-10 09:07:03: - S_CONFIG: 51.144.248.71:27015 2020-05-10 09:07:03: - ERROR: GeoIP method set to binary file lookup but Geo::IP::PurePerl module NOT FOUND 2020-05-10 09:07:03: - HLSTATSX: HLstatsX:CE 1.6.19 starting... 2020-05-10 09:07:03: - UDP: Opening UDP listen socket on 10.0.0.5:27500 ... ok 2020-05-10 09:07:03: - HLSTATSX: Tracking Trend of the stats are enabled 2020-05-10 09:07:03: - HLSTATSX: Maximum Skill Change on all servers are 25 points 2020-05-10 09:07:03: - HLSTATSX: Minimum Skill Change on all servers are 2 points 2020-05-10 09:07:03: - HLSTATSX: Minimum Players Kills on all servers are 50 kills 2020-05-10 09:07:03: - HLSTATSX: Players chat logging is enabled 2020-05-10 09:07:03: - HLSTATSX: Broadcasting public chat to all players is enabled 2020-05-10 09:07:03: - HLSTATSX: Event queue size is set to 10 2020-05-10 09:07:03: - HLSTATSX: HLstatsX:CE is now running (Normal mode, debug level 1) 2020-05-10 09:09:01: - HLSTATSX: No data since 120 seconds 2020-05-10 09:11:01: - HLSTATSX: No data since 120 seconds 2020-05-10 09:12:05: - HLSTATSX: Insert new server trend timestamp
@Czarintax can you link to HLStats source code?
We have pretty much differences with GoldSrc here, from undocumented parts like signatures to "sort of" documented like server queries. I doubt this mod uses engine API to fetch data otherwise it would just work.
mittorn said:
Try implement all you need from engine with physics interface or directly. sv/svs stucts are not stable and will break between builds, and will not work if plugin not developed specially for xash3d.
That's why it's important to make efforts (you, developers behind Xash3D FWGS), to try not break such important structures (server & server_static for example), for third-party mods addons (mainly AMX & AMXX). By at least try not modifying/removing actual content (unless you have no other choice), and, when you need to add new members, add them to the end (something which has not been done on the previous & abandonned Xash3D FWGS build). I do not think it's much problematic neither much asked, if you care a bit about such addons & users (& their developers, to avoid a bit at making their life harder) which are very much used and almost mandatory on game servers to extend management possibilities at a higher level. Those both structures contain precious informations that could be used for useful plugins (resources informations, etc.).
@StevenKal I think I've discussed this with you before. No one should access private parts of engine.
Yes, we discussed about that before via mail. For me you seem to use the "private" argument of a struct as an excuse for "do not access/touch the engine structs", but this is not an excuse for me. For me, private or public does not matter much.
I remember you that I did not asked you to "freeze them" (do not touch anything), but just update them "intelligently", which means do not modify existing types (bool to int for example), size (when array) and position of existing members of the structs (before/after), except when you have no other choice (as you need to enhance the size of a member array for extending).
However, if you do not care to respect my wishes, so be it, but in such case you probably did not much worked on your own third-party modules (like me with AMX Mod), or almost never, and do not care at making the life easier to developers like me who may try to add support for the good Xash3D. Because if I use such private struct members in my addon, and, you break them, you just annoy me and force me to waste my time at updating my stuff, and you also annoy the users who might use the addon, because the existing addon version they have do not work or crash due to that, until they update it to a patched version I'll release once I'm aware of your changes.
Also, such structures like "sv_client_t", "server_t", "server_static_t" can be really useful for programmers like me and provide more programming possibilities to users (exclusive features). For example, in AMX, we can make a hook of "SV_DropClient" in order to catch any disconnect call (since the "pfnClientDIsconnect" is not called when the is not yet in-game, at least under GoldSrc, Svengine, ReHLDS engines), and the function passes an "sv_client_t" pointer, and in AMX, we need some members of those structures in order to convert that into an ID usable for AMX... We can also use some members of the "server_t" structure for making an "Engine's Resources Informations" plugin, etc..
And if you really do not want to make efforts at not breaking such structures the way I've asked, and make life of guys like me easier, probably it will be nice to add an API, as you talked about in the email we exchanged. And if you provide direct access to those members, the best for me is via address (like "return &svs.maxclients"), so we can get or set the value from one function (no need two, one to get it, one to alter it, this make API functions listing shorter & so, simpler). I do not think you have to wait Uncle Mike's agreement for this... since your Xash3D is a fork where you change that you feel according to your needs and users's requests, so you can start with your own Xash3D FWGS's API the way you wish, or try discuting about this addition with him, but if he do not have time to make it, is lazy to do it, or just do not care (as for multiplayer & Linux part sadly!), start your own, because you do not know when he will do it and if he will.
PS: I checked the "server.h" file of your latest v0.20, and compared to the original Uncle Mike's Xash3D version (the b4344), I've seen you for example, added two new members to the "_sv_clientt" struct at the end ("extensions" & "useragent"). It's good to do as you did, I have almost nothing against new additions at the end (except when they are in substructures and might break others when they are referenced as "structure format rather than pointer", which changes the size like an array [like if you modify "server_log_t", this will affect size of "server_static_t" from "log" member]), and I hope you will continue doing "this way" for the possible new members you will add.
It is better to add some interface for admin modules instead of reading engine structures make single export linke it done in renderapi. for example, SERVER_GET_PARM(SV_STATIC_MAXCLIENTS, 0), SERVER_GET_PARM(SV_CLIENT_USERAGENT, 3) There are no such export in physics api and maybe we need to do separate api for that
Yes, and it will be welcomed for reliability with updates! But as long as there is no API for third-party addons, admin modules programmers have no other choice than using some "memory hacking" as workaround.
Yes, we definitely need an extended API but at first we must hit stable point, get MetaMod-P running, get AMX Mod X pass the testsuite. I've did it with old engine already.
Still, I don't think memory hacking is universal. It is delusion if you think you can rely on it. I'm not gonna announce breaking changes.
Well, what I can recommend... use tools like abidiff
. It helps you compare the ABI of two binaries, from removed functions to changed structure layout.
Hello developers,
I have acquired a couple of problems, so i would like to share it with you guys.
First of all, i want to ask why DHUD messages causes clients to crash? So instead of DHUD, normal HUD must be used, otherwise clients get crashed. Also normal HUD messages are not stable enough. They're constantly flickering and the channels are getting mixed up if there is multiple messages sent in a same period.
Second of, i get kicked from the server with svc_bad error when using any kill assist plugin. I gave a look at the source code and i have concluded that these lines occurs the error.
message_begin(MSG_ALL, SVC_UPDATEUSERINFO);
write_byte(killerEntIndex - 1);
write_long(get_user_userid(killerEntIndex));
write_char('\');
write_char('n');
write_char('a');
write_char('m');
write_char('e');
write_char('\');
write_string(name);
for(new i = 0; i < 16; i++)
write_byte(0);
message_end();
Yes, we definitely need an extended API but at first we must hit stable point, get MetaMod-P running, get AMX Mod X pass the testsuite. I've did it with old engine already.
Well, I'm using the Metamod version for Xash3D found on moddb, I did not figured out weird issues with it (I mean, issues which "jump to my eyes and make its use critical/unstable when I'm playing"). But for the AMX Mod [X] addons, people should never confound "running" (as "meta list" and "amx[x] plugins" returns "all running"...) and "working properly" (working fine when functions from core/modules/plugins are called, which is far to be the case without adjustements, sometimes deep ones). As they require some internal data & adjustements to be updated & adapted for Xash3D versions, and some plugins (like the kick system being different & bad [using ID - 1...] under the standard UM's Xash3D, some messages having different formats/parameters, etc.) need short or even deep rework too.
Still, I don't think memory hacking is universal. It is delusion if you think you can rely on it.
In my opinion, it is, and it can be very reliable, but only when the following conditions are reunited: o Internal data (signatures/symbols) properly maintained according to engine/game versions (takes some times & "determination" to handle all of this the right way, but possible if we want to). o No function "split in two" (something like) for hooks and so hooks not properly hookable via the conventional way (case sometimes for "just a few functions" in the official engine/game binaries, but case for almost everything in the ReHLDS engine, sadly, which makes it a terrible engine for programming possibilities in my opinion). o Usage of symbols for more safety/reliability (extra "facultative condition" for better reliability), usually the case everytime under Linux, but unfortunately not under Windows, but for sure I will like to have the "EXPORT" label for all the functions like some in game binaries ("CBaseEntity::SUB_CallUseToggle", etc.) rather than making signatures of bytes which is very long.
An "hardcoded API" (like the g_engfuncs, etc.) is more reliable, but not fully too depending on some conditions, it can also be broken in case of the developers changes some formats, or, the listing order, which is not the case usually but this may happen (for example, on the old versions of ReGameDLL_CS < v5.*). At the end, memory hacking possibilities remains a way much more incredible than an hardcoded API, look, in the binaries there are a bunch of variables, arrays, functions, sometimes it's about "hundred or even thousand of such", and memory hacking can catch & hook almost all of this, seriously, the end possibilities are very amazing, and probably no "hardcoded API" contains all of this, because it will be very long to implement and devs just do not do that.
Well, I have mainly written this in order to remember you devs how useful & powerful memhack can be, in an attempt at make you recall to try not breaking this as much as you can. However, I guess you already know all of that I said above!
I'm not gonna announce breaking changes.
What do you mean exactly? Do you mean "you are not gonna inform the people about possible changes that will break memory hacking", or, "you have no plans to break structures, etc."?
Well, what I can recommend... use tools like
abidiff
. It helps you compare the ABI of two binaries, from removed functions to changed structure layout.
I usually compare the structures from the sources you provide and via IDA (decompiled binaries), and also test myself in-game with debugging prints/logs or plugins's commands, but I'll take a look at this tool I did not know about it. Thanks for the advice.
@Czarintax: Why you are specifying ReHLDS in the top while you post on Xash3D FWGS engine repository? I guess you just mistaken right?
@Czarintax DHUD had problems in cs16-client. Do not use it.
I also do not recommend poking into SVC messages that are NOT used by HLSDK: https://github.com/FWGS/hlsdk-xash3d/blob/a758e80f71a3ce124b009ccc5fc8aca9cb66a57f/dlls/util.h#L463
In your example, you're writing an SVC_UPDATEUSERINFO which have completely different structure from what GoldSrc uses: https://github.com/FWGS/xash3d/blob/d2ef74ec17bedf4f734a028ffc4cd462eaa8aea6/engine/server/sv_client.c#L1128
I see there a big hole of incompatibility between AMXX and Xash3D. Again, first is written exclusively for GoldSrc, but the Xash3D have advanced message buffers that allows to write data by bits, rather than bytes as GoldSrc. I'm sure it's not allowed for server DLL to write data by bits. So, yeah, don't do that.
@StevenKal
properly maintained according to engine/game versions
That's what I'm trying to say. It is not and we're not gonna waste our time to maintain parts that you are not allowed to look into.
Usage of symbols for more safety/reliability (extra "facultative condition" for better reliability), usually the case everytime under Linux, but unfortunately not under Windows
We're compiling Xash3D with -fvisibility=hidden for better compatibility so you already should miss these symbols.
I know how memhack can be useful. But we're not proprietary software and moreover are not abandoned(unlike 22 year old GoldSrc). We need right tools for modders rather than unreliable memory hacking.
Do you mean "you are not gonna inform the people about possible changes that will break memory hacking", or, "you have no plans to break structures, etc."?
Kinda both. I don't think we will ever break these structures too often but I don't want to give you a hope that is not gonna happen.
Can't find correct signature for Cvar_DirectSet. Please, can anybody help me. I really need to make hook_cvar_change to work. Also can't autoexec config files because of it. BTW i'm using the old engine because of CS16Client.
"Cvar_DirectSet" // void Cvar_DirectSet(struct cvar_s *var, char *value);
{
"library" "engine"
"windows" "\x55\x8B\x2A\x81\x2A\x2A\x2A\x2A\x2A\x56\x8B\x2A\x2A\x57\x8B\x2A\x2A\x85"
"linux" "@Cvar_DirectSet"
"mac" "@Cvar_DirectSet"
}
@a1batross:
The fact is, untill you add an API (we do not know how long we will have to wait for such), this is not complicated and neither long to change things like:
void Cvar_DirectSet( convar_t *var, const char *value )
Into:
EXPORT void Cvar_DirectSet( convar_t *var, const char *value )
So we could easily find that function via symbol in a binary, for both Windows/Linux OS... and have realiability with updates, rather than using signatures of bytes which might be broken later and will require an update, this is that I'm trying to explain in my last "o" point! You take like one hour to list in a .txt file most of the functions you think addons/plugins developers like me might need, then you add that label. Also, knowing the following issue I'm talking about, I might recommend to do that for the g_engfuncs too.
Also, by searching global variables addresses recently, I figured out a weird thing/issue.
You know, for example at trying to find address of global variable like "svs", "sv", etc., I use the address of a function, so usually an engine functions of the API (g_engfuns) for simplicity, then get the "memory value" with an offset (offset is the value matching to the byte position where the variable is in the function), then I got the address of such variables, very useful for my stuff after that...
But, I'm wondered why the hell I got weird values outside the range with your engine, and I after that figured out that the address of a function like (void *)g_engfuncs.pfnGetCurrentPlayer
is not the same than the one we can get via memory hacking by searching with a signature of bytes (the memhack one is correct for my needs, but the g_engfuncs.* is fucked up for it), while it is THE SAME for other engines without problems.
I did not take time to figured out that's wrong, if this is a compiler option issue you use, Metamod's issue, or anything else, and I only tested under Windows, but that I can tell is that this only happens with YOUR Xash3D FWGS engine, I do not have such problem with all the other engines including the standard Xash3D by Uncle Mike, so I invite you to test yourself this quickly if you have some time for this. I'm not sure I can expect you to test it since you seem a bit against memory hacking, but well, I think this worth to take a look, because it's weird for me.
I also precise this happens on the older versions (v0.19.2...), not only the v0.20.
I've also spotted some weird issues in the v0.20, as the "-console" parameter not working for me, the fact the engine is very long to launch (around 15 seconds), same thing when I press the Windows key to go back to desk, I have a big white box instead of the "titles" in the top ("Configuration", "Multiplayer", etc.), and a few other troubles.
@Czarintax rebuild engine without -fvisibility=hidden. It's enforced in CMakeLists: https://github.com/FWGS/xash3d/blob/master/engine/CMakeLists.txt#L74 But I don't guarantee strange bugs will not appear since HLSDK and engine have same symbols names and they may conflict.
I hope you're running on Linux of course. Also library name must be replaced but I'm not sure tbh. Try it anyway.
@StevenKal I'm fine with just adding exports. Send a PR with them and we'll discuss it further.
searching global variables addresses
You... may not do it actually. You can declare these variables with EXPORT macro and dlsym them as well as functions. I think it must be more portable for your goal.
The bug you've explained may really happen because of the compilers. Unkle Mike builds his engine with MSVC6 which isn't really smart compiler(maybe actually the dumbest one). It is not used for our release builds.
For other problems, create new issue reports there.
Wontfix.
"Wontfix", are you talking about the issue itself (like providing signatures to the guy who requested that), or about "adding an API/exports" to Xash3D FWGS (what we talked together)?
By the way, what about that API, is it shill scheduled among your plans, and if yes, when do you think about adding it?
Signature-based modding is just big no-no for us. I would never support that.
New exports could be added but I don't really know what you guys need. As I suggested, you can open a new issue/PR, where we can discuss it further. I really would like to see AMX Mod X ~garbage~ stuff working on Xash but it's too complex and it needs proper cross platform solution.
Unfortunately, I probably lost my amxmodx fork that I patched to make it working on ARM devices with Xash. 🤷🏼
Well, exporting some global variables like "svs", "sv", and some others (cvar_vars, host...), also try to "not break" related structures (server_static, server...), I know we discussed that before and I heard your opinion about, but the fact it is not "public" is not a solid argument to break them without caring, there are a bunch of them which can be very useful for programmers and extend current possibilities, not all of the members can really be useful, but there are, like on the "sv_client_t", we can use the "pViewEntity" to get in-live the current view entity ID attached to a client (as a camera, etc.), same thing for some members of "server_t" (as checking resources...), etc.. Just to say that I do not ask you to "do not touch them", just add new members at the suite if you have to, on the previous v0.19.* you changed (even "fucked up"!!) a tons of things regarding this (this sounded like a complete rebuild of the Uncle Mike's Xash3D), while the latest v0.20 version seem more "clean", I mean, I have not looked compared in details, but it seems you kept some original parts but "extend" a lot by adding your features the right way. So seems better at first view at least. However, if you have to change the size of an existing member (like if it is an array and you redefine it) and can not do otherwise, so be it, but for new additions, add them at the suite. I know I insist a bit, but this is because this reduce my work for the future things I have plans to do & release, which might partially use such structures.
About the functions, well, a bunch can be exported, just if I look at "server.h" and at some other files, I see a bunch of them which could be exported, like: Cvar_DirectSet SV_DropClient SV_GetEntityClass (like "GetEntityInit" from that I recall) SV_FreeEdict SVUpdateMovevars SV*Index SV_SendResource[List] SV_ActivateServer SV_SpawnServer SV_ModelHandle SV_DeactivateServer SV_ClientPrintf Con_Printf Some related to commands (SV_Kick_f, SV_Status_f, etc.) Cbuf_Execute Cmd_ExecuteString Cmd_Exists Cmd_TokenizeString Host_Error SV_GetClientIDString SV_ClientBy<Id|Name> SV_FullClientUpdate SV_ExecuteClientMessage SV_UpdateServerInfo SV_RejectConnection SV_AddToResourceList SV_SendResources SV_SpawnEntities SV_StartSound
And maybe more of them...
Also, I remember seeing some kind of "partial API" for the file system, as Xash3D is not using exact original GoldSrc system, there were some kind of API at the end of "filesystem.c" file (g_fsapi exported via "FS_GetAPI" function). You could redo it like it was but fully filled, and properly export it. VALVe's file system searching at through other games paths (fallback & original "valve" directory) is good for various files (as precached files such as models, sounds, etc.).
Despite this could look "long", adding "EXPORT" label to all of those previous variables & functions I listed will probably take you like one hour, and once done it is done! You will not need to go back! Hehehe! You could also build a structure with a functions list, a bit like guys behind Re* binaries did, and export it, but in my opinion and for my needs, adding that "EXPORT" label in order to see it via IDA in "exports" tab and find the address easily in a reliable way via "symbols", is very enough and I even prefer like this. And about "providing hooks", well, while you do not fuck up your engine with "shitty compilers" like ReHLDS is victim, we can "memhack" those functions with ease and without troubles and I like that too. Of course this could stil be broken if in case you change like, the format (parameters) of a function, but well, this will be the same for an "hardcoded API" where major should be changed in such case so...
But after that, for the rest, I think you can "think yourself" about which functions could be useful for third-party addons, then export them yourself, as this can be done quickly for you.
Thanks in advance.
@Czarintax rebuild engine without -fvisibility=hidden. It's enforced in CMakeLists: https://github.com/FWGS/xash3d/blob/master/engine/CMakeLists.txt#L74 But I don't guarantee strange bugs will not appear since HLSDK and engine have same symbols names and they may conflict.
I hope you're running on Linux of course. Also library name must be replaced but I'm not sure tbh. Try it anyway.
@StevenKal I'm fine with just adding exports. Send a PR with them and we'll discuss it further.
searching global variables addresses
You... may not do it actually. You can declare these variables with EXPORT macro and dlsym them as well as functions. I think it must be more portable for your goal.
The bug you've explained may really happen because of the compilers. Unkle Mike builds his engine with MSVC6 which isn't really smart compiler(maybe actually the dumbest one). It is not used for our release builds.
For other problems, create new issue reports there.
I get this error on compile:
[100%] Linking C executable xash3d /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libSDL2.so when searching for -lSDL2 /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libSDL2.a when searching for -lSDL2 /usr/bin/ld: cannot find -lSDL2 /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libpthread.a when searching for -lpthread collect2: error: ld returned 1 exit status engine/CMakeFiles/xash.dir/build.make:2122: recipe for target 'engine/xash3d' failed make[2]: [engine/xash3d] Error 1 CMakeFiles/Makefile2:85: recipe for target 'engine/CMakeFiles/xash.dir/all' failed make[1]: [engine/CMakeFiles/xash.dir/all] Error 2 Makefile:149: recipe for target 'all' failed make: *** [all] Error 2
install the 32-bit versions of the libraries?
install the 32-bit versions of the libraries?
totally forgot about that ty
server crashes with the following error:
assert failed at /home/$USER/xash3d/engine/server/sv_client.c:738
PS: Is it possible to verify users game files with MD5 to prevent simple cheating by replacing smoke sprites etc. ?
I need appropriate signatures of engine for gamedata files to run AMX Mod X without errors.
Metamod version 1.21p38
AMX Mod X version 1.10.0.5392-dev
ReGameDLL version 5.16.0.460-dev
ReHLDS version 3.7.0.695-dev
Console log:
Signature corrections(for linux) that i need is below.
functions.engine.txt
globalvars.engine.txt