search

Source-Python-Dev-Team / Source.Python

This plugin aims to use boost::python and create an easily accessible wrapper around the Source Engine API for scripter use.
http://forums.sourcepython.com
GNU General Public License v3.0
164 stars 32 forks source link

Added feature to check data signatures at Source.Python setup. #467

Open CookStar opened 1 year ago

CookStar commented 1 year ago

This will allow us to deal with signature changes more quickly and accurately than before.

Output at https://github.com/Source-Python-Dev-Team/Source.Python/commit/9f387830a037fffd7b4dc9d680ddb7e4ac19e767

Output ``` [Source.Python] [SP] Encountered a Warning: File '../addons/source-python/packages/source-python/__init__.py', line 168: UserWarning Invalid signature detected in class data, specific function will not work. [Source.Python] Invalid signature detected. Name: set_attacker Signature: 55 89 E5 57 56 53 83 EC 2C C7 45 E4 00 00 00 00 8B 5D 08 8B 75 0C File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CBaseCSGrenadeProjectile.ini [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 5C 8B 75 08 C7 44 24 08 00 00 26 43 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSmokeGrenadeProjectile.ini [Source.Python] Invalid signature detected. Name: fire_output Signature: 55 89 E5 57 56 53 81 EC 7C 01 00 00 8B 55 08 8B 75 14 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entity_output/CBaseEntityOutput.ini [Source.Python] Invalid signature detected. Name: add_account Signature: 55 89 E5 57 56 53 81 EC 2A 01 00 00 8B 5D 08 0F B6 45 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini [Source.Python] Invalid signature detected. Name: buy_internal Signature: 55 89 E5 83 EC 78 0F B6 55 14 89 75 F8 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini [Source.Python] Invalid signature detected. Name: deafen Signature: 55 89 E5 83 EC 48 89 7D FC 8B 7D 08 89 5D F4 89 75 F8 8B 07 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini [Source.Python] Invalid signature detected. Name: _spawn Signature: 55 89 E5 57 56 53 83 EC 3C A1 2A 2A 2A 2A 8B 5D 08 89 04 24 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini [Source.Python] Invalid signature detected. Name: switch_team Signature: 55 89 E5 83 EC 68 89 75 F8 8B 75 0C 89 5D F4 8B 5D 08 89 7D FC 89 34 24 E8 2A 2A 2A 2A 85 C0 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini [Source.Python] Invalid signature detected. Name: start_timer Signature: 55 89 E5 53 83 EC 34 8B 5D 08 F6 83 2A 00 00 00 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 3C 8B 5D 08 89 1C 24 E8 2A 2A 2A 2A 85 C0 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini [Source.Python] Invalid signature detected. Name: do_detect_wave Signature: 55 89 E5 57 56 53 81 EC BC 01 00 00 8B 55 08 89 14 24 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 3C 8B 5D 08 F6 83 2A 00 00 00 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CDecoyProjectile.ini [Source.Python] [SP] Encountered a Warning: File '../addons/source-python/packages/source-python/__init__.py', line 185: UserWarning Invalid signature detected in global pointers data, may cause problems with Source.Python operation. [Source.Python] Invalid signature detected. Name: CEntityFactoryDictionary Signature: 55 89 E5 83 EC 28 80 3D 2A 2A 2A 2A 00 74 2A C9 B8 2A 2A 2A 2A C3 8D 76 00 8D BC 27 00 00 00 00 C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A 85 C0 74 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A C7 44 24 08 2A 2A 2A 2A C7 44 24 04 2A 2A 2A 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A B8 2A 2A 2A 2A C9 C3 89 45 F4 C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A 8B 45 F4 89 04 24 E8 2A 2A 2A 2A 90 8D 76 00 8D BC 27 00 00 00 00 55 89 E5 57 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini [Source.Python] Invalid signature detected. Name: CGlobalEntityList Signature: E8 2A 2A 2A 2A E8 2A 2A 2A 2A C7 2A 2A 2A 2A 2A 2A E8 2A 2A 2A 2A E8 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini [Source.Python] Invalid signature detected. Name: IServer Signature: 55 89 E5 53 83 EC 14 8B 45 0C C7 04 24 2A 2A 2A 2A File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini ```
Logs ``` 2023-02-14 16:40:10 - sp.hooks.warnings - WARNING [Source.Python] [SP] Encountered a Warning: File '../addons/source-python/packages/source-python/__init__.py', line 168: UserWarning Invalid signature detected in class data, specific function will not work. 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: set_attacker Signature: 55 89 E5 57 56 53 83 EC 2C C7 45 E4 00 00 00 00 8B 5D 08 8B 75 0C File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CBaseCSGrenadeProjectile.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 5C 8B 75 08 C7 44 24 08 00 00 26 43 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSmokeGrenadeProjectile.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: fire_output Signature: 55 89 E5 57 56 53 81 EC 7C 01 00 00 8B 55 08 8B 75 14 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entity_output/CBaseEntityOutput.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: add_account Signature: 55 89 E5 57 56 53 81 EC 2A 01 00 00 8B 5D 08 0F B6 45 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: buy_internal Signature: 55 89 E5 83 EC 78 0F B6 55 14 89 75 F8 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: deafen Signature: 55 89 E5 83 EC 48 89 7D FC 8B 7D 08 89 5D F4 89 75 F8 8B 07 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini 2023-02-14 16:40:10 - sp - WARNING [Source.Python] Invalid signature detected. Name: _spawn Signature: 55 89 E5 57 56 53 83 EC 3C A1 2A 2A 2A 2A 8B 5D 08 89 04 24 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: switch_team Signature: 55 89 E5 83 EC 68 89 75 F8 8B 75 0C 89 5D F4 8B 5D 08 89 7D FC 89 34 24 E8 2A 2A 2A 2A 85 C0 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CCSPlayer.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: start_timer Signature: 55 89 E5 53 83 EC 34 8B 5D 08 F6 83 2A 00 00 00 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 3C 8B 5D 08 89 1C 24 E8 2A 2A 2A 2A 85 C0 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: do_detect_wave Signature: 55 89 E5 57 56 53 81 EC BC 01 00 00 8B 55 08 89 14 24 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CSensorGrenadeProjectile.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: detonate Signature: 55 89 E5 57 56 53 83 EC 3C 8B 5D 08 F6 83 2A 00 00 00 10 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/entities/CDecoyProjectile.ini 2023-02-14 16:40:11 - sp.hooks.warnings - WARNING [Source.Python] [SP] Encountered a Warning: File '../addons/source-python/packages/source-python/__init__.py', line 185: UserWarning Invalid signature detected in global pointers data, may cause problems with Source.Python operation. 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: CEntityFactoryDictionary Signature: 55 89 E5 83 EC 28 80 3D 2A 2A 2A 2A 00 74 2A C9 B8 2A 2A 2A 2A C3 8D 76 00 8D BC 27 00 00 00 00 C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A 85 C0 74 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A C7 44 24 08 2A 2A 2A 2A C7 44 24 04 2A 2A 2A 2A C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A B8 2A 2A 2A 2A C9 C3 89 45 F4 C7 04 24 2A 2A 2A 2A E8 2A 2A 2A 2A 8B 45 F4 89 04 24 E8 2A 2A 2A 2A 90 8D 76 00 8D BC 27 00 00 00 00 55 89 E5 57 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: CGlobalEntityList Signature: E8 2A 2A 2A 2A E8 2A 2A 2A 2A C7 2A 2A 2A 2A 2A 2A E8 2A 2A 2A 2A E8 File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini 2023-02-14 16:40:11 - sp - WARNING [Source.Python] Invalid signature detected. Name: IServer Signature: 55 89 E5 53 83 EC 14 8B 45 0C C7 04 24 2A 2A 2A 2A File: /home/steam/csgo_ds/csgo/addons/source-python/data/source-python/memory/global_pointers.ini ```
jordanbriere commented 1 year ago

It works great and I really like the idea!

That said, I'm not sure that is something that should be done by default. I can see the usefulness when updating the data, but I don't think all signatures should be validated and cached on every servers. Perhaps we could add an additional setting to core_settings.ini to explicitly enable and perform the validation? That would overall seem better to me, since I don't think the average user care if a signature they don't use is outdated.

As for the implementation itself, I don't necessarily see the need to re-invent the wheel and manually parse the files. What about using a temporary TypeManager instance instead?

CookStar commented 1 year ago

That said, I'm not sure that is something that should be done by default. I can see the usefulness when updating the data, but I don't think all signatures should be validated and cached on every servers. Perhaps we could add an additional setting to core_settings.ini to explicitly enable and perform the validation? That would overall seem better to me, since I don't think the average user care if a signature they don't use is outdated.

The problem is that the user doesn't know which function's signature is incorrect. Rather than actually running the code path of the incorrect function and then realizing that the signature is incorrect, it is more useful to notify the user from the beginning that there is a problem with the signature, so that the problem can be resolved quickly. I also don't see the problem to notify all users with the invalid signatures since they may not be familiar with the signatures themselves.

Also, why is it such a problem to have signatures cached? I measured signatures that are not used by SP itself and only 775 bytes were used. If there is a problem with caching, I think m_Signatures should be changed to something like unordered_map.

As for the implementation itself, I don't necessarily see the need to re-invent the wheel and manually parse the files. What about using a temporary TypeManager instance instead?

check_pipe_signature_from_file and check_type_signature_from_file are not only for Source.Python, but also for plugin authors to check signatures. Since there is no way to handle errors without changing TypeManager's behavior, duplicate code is unavoidable.

jordanbriere commented 1 year ago

duplicate code is unavoidable.

How is it unavoidable? Let's say you want to validate the functions of a type, all you really have to do is ensure all attributes resolve properly:

# ../test.ini

binary = server

[function]
    [[foo]]
        identifier = 11 22 33 44 55 66 77 88 99 00

    [[bar]]
        identifier = 11 22 33 44 55 66 77 88 99 00

    [[baz]]
        identifier = 11 22 33 44 55 66 77 88 99 00

from memory.manager import TypeManager
from path import Path

Foo = TypeManager().create_type_from_file(
    'Foo',
    Path(__file__).parent / 'test.ini'
)

for attr in vars(Foo):
    try:
        getattr(Foo, attr)
    except Exception as e:
        print(f'Foo.{attr} -', e)
[SP] Loading plugin 'testing'...
Foo.foo - Could not find signature: 11223344556677889900
Foo.bar - Could not find signature: 11223344556677889900
Foo.baz - Could not find signature: 11223344556677889900
[SP] Successfully loaded plugin 'testing'.
CookStar commented 1 year ago

Forget about this, I will make full-fledged sanitizers.