search

OrfeasZ / Statman

Live statistics tracking for games
MIT License
40 stars 7 forks source link

HITMAN 2 support #7

Open hardware opened 6 years ago

hardware commented 6 years ago

Hi @OrfeasZ,

Do you have any plan to support the latest Hitman game ? Is it technically possibe ? if so, how would you proceed ? (I have some basic experience with Qt and C++, if i can help...).

OrfeasZ commented 6 years ago

I looked into Hitman 2016 a while ago, and it is significantly different compared to the previous Hitman games (and the fact that it uses Denuvo makes things slightly more complicated in some cases). I think I started adding support for 64-bit games to Statman at some point but I'm not sure if I ever finished that.

As for how you'd go about doing something like this: the simplest way to begin is by figuring out a reliable path to where the game stores information about the current play session. You can do this either by reverse engineering the game (more accurate, but more specialized, time-consuming and in this case slightly more challenging because of Denuvo) or by using similar techniques to people creating standard game trainers, using tools like Cheat Engine (you can look up trainer tutorials on YouTube).

From there, Statman provides some utilities to automatically hook into a process or read its memory (though some of this functionality may be incomplete for 64-bit). The API isn't the greatest right now, and certainly needs an improvement, but I haven't had the time to deal with it.

hardware commented 6 years ago

Denuvo is gone since june 2017.
https://www.dsogaming.com/news/denuvo-anti-tamper-tech-has-been-removed-from-hitman/

Thanks for taking the time to respond. I do not have the skills to reverse engineer the game unfortunately. Maybe someone will do it someday :)

EDIT: maybe i'll try to do something with C# .NET but without those memory addresses, it will be difficult . Moreover, they seem to change with each game update according to the developer of the Hitman camera tool mod : https://github.com/FransBouma/InjectableGenericCameraSystem/releases/tag/Hitman2016_v110

OrfeasZ commented 6 years ago

Denuvo being gone is good news. Also, updates tend to indeed break tools like these, but there are ways to make them more resilient (not something to be concerned with right now). As for this case, as I mentioned before, the simplest way to locate memory addresses is using something like Cheat Engine, and doesn't really require reverse engineering skills. Search for "game trainer tutorial cheat engine" on YouTube and I'm sure there's gonna be something helpful there.

OrfeasZ commented 6 years ago

I've started making some progress on this in branch engine/hm5. Not much to see yet, but there's some promising stuff.

hardware commented 6 years ago

It looks promising, thank you for taking the time to implement that mate, it will be very useful for the community :) Maybe multiple contributors can help you. Poke @nvillemin @SuiMachine @HHCHunter @swixel @TheKotti

I can help about the SA conditions. I am not a speedrunner but I know the game mechanics pretty well.

OrfeasZ commented 6 years ago

Right now all my research focuses around reverse engineering an engine data structure called ZAIGameState (no legal issues here, allowed under EU laws for interoperability reasons), which has an extensive bitfield, tracking a variety of different game-state related things (see here). This structure is also "synced" to game script (presumably the CppTemplates, whose format I have not looked into) with the help of the type information system you are talking about.

I have not looked into the scripting engine much, but it wouldn't be too difficult to intercept data going back and forth from native land, as Statman already supports injecting native DLLs into the game process. However, for now I'm trying to stick to static in-memory structures, as they look promising and are much easier to work with, much faster to parse, and in many cases more tolerant to game updates.

As for intercepting network traffic, I have no plan on doing that, because that starts getting into sketchy territory. If anything, I'd just hook the underlying subsystem.

hardware commented 6 years ago

@OrfeasZ Did you found this property in the game ?

https://github.com/OrfeasZ/Statman/blob/a9fd5cb14d971e91f3f946d6e7449f147f9750d1/StatModules/HM5/Src/HM5/Structs/ZAIGameState.h#L174

If yes, it is good to know because this feature is requested by the community since day one :D IOI said, 2 years ago, it was too complicated to implement blood pool detection by NPC because the engine was not designed for that.

OrfeasZ commented 6 years ago

@swixel ZAIGameState is statically allocated (and has been at least since the beta, which is the earliest binary I have in my hands), and it hasn't really changed much (except for some stuff being re-arranged or added).

@hardware Everything in there is pulled from the engine (I'm not currently making any naming assumptions), but this specifically has been there since beta too, so I wouldn't get my hopes up too much.

OrfeasZ commented 6 years ago

Some stupid shit: some stupid shit

Already in engine/hm5.

OrfeasZ commented 6 years ago

@swixel thanks for the flags! I already have ZString mapped out so it's fine. Right now I'm working on finishing the reflection stuff so I can move on to entities, at which point I'll have a much clearer picture of what's happening in the scripting engine.

Do you know if there are any resources online about what CppTemplates are and how they work? I assume they represent some kind of node-graph / circuit system, but I don't want to spend too much time looking into them.

OrfeasZ commented 6 years ago

0x578FBCEE is actually dubbed RuntimeResourceID (you were pretty close!) and there's an extra data type that I found:

0x64603664: RuntimeResourceIDAndResourcePtr (haven't looked into how it's actually serialized).

HHCHunter commented 6 years ago

I'm surprised I didn't get the GitHub notification for the tag on this one.

I'm really glad people are putting in effort for HITMAN. Unfortunately these days I don't really have the time for working on HITMAN due to full-time work.

Looking forward to what engine notes people come out with more work being put in 👍

OrfeasZ commented 6 years ago

Here's a full list of resource types and their corresponding engine typenames for anyone who may find them useful:

AIBB BehaviorTreeEntityBlueprint
AIBX BehaviorTreeEntityType
AIBZ CompiledBehaviorTreeResource
AIRG ReasoningGridResource
ALOC Physics
ASEB AspectEntityBlueprint
ASET AspectEntityType
ASVA AnimSetVariation
ATMD AMD
BMSK BoneMask
BORG AnimationBoneData
CBLU CppEntityBlueprint
CLNG DialogCascadingLanguageDependencies
CPPT CppEntityType
CRMD CrowdMapDataResource
DITL DialogSoundTemplateList
DLGE DialogEvent
ENUM EnumType
ERES EntityResource
FXAC Actor
FXAS AnimSet
GFXF ScaleformGFx
GFXI ScaleformGFx
GFXV ScaleformGfxVideo
HIKC HIKCharacter
JSON JsonResource
LINE TextLine
LOCR Localization
MATB RenderMaterialEntityBlueprint
MATE RenderMaterialEffect
MATI RenderMaterialInstance
MATT RenderMaterialEntityType
MRTN Network
MRTR Rig
NAVP NavpowerNavmesh
ORES OnlineResource
PRIM RenderPrimitive
REPO Repository
RTLV RuntimeLocalizedVideo
SCDA ScatterData
SCTX ScatterTexture
SDEF SoundAmbienceDefs
TBLU TemplateBlueprint
TELI TextList
TEMP Template
TEXD DummyTextureData
TEXT RenderTexture
UICB UIControl
UICT UIControl
VIDB VideoDatabaseResource
VTXD VertexData
WBNK WwiseBank
WSGB AudioStateBlueprint
WSGT AudioStateType
WSWB AudioSwitchBlueprint
WSWT AudioSwitchType
WWEM Wem
WWEV WwiseEvent
WWFX WwiseBankFX
YSHP PhysicsSystem
OrfeasZ commented 5 years ago

I've not made any progress on this since I've been waiting for HITMAN 2 (2018) to release, so I can port everything over to it, since presumably that's what people will be playing as it has all original content. If anyone gets access to the beta (if they have one, or even the sniper challenge thing) and is willing to share said access to help with the Statman integration please hit me up. My e-mail is on my profile page.

mike-koch commented 5 years ago

So I tried building / running this earlier today, and it seems to crash HITMAN (2016) upon launch (I'm not even bothering with HITMAN 2 at the moment until I can get it to work with HITMAN.... and I'm not sure if Denuvo will cause any issues in H2).

Has anyone had any luck running this against the latest build of HITMAN (or HITMAN 2 for that matter)?

OrfeasZ commented 5 years ago

@mike-koch HITMAN (2016) has received a few updates and I haven't updated Statman to support it (and I don't think I necessarily will). I've started looking into H2, but haven't had much time to do anything with it yet. I'm hoping Denuvo won't be an issue but we'll see.

OrfeasZ commented 5 years ago

Statman 2019, now with 100% more time tracking (click on the thumbnail): Statman2019

Relevant updates in engine/hm5 branch, with some updated structures for HITMAN™ 2 and sig-based pointer lookup and hooking, which should help this survive across updates. More things to follow whenever.

OrfeasZ commented 5 years ago

Some proper level detection too because that's always important: Screen

Notexe commented 5 years ago

Should the game time start when the cut scene is finished/skipped since that's how the game time is tracked for the in-game mission timer?

OrfeasZ commented 5 years ago

The timer in Statman starts as soon as the level is loaded, which matches the internal game timer. The one displayed on screen if you have the option turned on is probably tracked differently. I may or may not look into how that's done but that's not really a priority right now.

OrfeasZ commented 5 years ago

Updated list of resource types and their encoded names from HITMAN 2 for anyone who may have use for them:

AIBB BehaviorTreeEntityBlueprint
AIBX BehaviorTreeEntityType
AIBZ CompiledBehaviorTreeResource
AIRG ReasoningGridResource
ALOC Physics
ASEB AspectEntityBlueprint
ASET AspectEntityType
ASVA AnimSetVariation
ATMD AMD
BLOB ResourceBlob
BMSK BoneMask
BORG AnimationBoneData
CBLU CppEntityBlueprint
CLNG DialogCascadingLanguageDependencies
CPPT CppEntityType
CRMD CrowdMapDataResource
DITL DialogSoundTemplateList
DLGE DialogEvent
ECPB ExtendedCppEntityBlueprint
ECPT ExtendedCppEntityType
ENUM EnumType
ERES EntityResource
FXAC Actor
FXAS AnimSet
GFXF ScaleformGFx
GFXI ScaleformGFx
GFXV ScaleformGfxVideo
GIDX GlobalResourceIndex
HIKC HIKCharacter
IMAP IDMap
JSON JsonResource
LINE TextLine
LOCR Localization
MATB RenderMaterialEntityBlueprint
MATE RenderMaterialEffect
MATI RenderMaterialInstance
MATT RenderMaterialEntityType
MJBA Animation
MRTN Network
MRTR Rig
NAVP NavpowerNavmesh
ORES OnlineResource
PREL Preload
PRIM RenderPrimitive
REPO Repository
RTLV RuntimeLocalizedVideo
SCDA ScatterData
SDEF SDefs
SLMX SelectMatrix
TBLU TemplateBlueprint
TELI TextList
TEMP Template
TEXD RenderTextureData
TEXT RenderTexture
UICB UIControl
UICT UIControl
VIDB VideoDatabaseResource
VTXD VertexData
WBNK WwiseBank
WSGB AudioStateBlueprint
WSGT AudioStateType
WSWB AudioSwitchBlueprint
WSWT AudioSwitchType
WWEM Wem
WWEV WwiseEvent
YSHP PhysicsSystem

@HHCHunter

OrfeasZ commented 5 years ago

Latest commit adds some preliminary rating tracking:

Statman

Keep in mind that this might be entirely inconsistent with what the game actually presents at the end because rating computation seems to happen entirely server-side, so we can only make assumptions on what actually matters. I've added the indicators that seem to affect it based on feedback by Kotti, and I'll be adding some more in-depth things soon. If everyone has any additional insight on how it works please let me know.

Here's a binary build for those of you who are daring and want to try this (no guarantees this will work on anything but my computer): https://i.nofate.me/QUQaH1EvkddXfJDI.zip

hardware commented 5 years ago

Omg, thank you very much for your work. I will try this.

hardware commented 5 years ago

Quick feedback :

HITMAN 2 : https://www.youtube.com/watch?v=6LgWIMfzD4c HITMAN 1 : https://www.youtube.com/watch?v=_ejLkUfPl5c

Issues :

image

Really great job, you nailed it ! 👍

mike-koch commented 5 years ago

Did a couple tests myself, and noticed the following:

  1. Eliminating either Harry or Marv in Holiday Hoarders causes HITMAN 2 to crash (Statman survived the first crash, but stopped responding after the second). No crash occurs when eliminating Novikov during The Showstopper.
  2. Spotted / Noticed Kill / Non-Target Kill / Body Found flags don't reset when loading a save game. They do reset when starting a new mission, however. (I haven't had a chance to take a look at the code yet to see if it's feasible to re-retrieve these values when loading in a save game or not, or if they're only picked up as the event occurs in-game).
  3. Not really an issue per-se, but loading a save game sets the "Game Time" to the in-game timer during the save (my save had a game time of 00:09:00.181), but the real time timer starts at zero. I understand that it's impossible to know how much real time I spent on a mission, but would it make more sense to start the real time at the same time as the game time's value on load?

Overall I am extremely impressed at how fast you figured all of this out. Thanks for all of the effort you are putting into this; it is greatly appreciated 😃

mike-koch commented 5 years ago

Edit 2019-01-06: I believe I got the main portion of cameras working properly. I posted a sample video over at https://www.youtube.com/watch?v=RBbkbQNlx-s

I did a little investigating into the camera events (I saw your TODO about SecuritySystemRecorder events not triggering as expected). Here's what I personally have observed:

  1. A "spotted" event is triggered for each camera only once, regardless of how many times you step into the camera's FOV. I suppose once you're spotted by a camera, there's no point recording that you've been spotted by the same camera again.
  2. A "destroyed" event is triggered when destroying the camera recorder, regardless if you've been recorded on any cameras or not.
  3. There doesn't seem to be a specific event when caught on camera when trespassing or holding an illegal item. However, several other events do trigger in these situations which I've outlined below. Hopefully this helps in some way 😛

Spotted by a camera when trespassing:

ZAchievementManagerSimple Event! {
    "Name": "47_FoundTrespassing",
    "Origin": "gameclient",
    "Timestamp": 39.735672,
    "Value": ""
}
ZAchievementManagerSimple Event! {
    "Name": "AmbientChanged",
    "Origin": "gameclient",
    "Timestamp": 40.352379,
    "Value": {
        "Ambient": "AlertedLow",
        "AmbientValue": 4,
        "PreviousAmbient": "Ambient",
        "PreviousAmbientValue": 1
    }
}

Spotted by a camera when performing an illegal action (i.e. holding weapon when wearing a suit)

ZAchievementManagerSimple Event! {
    "Name": "AmbientChanged",
    "Origin": "gameclient",
    "Timestamp": 17.065767,
    "Value": {
        "Ambient": "Arrest",
        "AmbientValue": 7,
        "PreviousAmbient": "AlertedLow",
        "PreviousAmbientValue": 4
    }
}

ZAchievementManagerSimple Event! {
    "Name": "DisguiseBlown",
    "Origin": "gameclient",
    "Timestamp": 17.065767,
    "Value": "69aac6db-461e-43af-89bc-2c27e50d430f"
}

ZAchievementManagerSimple Event! {
    "Name": "Witnesses",
    "Origin": "gameclient",
    "Timestamp": 17.065767,
    "Value": [
        "4a69e8ca-96c9-4f6d-b977-65c10f3ca202"
    ],
    "XboxDifficulty": 0.0,
    "XboxGameMode": 3.0
}
mike-koch commented 5 years ago

It looks like the latest HITMAN 2 update broke Statman 😒 ....any chance of a fix?

OrfeasZ commented 5 years ago

I'm out of town. I will take a look when I'm back and I have some time at some point in the next couple of weeks

Notexe commented 5 years ago

Denuvo has been removed in the latest Hitman 2 update.

OrfeasZ commented 5 years ago

Damn, that was fast. I guess that's why some of my stuff broke. That's pretty good news overall. Denuvo is pretty annoying.

OrfeasZ commented 5 years ago

Statman in the engine/hm5 branch now supports the latest update.

hardware commented 5 years ago

@OrfeasZ can you provide a binary build of the latest version ?

OrfeasZ commented 5 years ago

@hardware Here you go

OrfeasZ commented 5 years ago

Latest version in the engine/hm5 branch features an experimental implementation of Kotti's rating system concept (found here) and also fixes support for the latest game update (would previously crash the game). I've attached a binary release below for anyone interested in playing with it.

Here's a video of the advanced rating system in action: https://i.nofate.me/3eS9Viyntdkm.mp4

To use it, simply right click on the Statman logo and press Toggle Rating Mode. Everything from the tracking logic to the UI is very WIP, so keep that in mind. These are the things that are currently tracked:

Release.zip