WAZAAAAA0 / TekkenBot

AI and tools for playing and understanding Tekken 7
MIT License
210 stars 45 forks source link

how to update memory_address.ini #57

Open dcep93 opened 5 months ago

dcep93 commented 5 months ago

Hi! I'm an engineer trying to understand how to get the values to populate memory_address.ini

I've learned cheat engine basics and seen Roguelinke's video, but I think with a little more support, I can better understand all the moving parts.

Specifically, I'm comfortable on fetching p2_data_offset and rollback_frame_offset, but I'm totally lost on how to efficiently get player_data_pointer_offset. Also, I don't know the best way of finding updates like facing or if I wanted to add current_health as a feature in addition to damage_taken.

Once we have a repeatable pattern, I think I can build a python script that will automatically perform these steps and update the memory address file. Even if that's not possible, I can help to post updated files and document these steps so that others can follow too.

Would it be possible to set up some time to hop on a call with @WAZAAAAA0 or @FimoX, since they seem to understand this stuff a lot better than I do? I promise I'll do the heavy lifting, and I think a 30-minute sync will be plenty to get me up to speed.

There's a Tekken patch coming May 8 - perhaps we can grab those updated values together!

FimoX commented 5 months ago

No problem I can answer you on this thread.

player_data_pointer_offset First you have to look at the p1_move_id 32769 = idle/standing 32770 = fully crouching by holding down You will end with 97 adresses: 1st one must be the move_id in real time then 32 frames X 3 move_id Do then a pointer scan to the the 2nd adress (max value 24k, max level = 6... or 5) Do a sort on the last offsets (click on the last columns) Knowing that the offset for move_id is 518, select one offset that should look like: "Polaris-Win64-Shipping.exe"+08E732C8 with offsets 10 68 8 30 518 It means then that player_data_pointer_offset = "Polaris-Win64-Shipping.exe"+08E732C8 10 68 8 30

facing and other player_data offsets In cheat engine try to reduce the memory size with start = player_data_pointer_offset end = player_data_pointer_offset + rollback When you will find one adress, dont do a pointerscan because it will take too much time. That's better to compare the shift with move_id (offset 518) and your new adress

no player_data offsets opponent_side takes me at least one hour to find the adress, that's a nightmare: go to quickselect play left/right, scan all memory for 0 or 1 ... rince and repeat at least 40 or 50 times to reduce the list at around 10-15 adresses Then pointer scan all those adresses, one of those adresses must have only one single offset as for intance "Polaris-Win64-Shipping.exe"+094B45B0 4

But I saved all the array of bytes of this adress in v1.02 = 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00 40 00 00 00 40 00 00 00 40 00 00 00 40 00 00 00 40

In v1.03 I did a search on that array of bytes, and bingo I found the new adress in less than 10 seconds ! :)

PS: last remark about one differences between T7 and T8 In T7 frame_count is always smaller by 1 compared to p1_move_timer In T8 frame_count = p1_move_timer

WAZAAAAA0 commented 5 months ago

added an experimental ghetto OPPONENT_SIDE "quick finder script" using the instructions above me

as for player_data_pointer_offset, I think I fixed the "quick finder" to work more reliably in more stages/modes/situations etc. (I took a full 8 GB RAM dump of the game process in February using Windows Task Manager and I'm still using it to come up with AOB signatures lol)

rest of the CT won't last long, incoming patches(-_- ) TK8BotPrime_2024-05-03.zip

dcep93 commented 5 months ago

this is awesome, thanks!

I'm making progress on this python script to automatically detect everything, but it's hard to say when it'll be done, maybe a few weeks.

Just a few questions while I have people's attention

  1. how do we know the move_id offset is 518?
  2. why did you pick the offsets "Polaris-Win64-Shipping.exe"+08E732C8 with offsets 10 68 8 30 518? my pointer scan had many many results where the final offset was 518
WAZAAAAA0 commented 5 months ago

how do we know the move_id offset is 518?

the devs placed it there and that's where all the scanning brought us. It just be like that

my pointer scan had many many results

no it didn't, those were all invalid except 1. Try this: keep the "found pointer list window" open, restart the game, go back to training, reattach CE to the game process, click Pointer scanner -> Rescan memory -> Address to find: the-new-correct-address and watch the list get pruned with only 1 survivor.

Once we have a repeatable pattern, I think I can build a python script that will automatically perform these steps and update the memory address file.

imo, because of HOW the game gets patched, I doubt it's possible to keep it updated fully automatically with no human intervention whatsoever. Let's say we were to find a solid way to get the player_data_pointer_offset memory area (with a so-called "AOB signature" usually) that keeps all the hot and spicy addresses close to each other such as:

blabla1 + 518  = move_id
blabla1 + 57C  = recovery
blabla1 + 5B8  = hit_outcome
blabla1 + B388 = attack_startup
etc.

well, unfortunately in the next update the structure may or may not change UNEVENLY! (real example btw)

blabla2 + 518  = move_id
blabla2 + 57C  = recovery
blabla2 + 5B8  = hit_outcome
blabla2 + B438 = attack_startup <<<<<<
etc.

An observation: Honestly, if you're in the process of reworking the bot, there's a lot to gain by making it more resilient to incorrect addresses/values. Sure we've made the process of providing updates a bit easier by using the memory_address.ini fork, by documenting the steps in a wiki, and most importantly by crowdsourcing the efforts in a centralized place... but there's no good reason for the bot to crash and burn entirely when it can't find a valid damage_taken or whatever for example (that one's not even displayed anywhere lol)...

FimoX commented 5 months ago
1. how do we know the move_id offset is 518?

You have to imagine that in the original source code you have the player_data as an array (X32):

  • the first variables dont change when there is a new update, but they may add some new variables: As Waza showed you, they may add a new variable between hit_outcome and attack_startup and that's why all variables may be shifted by +100 after offset B338
    1. why did you pick the offsets "Polaris-Win64-Shipping.exe"+08E732C8 with offsets 10 68 8 30 518? my pointer scan had many many results where the final offset was 518 We are expecting to have 5 offsets for move_id. When you will run your pointerscan, set Max Level = 5. Then sort the results on the last column (offset 4) and scroll down to the line that ends with 518
dcep93 commented 5 months ago

Wow, thanks so much for the context! My understanding is much better now - here's my progress so far

  1. I've been working in my own repo - I find that my fork is quite resilient! I use it often, and it basically never crashes. It's not quite ready to be showcased, since I'm doing heavy construction on it while experimenting with update_memory_address.py. I expect to be all done with my development on it by early June.
  2. Speaking of which, I'm making tons of progress on update_memory_address.py. If we assume that move_id_offset is constant at 0x518, it's able to determine most of the values that make it to the ini file. It takes about 5 minutes to scan the entire memory state and build a pointer map. Once it does, it asks for a few user inputs, like p1_high_attack or p2_crouch. I tried to make it automatically do the inputs, but it was tricky to force the inputs to go to the Tekken process, and tricky to send inputs to p1 vs p2.

Currently, it determines values for

  1. expected_module_address
  2. rollback_frame_offset
  3. p2_data_offset
  4. frame_count
  5. attack_type
  6. player_data_pointer_offset
  7. opponent_side

With a bit more work, I should be able to do many of the other addresses. The code is here, but unfortunately, it's not the most legible, so I don't imagine many people will be able to add features. I'm going on a week vacation tomorrow, but I hope to implement the rest of the PlayerDataAddress values soon.

FimoX commented 5 months ago

I gave a try to your update_memory_address.py but it doesn't work, because you did some customisations. Line 9 error: from game_parser import GameReader, MoveInfoEnums .... module GameReader not found...

FimoX commented 5 months ago

In v1.04 the last offset for move_id is 51c ... no more 518 That update is huge, all offsets have moved.

dcep93 commented 5 months ago

my tool worked! (I think)

right now, you have to manually find move_id_offset, but in the future, perhaps it could detect it automatically - seems annoying, but potentially possible

here's what I have so far, I want to check some things before making an announcement in the other thread - probably can post there by Monday

let me know what necessary offsets are missing - these are the only ones my fork needs

https://github.com/dcep93/TekkenBot/blob/5990de0e4c97093fed4757822da0cf2869f8b7e0/TekkenBot420/assets/config/memory_address.ini

FimoX commented 5 months ago

@dcep93 I've found 3 different offsets: facing = 0x1dbc stun_type = 0x61c complex_move_state = 0x654

dcep93 commented 5 months ago

Interesting - it's understandable that it found different locations for facing and stun_type, because it looks for the lowest possible offset. but it deemed 0x654 as impossible for complex_move_state

it might be nice to see the values located at these offsets side-by-side to compare them - that'll be the first place I'll look when things get fishy

dcep93 commented 5 months ago

ok, I think things are stable and working for me, curious if stuff is broken for others, but perhaps best time to test is next patch

for facing, I do see 4 frames of discrepancy between our values while the side is switching, but they are identical outside of that - TBD if that will cause problems

for stun_type, seems that for the player doing the hitting, 0x61c reports "1" for 18 frames during the jab, which sorta makes sense, but isn't a valid value for StunStates. Also, it reports StunStates.NONE=0 for those same 18 frames for the player getting hit, whereas 0x614 reports StunStates.GETTING_HIT=256.

for complex_move_state, I just see 0's always for 0x654, but 0x65c reports the various values for both p1 and p2. Not sure if it's legit, but that's what I see so far...

I think the weirdest thing is that expected_module_address isn't consistent. It doesn't really affect anything, but in previous versions, the value returned from windll.kernel32.Module32First was always the same. I don't really understand the implications here, but just thought I'd mention.

zeratul7x commented 3 months ago

What about update this bot for 1.05 patch? update_memory_address.py doesn't work and have an error. =(

dcep93 commented 3 months ago

aww sorry it doesn't work for you! my sister just got married, so I took a break from Tekken last month

I was planning to get back into it once Lidia comes out, so I'll probably skip patch 1.05

I'll update https://github.com/dcep93/TekkenBot/blob/master/TekkenBot420/assets/config/memory_address.ini with patch 1.06 if/when it comes out, and hopefully update update_memory_address.py too so it will be more reliable

FimoX commented 3 months ago

@dcep93 facing = 0x2D3C stun_type = 0x614 complex_move_state = 0x65c player_data_pointer_offset = 0x0936CEE8 0x10 0x68 0x8 0x30 memory_address.zip

zeratul7x commented 1 month ago

@dcep93 What about update your bot for v 1.07? Also, Lidia's moves doesn't display in the log. Fix of this is reqired too =(

dcep93 commented 1 month ago

working on it, just got back from work travel, probably will fix Monday or Tuesday

FimoX commented 1 month ago

I hope it will help CE_TekkenBot V1.07.01.zip

dcep93 commented 1 month ago

thanks @FimoX - that helped a lot!

part of the reason for the delay was opponent_side - it seems the string I'm looking for has changed to 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 02 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 04 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 08 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 10 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 40 00 00 00 40 00 00 00 40 00 00 00 40

notice the change from previously ending with: ...40 00 00 00 40 00 00 00 40 00 00 00 40 00 00 00 40

https://github.com/dcep93/TekkenBot/blob/master/TekkenBot420/src/main/update_memory_address.py#L817

@zeratul7x it should be updated on my main branch - btw, how do you run my app? Do you run the exe that I build, or do you run the code from python yourself? I recommend running the exe, but curious what works for you!

zeratul7x commented 1 month ago

@dcep93 I'm running exe file from release section. But, when new patch is come, i try to update it with py script tool. Sometimes, it's work correct ) So, is Lidia move display problem resolved in this release?

dcep93 commented 1 month ago

yes, Lidia is fixed

new character releases require an extra step for me, I just need to not forget to publish it

FimoX commented 1 month ago

@dcep93 About opponent_side I have shortened the search to: image

olaHalo commented 4 weeks ago

I'm not sure why I am struggling with this. Following @FimoX's instructions, everything is fine until I do the pointer scan Do then a pointer scan to the the 2nd adress (max value 24k, max level = 6... or 5) When I do this, the scan takes a long time (hours if Max Level =6) and ends up over 15000000 pointer paths. Are we using the "Pointers must end with specific offsets" to help filter this down? Am I missing something?

@dcep93's python script works great. But with the larger update coming out in a week, I'd rather not rely on someone else's tool. Any advice is appreciated.

FimoX commented 4 weeks ago

@olaHalo

dcep93 commented 3 weeks ago

For me, the pointer scan takes about 30 minutes, I think it depends on the performance of your machine

I just updated for 1.08! I couldn't find the offset for facing, but it's not really necessary

Also, for the future, when a new character is introduced, it won't crash the game.

FimoX commented 3 weeks ago

My CE table for v1.08.01 CE_TekkenBot V1.08.01.zip