Spuckwaffel / UEDumper

The most powerful Unreal Engine Dumper and Editor for UE 4.19 - 5.3
MIT License
707 stars 135 forks source link

assist me with finding GNames, GObjects and GWorld please! #8

Closed vxcall closed 1 year ago

vxcall commented 1 year ago

Hi, first of all thanks for the cool project! Since im not well-familiar to UE and reversing, I have couple of silly troubles I hope you can help. Im at a very starting point and looking for F(G)Names, UObject and GWorld but somehow it's not as easy as I thought.

The game im looking at is Atomic Heart and it uses UE-4.27. I initially did tiny reversing back in April and found FNames and UObject, at least at that point it seems the game uses slightly modified variant of UE. (I saw some functions are merged into 1 than how it should be and not sure if it's an optimization or not)

And today I fire up IDA and load this game with PDB file of UE4.27 sample game i made ( even sample game's PDB helps me reversing ), but i couldn't find strings that im looking for. IDA string view doesnt show me strings such as 'SeemlessTravel FlushLevelStreaming' or 'UObjectBaseInit' which i used to levarage in order to find UObject.

IDA string view does show bunch of strings but they're not what im looking for. The game is just single player game so it's unlikely compile-time string enctyption imho.

What would be the best approach i can take in this situation? I dont think i can fully reverse this game cuz it's rediculously huge size.

vxcall commented 1 year ago

Hi, I've just read this post on unknowncheats. Possibly dumping the process after launch may solve this problem ( He indeed suspects string encryption in some games i suppose?).

I'll try this later anyways.

Spuckwaffel commented 1 year ago

Hey there, sorry for the late reply, i've been very busy lately. The strings you are looking fore aren't showing up because you didn't select the proper string settings in IDA. What you have to do is right click into the Strings window > Setup > enable Unicode C-Style (16 Bits). Then you should find the proper strings. If not, enable the other disabled options.

To the functions, it's completely normal if functions are merged, the compiler sometimes decides to do so.

Hope this helps.

vxcall commented 1 year ago

Hi, Thanks for the response. It's all fine take your time mate. I respect it. Im sorry to occupy your time with such a beginner issue. Im heading home right now so ill test your suggestion once i get home. Oh I didnt know that functions getting merged are ordinal thing, thanks again.

vxcall commented 1 year ago

I got to find the OutWorld ( = GWorld), thanks! image

In terms of GObject, I also could find 2 candidates of GUObjectArray using strings "Failed to find commandlet class" and "r.OneFrameThreadLag". If im not mistaken, what this project refered to as "GObjects" is GUObjectArray[FUObjectArray] -> ObjObjects[TUObjectArray] -> Objects[FUObjectItem**] as far as i look up the UE source am i correct?

If it's not too much trouble, may I update you about FNames and other settings for this project? I might need further help later.

Spuckwaffel commented 1 year ago
  1. No problem
  2. Yes i'm speaking of that. The pointer should lead to the "Struct" that holds the double pointer to the objects and also the information about the ChunkCount and ObjectCount. Take a loot at Core.cpp file for further information. I have also included signatures in the Offset file you can use in order to find the GObject offset or FName offset. They might work.
  3. Sure you can keep me updated.
vxcall commented 1 year ago

"Struct" that holds the double pointer to the objects and also the information about the ChunkCount and ObjectCount

I guess it's pointer to FChunkedFixedUObjectArray (UE source) ?

for GNames,( I presume it's identical to FNames ) no idea what it is. I spotted possible FName thing in UE source but no idea about its array. also your sig didnt work for my game. But I managed to find pointer to FNameEntryAllocator in my game and maybe its Block[] member variable could be what referred to as GNames?

Spuckwaffel commented 1 year ago

The Gnames offset is a pointer to the Name array. The easiest way it can be found is by (as you have the pdb) jump to the Fname::ToString(this, FString out) function in IDA. You should have the names everywhere so it will be a pretty easy find. Compare the code to the code in Core.cpp@FnameToString and you should find there a pointer that will be the Gnames offset. Most likely it will look in ida in some function like this:

if(byte_4729472) {
   return unk_57394724; //this is gnames
}
else {
   unk_57394724 = sub_3345729();
   byte_4729472 = 1;
   return unk_57394724;
}

just something like that, and just look out for that if statement and it either returns the gnames pointer or it does some stuff like it but no matter what, that unk will always be the gnames offset.

vxcall commented 1 year ago

note: guobjects: base + 6D6FB00 gnames: base + 6F96C40 gworld: base + 6F93418

vxcall commented 1 year ago

@Spuckwaffel

So, I've finally collected 3 offsets in my hands. I want you to validate them. Each of these looks like this:

GObjects

image

GNames

image

GWorld

image

but the editor didnt work. This is error log I guess the pointer i've got is close but incorrect, but idk how it's incorrect.

0 [23:16:13 - MAIN]: Loading program...
1 [23:16:13 - MAIN]: Loaded imgui helper library...
2 [23:16:13 - MEMORY]: Initializing memory class...
3 [23:16:13 - MEMORY]: Initialized Memory class!
4 [23:16:50 - MEMORY]: Loaded Memory class!
5 [23:16:53 - DUMPPROGRESS]: Starting dump...
6 [23:16:53 - ENGINECORE]: Loading core...
7 [23:16:53 - ENGINECORE]: TUObject -> 0x00006F0400006F05
8 [23:16:53 - ENGINECORE]: TUObject elements: 0
9 [23:16:53 - ENGINECORE]: UObjectArray seems empty!
10 [23:16:53 - DUMPPROGRESS]: Failed to initialize EngineCore!
vxcall commented 1 year ago

So I read the source code and made it work by adding 0x10 to my GObjects offset. Cuz the source code expects the UObject array at offset 0, so i add 0x10 to make array top of the address . only this didnt make it work so I added 0x10 to GNames ( I just suspect by looking at the Reclass cuz the first 0x10 bytes look not part of the array ).

Then it finally looks started working. the string class not found! is on console tho.

image

image

vxcall commented 1 year ago

nah, its broken.

image

0 [00:25:18 - MAIN]: Loading program...
1 [00:25:18 - MAIN]: Loaded imgui helper library...
2 [00:25:18 - MEMORY]: Initializing memory class...
3 [00:25:18 - MEMORY]: Initialized Memory class!
4 [00:25:35 - MEMORY]: Loaded Memory class!
5 [00:25:41 - DUMPPROGRESS]: Starting dump...
6 [00:25:41 - ENGINECORE]: Loading core...
7 [00:25:41 - ENGINECORE]: TUObject -> 0x00000000111AF040
8 [00:25:41 - ENGINECORE]: TUObject elements: 89944
9 [00:25:41 - ENGINECORE]: Allocating 0x000000000020F040 bytes of memory for GObjectPtrArray at 0x000001AC9DD0D040
10 [00:25:41 - ENGINECORE]: Loaded GObjectPtrArray succesfully!
11 [00:25:41 - ENGINECORE]: Allocating 0x36E5C0 bytes of memory for UBigObjectArray at 0x000001ACA8A3D040
12 [00:25:41 - ENGINECORE]: Could not resolve address for obect 28444!
...
511 [00:25:41 - ENGINECORE]: Could not resolve address for obect 89874!
512 [00:25:41 - ENGINECORE]: Could not resolve address for obect 89941!
513 [00:25:41 - ENGINECORE]: Loaded UBigObjectArray succesfully!
514 [00:25:41 - ENGINECORE]: Caching FNames...
515 [00:25:41 - ENGINECORE]: Cached all FNames!
516 [00:25:41 - ENGINECORE]: Caching all Packets...
517 [00:25:41 - ENGINECORE]: reading overriding structs....
518 [00:25:41 - ENGINECORE]: adding custom structs....
519 [00:25:41 - ENGINECORE]: adding overrigind unknown members....
520 [03:02:19 - ENGINECORE]: Total packages: 0
521 [03:02:19 - ENGINECORE]: Done generating packets!
522 [03:02:19 - DUMPPROGRESS]: Finished dumping!
523 [03:02:19 - DUMPPROGRESS]: Finished everything with 327002 memory operations!
524 [05:22:10 - PACKAGE]: opened package 0
525 [05:22:39 - PACKAGE]: opened package 0
526 [05:22:44 - PACKAGE]: opened package 0
Spuckwaffel commented 1 year ago

hey there sorry for the late reply. i might go look into the game once im back from holiday but this doesnt ssem correct. Inbstead of digging around some holes to fix the issue i would suggest as i said setting breakpoints in the fnamrtostring function as the string function is the most important one for package generation. And i also assume that your strings are broken due to finding 0 packages. It is completely normal that there are many unresolved objects over 25k as they are dynamic ojects that come and go but never get deleted, so they are all invalid. However this will not affect the dump. Im not sure why wou add padding in the names, but yes for gobject you got it right, though im unsure why your gobject offset even starts at that weird pos.

Spuckwaffel commented 1 year ago

if you are sure that your gnames offset is correct, try setting CASE_PRESERVING_NAME to TRUE in the uedefs, this might resolve the issue, but please try without padding first

vxcall commented 1 year ago

Sure i'll try. FYI Guided hacking unreal engine dumper is also not able to dump my game automatically.

Btw i wonder if i purchase "Only Up" and try it first to know how everything works helps maybe. Big part of the issue im having is due to lacking of my knowledge. If you can provide me a updated offset or sig for Only up's gnames, gobjects and gworld I'll buy it and test there first.

Spuckwaffel commented 1 year ago

the signatures i've used are in the comments in the offsets file. You can also always double check by looking at the provided offsets in the file, they are still up to date.

vxcall commented 1 year ago

Thanks!!

and I didn't get to read your first comment today

i might go look into the game once im back from holiday but this doesnt ssem correct.

I didnt know ive been bothering your holidays. Thanks, im looking forward to it. Unreal Engine 5 is getting more and more popular so considering reversing Atomic Heart makes this tool better, the idea is great .:mage_woman:

Im not sure why wou add padding in the names

Hahaha i neither bro. I just added padding while praying that the program would work. cuz i have no really idea if gnames offset im having is correct or incorrect.

Anyways I'll test CASE_PRESERVING_NAME and addjust my offset around and test today.

vxcall commented 1 year ago

Last time I load pdb, somehow FName::ToString didn't appear but this time I've got it. image

Spuckwaffel commented 1 year ago

that ain't the right one, i said check out the one that has a FString param

vxcall commented 1 year ago

i said setting breakpoints in the fnamrtostring function as the string function is the most important one for package generation

I misinterpreted this, my bad. FNameToString you meant is this programs'. Ok, let me take a time read your source a bit more seriously and understand it. :smiley_cat:

Spuckwaffel commented 1 year ago

steps my dumper does

step 2 would fail if the uobject offset is wrong which you will also notice in the log when it (for example) tries to allocate a buffer for -1 objects.

step 3 would still succeed even if the gnames offset is wrong. step 4 would break as broken names will not find you the struct objects resulting in 0 packages created.

if step 4 succeeds step 6 breaks, its some random uedefinition which is different than the default ones which is almost never the case

that's the entire dumper logic easily explained

Spuckwaffel commented 1 year ago

in the latest commits of the development branch i added better error handling just as a side note

vxcall commented 1 year ago

Btw I'll do this from time to time, i have many things want to do besides this including my job, hobby and stuff so bear with me. I'll do the debugging this evening and update you, besides I just bought Only Up!, so i'll check out what's there at offset you provide in Offset.h and examples I appreciate your generous support buddy! :smiling_face_with_three_hearts:

vxcall commented 1 year ago

So during the dumping process, i run across bunch of "ERROR! Could not find ptr in linkedUObjectPtrs 3!", error. usually the Ptr number is 3, 1 or could be a few more values.

It's raised in

  1. EngineCore::generatePackages func
  2. if (!object->IsA<UStruct>() && !object->IsA<UEnum>()) ( Is_A() func I think is UStruct one)
  3. super = super->getSuper<UClass>() in for loop condition
  4. getUObject()
  5. existsRealPtr()
  6. Then when UObjectPtr parameter is 0x0000000000000003 or whatever, the error happens.

I tested with OnlyUP, but UObjectPtr parameter has never been 0x3.so somehow the UObjectPtr is being invalid during generate Packages func.

debug view

Call stack is identical to my chart above. image

another debugging vector

Also the place the dumper stuck is when Caching Packages, so i should investigate the function in charge.

image

Spuckwaffel commented 1 year ago

yeah ur uobjects are unfortunately wrong as the super pointer is invalid (0x3) image it works for me fine after updating the gobject offset to 0x7723D90.

vxcall commented 1 year ago

@Spuckwaffel I'm sorry i may've make you confuse, the all debugging process was done with AtomicHeart.

I tested with OnlyUP, but UObjectPtr parameter has never been 0x3.so somehow the UObjectPtr is being invalid during generate Packages func.

This sentence i sent means, I tested the debug also with OnlyUP but UObjectPtr was fine which then i can say that UObjectPtr being 0x3 in AtomicHeart is unhealthy.

it works for me fine after updating the gobject offset to 0x7723D90.

So yeah, OnlyUP works fine with the offset. Moreover, my offsets seems pointing to the same structure as OnlyUP's, so the internal structure must be slight different compared to normal one i guess. should debug a bit more. it's gonna be tough not gonna lie but do it little by little haha

Spuckwaffel commented 1 year ago

unfortunately i cant try atomicheart, the game looks fun but 50 bucks just for dumping is alot imo lol. However you can try using the newest release for better error handling, but that will mostlikely not fix our issue here. However you could try setting USTRUCT_FAST_ISCHILDOF_IMPL to FALSE. And im not sure but did you debug the fnames already?

Spuckwaffel commented 1 year ago

if you are 100% sure everythings right, try this. In the file UnrealClasses.h add a uint64_t pad in the UField class like this:

class UField : public UObject {
public:
    using UObject::UObject;

    /** Next Field in the linked list */
    UField* Next;

    uint64_t pad; //<---------------- this

    UField* getNext() const;

    static UClass* staticClass();
};

this was the issue on another game i tried dumping but it makes no sense to me how this padding occurs because theres literally nothing in the ue source

vxcall commented 1 year ago

the heck, the padding WORKED i think :sun_with_face: if it's typical that some of the packages doesnt have info in it. Do you think it's valid?

yay, this is first time i dump and generate UE game's sdk bro.

image image image

vxcall commented 1 year ago

And im not sure but did you debug the fnames already?

Btw I've been unsure what to see during FNameToString function debugging. finalName looked correct to me, I only checked some of them cuz it's iterating for hundreds times. Since I couldn't get much info, i lastly tried to back trace from where the error arised. Sadly I didnt get to spend enough amount of time due to seduction of Valorant lately hahaha. my bad

vxcall commented 1 year ago

this was the issue on another game i tried dumping but it makes no sense to me how this padding occurs because theres literally nothing in the ue source

yeah it does seem like there are no other member variables. My understanding of how C++ functions, constructors, and typedefs appear in memory is vague, so I can't say for sure, but they shouldn't appear in it i think. so... yeah idk.

image
Spuckwaffel commented 1 year ago

And im not sure but did you debug the fnames already?

Btw I've been unsure what to see during FNameToString function debugging. finalName looked correct to me, I only checked some of them cuz it's iterating for hundreds times. Since I couldn't get much info, i lastly tried to back trace from where the error arised.

Sadly I didnt get to spend enough amount of time due to seduction of Valorant lately hahaha. my bad

well if finalname looked good then dw

Spuckwaffel commented 1 year ago

this was the issue on another game i tried dumping but it makes no sense to me how this padding occurs because theres literally nothing in the ue source

yeah it does seem like there are no other member variables. My understanding of how C++ functions, constructors, and typedefs appear in memory is vague, so I can't say for sure, but they shouldn't appear in it i think. so... yeah idk.

image

no, functions, constructors or typedefs don't increase the class size, functions and constructors are scattered in the binary. Well i have no idea where the padding comes from but it resolved an issue in a different game

vxcall commented 1 year ago

Well this was a long journey. Thanks for your help it really means huge, buddy. It didn't cross my mind that Unreal Engine modding is such a drag ( I'm not even modding anything yet XD ). Indeed I still need to look up things to properly utilize the SDK. It was enjoyable discuss things w u. hope this issue helps someone else too. You can close this! Thanks :fire:

Spuckwaffel commented 1 year ago

well did you create a valid sdk now? Also no problem i hope you got a better understanding of the engine now, however a 1 hour video would've done the job too lol