Add Memory Dumps Folder option generates 0 pointers

I've noticed that the option to load an entire folder of dumps results in 0 pointers being shown. Changing none of the other settings, but removing all addresses and adding them individually fixes the issue. I can upload a zip with the dumps if you need.

Yeah, please upload your memory dumps. I suspect it might be related to the naming of your memory dumps since they the addresses are drawn from them.

Sure thing. It's too big for Github, so I'll put it on GDrive. I've got them named as GameTitle_0xStartingAddr_0xEndingAddr_0xAddr.bin and it looks like either way the correct addresses get parsed and filled in. Just something about adding them all at once messes up.

Significant settings for this set are: Depth: 1-2 Offset range: 0-2000 Byte order: Little Starting address: 81000000

https://drive.google.com/open?id=1Jaqe_KPj4G339pz50Hc3KXzpskrfAxrE

It also looks like adding them in different orders results in different pointers being shown, although that's probably just based on the first one added... My TempAR does something similar, so I kinda understand that. But it shouldn't show 0 pointers, no matter what order they're added, so I'm not quite sure what's causing that.

Any ideas about why this would be happening?

I just tried it and it looks normal to me. If I add the entire folder, it gives me 4 memory dumps. When performing a search, I get 0 results. If I delete the 4th memory dump and perform a search, I get 4 results. If I manually add all 4 memory dumps and perform a search I also get 0 results. Where exactly do you see the bug?

As you seem to have noticed, the first memory dump is used as the base and the others are only used to filter the results of the first memory dump. All valid pointers will be outputted regardless of the order they were added since they would pass all memory dump checks.

You can see that happening if you e.g. add the 1st memory dump, then the 3rd, then the 2nd and perform a search. You get 4 results, like before. Also the same thing for adding the 3rd, 2nd and first.

Also note that adding the ending address of the memory dump to the file name is redundant for my software since it will read the file size of the memory dump and therefore does not need to be told where the memory dump ends.

The target address is parsed from the right side of the file name (ignoring the extension), that's why it still detected the address(es) correctly.

The ending address is part of the dumping software... It actually names them Starting_Ending_DumpNumber.bin . It's important to me to know what the ending address is because we ONLY dump what is necessary, so I keep it there. It hasn't seemed to affect the pointer search, so I kept it that way. :P I wasn't sure if you specifically programmed it that way or not, and that explanation makes sense. Glad you did it as you did, as that's very helpful.

The issue is, that if a pointer is found in dump order 1,3,2,4 then it should also be found in dump order 4,3,2,1... at least, I would think it should be... I've found pointers in all 4 dumps that are valid, and can be followed exactly in HxD. Maybe I need to look better at the fourth dump, but I could have sworn that I had followed the same pointers in it.

For instance, this pointer works in all 4 dumps : [[0x81776558] + 0x24] + 0x1238 I have checked in HxD in all 4 dumps, and it's within the 0x2000 offset limit. So this pointer should show up, no matter what order the dumps are put in. I didn't check the other 3 pointers that I've found, but I think I checked them before and they all worked fine.

But, I also just did what you did and took out 1 address from the 4. I wasn't sure which one you took out, so I added the folder, then took out 1 address 4 different times. All resulted in 0 pointers, except when I removed the "...F88.bin" address. Removing that one showed [[0x81776558] + 0x24] + 0x1238 in the pointer list. Following that pointer, in the missing dump found that one of the addresses pointed to was in the 0x89's, outside the dumped range. So that's correct, at least. It explains why THAT pointer doesn't appear if the "...F88.bin" dump is used.

So, then I tried switching the remaining dumps around. Dumps 1-2-3 found the above pointer, so does 1-3-2. but dumps 3-2-1 and 3-1-2 found several pointers, including the above one. But 2-1-3 and 2-3-1 found 0 pointers, not even the one above that I've already checked is valid in all 3 dumps.

I find it odd that simply changing the order of the pointers can either show 1, many or 0 pointers, even if there ARE pointers that can be followed in every single one.

I'm not sure how your pointer searcher decides which pointers to throw out and when, but that seems to be the issue, the more I look into it... Not the adding a folder at a time, but just changing order seems to break some results...

In TempAR (the pointer searcher I kinda take care of) the way it searches I think is that it find all pointers to a single dump, then it finds all pointers to the second dump, and compares the lists and any pointers that they share get "Upgraded" to a higher level. In the next dump, it does the same but only compares the current dump to dump 1 and continues to "Upgrade" pointers. Eventually, you're left with a list of pointers that's earned so many levels (with higher leveled pointers matching more dumps, and lower levels matching only 1 dump)... so no matter the order, if a pointer shows up in all dumps, it will always show up.... unfortunately it also means there's a ton of other addresses of various levels that also show up (Hence why I MUCH prefer Universal Pointer Searcher).... at least they have colors associating the different levels....

I'm not sure how yours works, though... so I'm not sure why sometimes it rejects a pointer simply based on dump order, even if that pointer is valid in all of the dumps.... this post has gotten WAY longer than I meant for it to, I'm so sorry!

Hi @Yohoki, it's been a while. Would you please try out my Universal Pointer Searcher Engine command line application and give your feedback? It might fix your problem with 0 pointers found by re-ordering the memory dumps and/or using the Add memory dumps folder feature. Furthermore, it has validating pointers so you don't have to trace in HxD manually.

It sure has been a while. I've mostly left the hacking scene for vita, because some new script kiddies have started releasing really unstable codes that can cause serious damage, and are absolutely adamant that this is how the scene should move forward... I'm not going to be a part of that movement. XD

But, I do still hack my own games on my free time, so I'll give this one a run and see if it fixes that issue. I should still have my old dumps on an HDD somewhere on my network. I'll check it out this weekend and see if I can figure out how to use it and if it fixes the problem.

edit: I may be doing something wrong?

universalpointersearcher --help
2021-03-03 08:45:25.474 (   0.001s) [        2C554040]               main.cpp:33    INFO| Windows C++ Universal Pointer Searcher Engine v3.3
Release Build
Copyright (c) 2018 - 2021 BullyWiiPlaza Productions
2021-03-03 08:45:25.499 (   0.025s) [        2C554040]   ProgramValidator.cpp:104   INFO| Validating application...
2021-03-03 08:45:25.916 (   0.442s) [        2C554040]               main.cpp:624    ERR| ios_base::failbit set: iostream stream error

I just reinstalled the redist but it's showing errors still.

It sure has been a while. I've mostly left the hacking scene for vita, because some new script kiddies have started releasing really unstable codes that can cause serious damage, and are absolutely adamant that this is how the scene should move forward... I'm not going to be a part of that movement. XD

Exactly, this kind of stuff always happens. Kind of ruins the fun in this "hobby" which is sad. This makes me not want to release any codes or hacks to begin with, people just steal them and abuse my work sooner or later. Anyway...

But, I do still hack my own games on my free time, so I'll give this one a run and see if it fixes that issue. I should still have my old dumps on an HDD somewhere on my network. I'll check it out this weekend and see if I can figure out how to use it and if it fixes the problem.

edit: I may be doing something wrong?
universalpointersearcher --help
2021-03-03 08:45:25.474 (   0.001s) [        2C554040]               main.cpp:33    INFO| Windows C++ Universal Pointer Searcher Engine v3.3
Release Build
Copyright (c) 2018 - 2021 BullyWiiPlaza Productions
2021-03-03 08:45:25.499 (   0.025s) [        2C554040]   ProgramValidator.cpp:104   INFO| Validating application...
2021-03-03 08:45:25.916 (   0.442s) [        2C554040]               main.cpp:624    ERR| ios_base::failbit set: iostream stream error
I just reinstalled the redist but it's showing errors still.

Yep, this just surprised me for a moment since it works flawlessly on 3 of my laptops but I got you haha nice job, you already found a bug. You need to type UniversalPointerSearcher.exe --help for it to work currently. Leaving out the .exe causes the bug. I'm going to make an update to fix that though. Other than that it should be working.

EDIT: I updated it and fixed the bug.

lol, I'm so used to running batch files from cmd that it hadn't even occurred to me that the .exe might be the issue. Ok, i've updated (this is mandatory, it looks like?) I'll read that help file and set it up when I can get a chance to sit down and look at it. I should have a couple hours this weekend that I can sit at my desk and try out moving the dump orders around and things and see if I can break it again. XD

Is "--initial-endian little little little" not correct?

universalpointersearcher --initial-file-path "./PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" "./PCSE00008_0x81000000_0x89000000_0x8573F908.bin" "./PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" --initial-starting-address 0x81000000 0x81000000 0x81000000 --initial-endian little little little --initial-address-size 4 4 4 --target-address 0x859E5AF8 0x8573F908 0x859E5AF8 --comparison-file-path "./PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" "./PCSE00008_0x81000000_0x89000000_0x8573F908.bin" "./PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" --comparison-starting-address 0x81000000 0x81000000 0x81000000 --comparison-endian little little little --target-address 0x859E5AF8 0x8573F908 0x87557BA8 --pointer-offset-range 0x0,0x2000 --pointer-depth-range 1,3 --maximum-pointers-printed-count 1000 --verbose true >results.txt

universalpointersearcher --initial-file-path "./PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" "./PCSE00008_0x81000000_0x89000000_0x8573F908.bin" "./PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" --initial-starting-address 0x81000000 0x81000000 0x81000000 --initial-endian little little little --initial-address-size 4 4 4 --target-address 0x859E5AF8 0x8573F908 0x859E5AF8 --comparison-file-path "./PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" "./PCSE00008_0x81000000_0x89000000_0x8573F908.bin" "./PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" --comparison-starting-address 0x81000000 0x81000000 0x81000000 --comparison-endian little little little --target-address 0x859E5AF8 0x8573F908 0x87557BA8 --pointer-offset-range 0x0,0x2000 --pointer-depth-range 1,3 --maximum-pointers-printed-count 1000 --verbose true  1>log.txt
2021-03-03 19:41:19.772 (   0.000s) [        67D6229C]               main.cpp:39    INFO| Windows C++ Universal Pointer Searcher Engine v3.4
Release Build
Copyright (c) 2018 - 2021 BullyWiiPlaza Productions
2021-03-03 19:41:19.780 (   0.008s) [        67D6229C]   ProgramValidator.cpp:106   INFO| Validating application...
2021-03-03 19:41:19.784 (   0.012s) [        67D6229C]   ProgramValidator.cpp:61    INFO| Validating environment...
2021-03-03 19:41:19.787 (   0.015s) [        67D6229C]   ProgramValidator.cpp:113   INFO| Retrieving latest version details...
2021-03-03 19:41:20.110 (   0.338s) [        67D6229C]   ProgramValidator.cpp:140   INFO| Verifying binary...
2021-03-03 19:41:20.157 (   0.385s) [        67D6229C]   ProgramValidator.cpp:166   INFO| Application validation passed...
The following arguments were not expected: little little little --comparison-endian 4 4 4 --initial-address-size little little little --initial-endian
Run with --help for more information.

Edit: nvm. you changed "--initial-address-size" to "--address-size" at some point, and I was copying the video. XD Makes sense, the game probably isn't going to suddenly go 64bit by itself. lol

update: This error may also be because I tried following the outdated video. I was able to get it to run by having a single "--initial..." and a single "--Comparison..." . Thank god it checks ram usage, because it quickly hit 99% and froze everything, but recovered after a few minutes. lol Thank you for that one.

Original: My head hurts. lol. I'm gonna have to write a batch just to BUILD the search strings. But I gotta get it working first. I'm not sure how to fix this error, now.

universalpointersearcher --initial-file-path "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" --initial-starting-address 0x81000000 --endian little --address-size 4 --target-address 0x859E5AF8 --comparison-file-path "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" "PCSE00008_0x81000000_0x89000000_0x8573F908.bin" "PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" --comparison-starting-address 0x81000000 0x81000000 0x81000000 --target-address 0x859E5AF8 0x8573F908 0x87557BA8 --pointer-offset-range 0x0,0x2000 --pointer-depth-range 1,3 --maximum-pointers-printed-count 1000 --verbose true  1>results.txt
2021-03-03 20:07:29.974 (   0.000s) [        787EDBC7]               main.cpp:39    INFO| Windows C++ Universal Pointer Searcher Engine v3.4
Release Build
Copyright (c) 2018 - 2021 BullyWiiPlaza Productions
2021-03-03 20:07:29.982 (   0.007s) [        787EDBC7]   ProgramValidator.cpp:106   INFO| Validating application...
2021-03-03 20:07:29.986 (   0.011s) [        787EDBC7]   ProgramValidator.cpp:61    INFO| Validating environment...
2021-03-03 20:07:29.989 (   0.014s) [        787EDBC7]   ProgramValidator.cpp:113   INFO| Retrieving latest version details...
2021-03-03 20:07:30.323 (   0.348s) [        787EDBC7]   ProgramValidator.cpp:140   INFO| Verifying binary...
2021-03-03 20:07:30.363 (   0.389s) [        787EDBC7]   ProgramValidator.cpp:166   INFO| Application validation passed...
2021-03-03 20:07:30.368 (   0.394s) [        787EDBC7]               main.cpp:212   INFO| *** We're forced to be donator ***
2021-03-03 20:07:30.371 (   0.397s) [        787EDBC7]               main.cpp:214   INFO| If you appreciate this tool to be made available free of charge in its full version at this time, feel free to donate any amount using the following link: https://www.paypal.me/BullyWiiPlaza
2021-03-03 20:07:30.380 (   0.405s) [        787EDBC7]     MemorySnapshot.cpp:252   INFO| Parsing pointer map from 1 memory snapshot file(s)...
2021-03-03 20:07:30.384 (   0.409s) [        787EDBC7] MemorySnapshotFile.cpp:25    INFO| Getting file size of "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin"...
2021-03-03 20:07:30.388 (   0.413s) [        787EDBC7]          Benchmark.cpp:26    INFO| Getting file sizes took 0.005 second(s)
2021-03-03 20:07:30.393 (   0.418s) [        787EDBC7]          Benchmark.cpp:26    INFO| Building memory snapshot address boundaries took 0 second(s)
2021-03-03 20:07:30.398 (   0.423s) [        787EDBC7]     MemorySnapshot.hpp:79    INFO| Occupied physical memory: 57.8649%
2021-03-03 20:07:30.401 (   0.426s) [        787EDBC7] MemorySnapshotFile.cpp:36    INFO| Loading file "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" into memory...
2021-03-03 20:07:30.549 (   0.574s) [        787EDBC7]          Benchmark.cpp:26    INFO| Loading file took 0.148 second(s)
2021-03-03 20:07:30.581 (   0.606s) [        787EDBC7]     MemorySnapshot.hpp:79    INFO| Occupied physical memory: 61.0511%
2021-03-03 20:07:30.585 (   0.611s) [        787EDBC7]     MemorySnapshot.hpp:111   INFO| Scanning memory dump(s) for valid pointers...
2021-03-03 20:07:30.590 (   0.615s) [        787EDBC7]     MemorySnapshot.hpp:125   INFO| Parsing address/value pairs from memory dump starting offset 0 till 0x8000000...
2021-03-03 20:07:30.641 (   0.666s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 4%
2021-03-03 20:07:30.690 (   0.715s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 8%
2021-03-03 20:07:30.743 (   0.768s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 12%
2021-03-03 20:07:30.795 (   0.820s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 16%
2021-03-03 20:07:30.861 (   0.886s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 20%
2021-03-03 20:07:30.909 (   0.935s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 24%
2021-03-03 20:07:30.977 (   1.002s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 28%
2021-03-03 20:07:31.049 (   1.075s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 32%
2021-03-03 20:07:31.133 (   1.158s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 36%
2021-03-03 20:07:31.182 (   1.207s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 40%
2021-03-03 20:07:31.228 (   1.254s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 44%
2021-03-03 20:07:31.272 (   1.298s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 48%
2021-03-03 20:07:31.319 (   1.345s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 52%
2021-03-03 20:07:31.367 (   1.393s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 56%
2021-03-03 20:07:31.468 (   1.493s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 60%
2021-03-03 20:07:31.517 (   1.543s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 64%
2021-03-03 20:07:31.566 (   1.591s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 68%
2021-03-03 20:07:31.614 (   1.639s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 72%
2021-03-03 20:07:31.666 (   1.691s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 76%
2021-03-03 20:07:31.790 (   1.816s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 80%
2021-03-03 20:07:31.840 (   1.865s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 84%
2021-03-03 20:07:31.890 (   1.916s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 88%
2021-03-03 20:07:31.942 (   1.967s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 92%
2021-03-03 20:07:31.990 (   2.015s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 96%
2021-03-03 20:07:32.168 (   2.194s) [        787EDBC7]ProgressPrintManager.hp:48    INFO| PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin: 100%
2021-03-03 20:07:32.173 (   2.198s) [        787EDBC7]     MemorySnapshot.hpp:79    INFO| Occupied physical memory: 63.64%
2021-03-03 20:07:32.176 (   2.202s) [        787EDBC7]     MemorySnapshot.hpp:175   INFO| Wiping file contents of "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin"...
2021-03-03 20:07:32.190 (   2.216s) [        787EDBC7]     MemorySnapshot.hpp:79    INFO| Occupied physical memory: 62.0961%
2021-03-03 20:07:32.195 (   2.220s) [        787EDBC7]          Benchmark.cpp:26    INFO| Parsing file took 1.609 second(s)
2021-03-03 20:07:32.199 (   2.224s) [        787EDBC7]     MemorySnapshot.hpp:181   INFO| 3583663 address value pair(s) added from the current memory dump file "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin"...
2021-03-03 20:07:32.347 (   2.372s) [        787EDBC7]     MemorySnapshot.cpp:300   INFO| 3583663 address value pair(s) found so far...
2021-03-03 20:07:32.352 (   2.377s) [        787EDBC7]     MemorySnapshot.cpp:301   INFO| 1/1 processed so far
2021-03-03 20:07:32.407 (   2.432s) [        787EDBC7]     MemorySnapshot.cpp:305   INFO| A total of 3583663 address value pair(s) found
2021-03-03 20:07:32.412 (   2.437s) [        787EDBC7]    PointerSearcher.cpp:434   INFO| Initializing pointer map sorted by values...
2021-03-03 20:07:32.515 (   2.541s) [        787EDBC7]    PointerSearcher.cpp:445   INFO| Started sorting pointer map by address and value in parallel...
2021-03-03 20:07:32.521 (   2.546s) [        5CA8C953]     MemorySnapshot.cpp:183   INFO| Checking if the pointer map is sorted by address...
2021-03-03 20:07:32.521 (   2.546s) [        544C7278]     MemorySnapshot.cpp:198   INFO| Sorting pointer map with 3583663 element(s) by value...
2021-03-03 20:07:32.544 (   2.569s) [        5CA8C953]     MemorySnapshot.cpp:192   INFO| Already sorted by address...
2021-03-03 20:07:32.782 (   2.808s) [        544C7278]          Benchmark.cpp:26    INFO| Sorting pointer map by value took 0.261 second(s)
2021-03-03 20:07:32.786 (   2.811s) [        787EDBC7]    PointerSearcher.cpp:452   INFO| Sorting pointer map by address and value in parallel finished...
2021-03-03 20:07:32.790 (   2.815s) [        787EDBC7]    PointerSearcher.cpp:439   INFO| Using target address: 0x859E5AF8
2021-03-03 20:07:32.794 (   2.819s) [        787EDBC7]               main.cpp:399   INFO| Parsing pointer offset range 0x0,0x2000...
2021-03-03 20:07:32.798 (   2.823s) [        787EDBC7]               main.cpp:424   INFO| Parsing comparison memory snapshots...
2021-03-03 20:07:32.802 (   2.827s) [        787EDBC7]MappedMemorySnapshot.cp:28    INFO| Mapping file "PCSE00008_0x81000000_0x89000000_0x859E5AF8.bin" into memory...
2021-03-03 20:07:32.806 (   2.831s) [        787EDBC7]MappedMemorySnapshot.cp:32    INFO| Concurrently opened file count: 1
2021-03-03 20:07:32.810 (   2.835s) [        787EDBC7]          Benchmark.cpp:26    INFO| Mapping file bytes took 0.009 second(s)
2021-03-03 20:07:32.814 (   2.839s) [        787EDBC7]MappedMemorySnapshot.cp:28    INFO| Mapping file "PCSE00008_0x81000000_0x89000000_0x8573F908.bin" into memory...
2021-03-03 20:07:32.818 (   2.844s) [        787EDBC7]MappedMemorySnapshot.cp:32    INFO| Concurrently opened file count: 2
2021-03-03 20:07:32.822 (   2.847s) [        787EDBC7]          Benchmark.cpp:26    INFO| Mapping file bytes took 0.007 second(s)
2021-03-03 20:07:32.826 (   2.851s) [        787EDBC7]MappedMemorySnapshot.cp:28    INFO| Mapping file "PCSE00008_0x81000000_0x89000000_0x87557BA8.bin" into memory...
2021-03-03 20:07:32.831 (   2.856s) [        787EDBC7]MappedMemorySnapshot.cp:32    INFO| Concurrently opened file count: 3
2021-03-03 20:07:32.834 (   2.860s) [        787EDBC7]          Benchmark.cpp:26    INFO| Mapping file bytes took 0.008 second(s)
2021-03-03 20:07:33.010 (   3.035s) [        787EDBC7]               main.cpp:24     ERR| Mismatch between read pointer map file paths, comparison file path vector size and target addresses size
Backtrace:
 0# 0x00007FF79EDCFF40 in UniversalPointerSearcher
 1# 0x00007FF79EDBA822 in UniversalPointerSearcher
 2# 0x00007FF79F0855EB in UniversalPointerSearcher
 3# 0x00007FF79EFAF1E0 in UniversalPointerSearcher
 4# 0x00007FF79EFA5489 in UniversalPointerSearcher
 5# RtlCaptureContext2 in ntdll
 6# 0x00007FF79EDC058A in UniversalPointerSearcher
 7# 0x00007FF79EF728EC in UniversalPointerSearcher
 8# BaseThreadInitThunk in KERNEL32
 9# RtlUserThreadStart in ntdll

Just finished running my first successful run and I'm noticing a few things. you asked for feedback, so I'm gonna give you some feedback. Hopefully it's good and (more importantly) useful to you.

Showing exclusions:

2021-03-03 20:21:38.601 ( 121.914s) [        18FCFCE6]    PointerSearcher.cpp:386   INFO| [[[0x86409AFC] + 0x12E4] + 0x199C] + 0x1A30 has been excluded: {0x86409AFC -> 0x86409AFC -> 0x8640A1B4 -> 0x859E5AF8}
2021-03-03 20:21:38.604 ( 121.918s) [        E7CB91F9]    PointerSearcher.cpp:386   INFO| [[[0x859E4678] + 0xD1C] + 0x940] + 0x1858 has been excluded: {0x859E4678 -> 0x859E4678 -> 0x859E429C -> 0x859E5AF8}

While probably not useful to most users, I REALLY like seeing WHY it's kicked out the pointers. In these 2 cases, it's found a loop and kicked it out. No point following the loop when we can just enter the pointer from further down the line. Makes sense, and I'd have had to checked HxD before I found out that it was looping.

RAM Monitoring:

2021-03-03 20:19:50.633 (  13.946s) [        4369931C]    PointerSearcher.cpp:116   INFO| Memory utilization fraction: 99.2705%
2021-03-03 20:19:50.637 (  13.950s) [        4369931C]    PointerSearcher.cpp:118   WARN| Memory utilization threshold of 99% exceeded
2021-03-03 20:20:00.595 (  23.909s) [        15E44326]    PointerSearcher.cpp:154   INFO| 20412679 new memory pointer(s) found...
2021-03-03 20:20:31.957 (  55.270s) [        15E44326]    PointerSearcher.cpp:170   INFO| Memory pointer count: 20943359

Oh, thank jesus. You were doing me a concern there. O.O I'm concerned that the ram may have halted searching for more pointers, though. But I also don't know an easy way to solve this. I'd rather it stop early, than crash to blue screen, so that's better than before, at least.

Documentation:

The video looks like it's outdated and some of the --help text is not easy to understand. I was able to get it mostly working by watching the video and comparing my string with the --help text, at least... but had I only had one or the other, I don't think it would have worked. It seems like you're not interested in adding a gui for this one, so if you want it to be used, it's got to be clear how to use it. Thankfully, it should be pretty easy to make a gui (or even a prompt system with a bat file) that simply takes in the user's info, and passes it all to the cmd. I'll probably make a bat file for myself, so if you haven't made anything by then, I'll let you know in case you want to use it.

Speed:

took 1500 seconds to run for me. My RAM got in the way, and I think my Virus Scanner was also doing a scan at the same time, so I probably had a huge throttling issue going on. but, here's how the times went.

Initial Scan - 52 Seconds - 20.9million results
Excluding Cycles - 1211 seconds - 20.8million results
Validation - 122 seconds - 5,000 results
Sorting - .05 seconds - 5000 results
Excluding Duplicates - .001 seconds - 2550 results
Trimmed (Final) - 1000 results

I do have one concern here. The cycles step took up the most amount of time, and only excluded a very small percent of the pointers. Could this be moved to a later time in the order of events? Say, after validation, but before duplicates removal? I'm not sure what the validations step does, yet. But if it won't break it to run that step slightly slower, it would maybe cut the running time down by a lot if the excluding cycle was performed with only 5000 results, instead of 20 million.

Printing pointers:

I didn't see where it printed the pointer out at..... but I also ran it with ">Results.txt" in the commandline.... so If it was supposed to print out to that already, that would be why.... it also didn't display the pointers on screen in the cmd window, but again I printed it out to results.txt hoping to also catch the errors and things in the txt too.... So, that one's probably me messing up your code.

Overall, I like it. It's a bit clunky right now as I'm learning it, and it could really use some documentation.... but I like it so far, and it should be pretty easy to make something to pass the arguments to it, so that noobs don't bother you with questions for years. :P

Thanks a lot for taking the time to write such detailed feedback, it's highly appreciated!

lol, I'm so used to running batch files from cmd that it hadn't even occurred to me that the .exe might be the issue. Ok, i've updated (this is mandatory, it looks like?)

Yes, it's mandatory.

Edit: nvm. you changed "--initial-address-size" to "--address-size" at some point, and I was copying the video. XD Makes sense, the game probably isn't going to suddenly go 64bit by itself. lol

Exactly, I renamed those 2 switches.

update: This error may also be because I tried following the outdated video. I was able to get it to run by having a single "--initial..." and a single "--Comparison..." . Thank god it checks ram usage, because it quickly hit 99% and froze everything, but recovered after a few minutes. lol Thank you for that one.

Original: My head hurts. lol. I'm gonna have to write a batch just to BUILD the search strings. But I gotta get it working first. I'm not sure how to fix this error, now.

Yeah, the check is:

if (read_pointer_maps_file_paths.size() + comparison_file_paths_vector.size() != target_addresses.size() - 1) {
    throw std::invalid_argument("Mismatch between read pointer map file paths, comparison file path vector size and target addresses size");
}

Essentially the target address count needs to be 1 less than the comparison file paths.

Documentation:

The video looks like it's outdated and some of the --help text is not easy to understand. I was able to get it mostly working by watching the video and comparing my string with the --help text, at least... but had I only had one or the other, I don't think it would have worked. It seems like you're not interested in adding a gui for this one, so if you want it to be used, it's got to be clear how to use it. Thankfully, it should be pretty easy to make a gui (or even a prompt system with a bat file) that simply takes in the user's info, and passes it all to the cmd. I'll probably make a bat file for myself, so if you haven't made anything by then, I'll let you know in case you want to use it.

Yeah, I think the --help documentation is kind of enough to get it working but it requires some thinking and learning I guess. There is definitely some more room for improvement here though. The design of the tool is quite new so I was hoping for some good feedback so I can finalize it eventually and then add more documentation. Users seemingly want a GUI badly though. I might actually do that later after all.

Speed:

took 1500 seconds to run for me. My RAM got in the way, and I think my Virus Scanner was also doing a scan at the same time, so I probably had a huge throttling issue going on. but, here's how the times went.

What happens here is that you're at 99% RAM but the excluding cycles step goes through the pointer list and copies over the ones without cycles into a new list which duplicates your RAM usage temporarily until it's done. Since you're already maxed out, it will slow down tremendously due to Windows writing out RAM pages to the disk and swapping them back. One suggestion would be to set the RAM threshold to 85% (this can be configured via command line argument).

Changing the order of the steps is a good idea, I will think about that.

Printing pointers:

I didn't see where it printed the pointer out at..... but I also ran it with ">Results.txt" in the commandline.... so If it was supposed to print out to that already, that would be why.... it also didn't display the pointers on screen in the cmd window, but again I printed it out to results.txt hoping to also catch the errors and things in the txt too.... So, that one's probably me messing up your code.

Normally they are all printed at the end but via command line switch you can specifically write only the pointer results to a file. You don't need to do the piping.

Thanks a lot for taking the time to write such detailed feedback, it's highly appreciated!

No problem. Hopeful it's useful. I don't know how to code well, so I gotta rely on people like you making things that work good. :P

Yeah, the check is:

if (read_pointer_maps_file_paths.size() + comparison_file_paths_vector.size() != target_addresses.size() - 1) {
  throw std::invalid_argument("Mismatch between read pointer map file paths, comparison file path vector size and target addresses size");
}

Essentially the target address count needs to be 1 less than the comparison file paths.

I'm not sure I'm understanding this part. I was trying to copy your video, and it looked like the first example showed you adding 2 dumps as "Initial" dumps and the same 2 again as "Comparison" dumps. I assumed this was to avoid the issue I was having with re-ordering dumps, by running the check against multiple types of both. But, again, the video is probably outdated and has maybe changes considerably since uploading. I'll have to play with it more to figure out how to use it well.

I did notice, though that the video only showed 1 target address instead of 2, since there were 2 dumps added as initial/comparison. which didn't make sense to me, so I tried adding 1 address per dump. Possibly that's the issue?

Yeah, I think the --help documentation is kind of enough to get it working but it requires some thinking and learning I guess. There is definitely some more room for improvement here though. The design of the tool is quite new so I was hoping for some good feedback so I can finalize it eventually and then add more documentation. Users seemingly want a GUI badly though. I might actually do that later after all.

Some of it looks a bit confusing if you don't already know how the input should look... Like this for instance:

--pointer-offset-range TEXT The pointer offset range to use for the pointer search

From the video, I saw this should be "1,3". but will "1-3" work? will simply putting "3" work? If I hadn't seen the video and read the text on screen, it wouldn't have been clear to use a comma, and I probably wouldn't have thought to use one at all. An example would clear it up immediately, for those that don't already know what it should look like. I don't think a GUI is necessary, and if it's tiresome to make one (I know that part is boring) then don't worry about it. There's plenty of command line tools out there for other console things. And what you're making isn't necessarily for the beginner. But I also know you'll get people asking "How use this it broke?" if you don't. So it'd probably be best to at least make sure the readme is clear. I'm also pretty passionate about having easy-to-use interfaces.... So if it's not something you're interested in, don't let my feedback here sway you, because I'm going to be overly talkative about this particular part to begin with. XD

Plus, I will most likely make a batch file myself for this that steps through each of the arguments and gives a prompt. If I make it, I'll gladly share it with you to do what you want with. :)

What happens here is that you're at 99% RAM but the excluding cycles step goes through the pointer list and copies over the ones without cycles into a new list which duplicates your RAM usage temporarily until it's done. Since you're already maxed out, it will slow down tremendously due to Windows writing out RAM pages to the disk and swapping them back. One suggestion would be to set the RAM threshold to 85% (this can be configured via command line argument).

Changing the order of the steps is a good idea, I will think about that.

I did run the search again shortly after with the throttle set to 80, and it does look like it cuts off search when the throttle limit is reached.... which is unfortunate, because at 80% I had almost 5million less pointers when done scanning. I definitely lose a good amount of starting pointers before the validations and exclusions for that cost of extra speed. How to go about getting those back in? I don't know... maybe I can step through depths and only search 1 level deep at a time, and then increase to only 2 levels, or something like that. But I'm not sure this would even save any ram usage... ah. I'm not that worried about it. It's my old Mobo's 8gb limit that's the real issue here.

But changing the order of steps might help reduce the time waiting at the end. The loops checking seems to take the longest amount of time while cleaning up, so saving it until later might be a good thing. You'd have to try it out and see if it does anything good or just makes the rest take longer. I can't see your code, so I can't even guess what all's going on. :P

Normally they are all printed at the end but via command line switch you can specifically write only the pointer results to a file. You don't need to do the piping.

Ah. Ya, running it again without the pipe displays it on screen. No need to pipe anyway, because the pipe isn't catching any of the error messages or anything anyway. I'll just use the switch, then. lol

From the video, I saw this should be "1,3". but will "1-3" work?

No. You need to give it the start value, a comma and an end value. I added this clarification to the --help output.

I did notice, though that the video only showed 1 target address instead of 2, since there were 2 dumps added as initial/comparison. which didn't make sense to me, so I tried adding 1 address per dump. Possibly that's the issue?

Initial means "the set of memory dumps to use as the initial memory snapshot", e.g. some systems have more than 1 memory dump which need to be treated as a single input but it only has 1 target address no matter how many memory snapshot files it is divided into. In your case it's only 1 memory snapshot file though. The comparison memory snapshots are all additional memory dumps.

In the video it was I assume the MEM80 and MEM90 dump from Super Mario Galaxy. The comparison memory snapshots are the ones you want to compare/validate the pointers against to sort out the pointers which don't work on all memory dumps.

Since you asked for better documentation, I wrote a glossary of the terms I'm using in the README.md of the project which should be very helpful. Also I improved some error messages to explain the reasons for the invalid input better.

Another thing you can do is save pointers to a file and then load that again to validate against another memory snapshot if you don't want to repeat the whole search with another comparison memory snapshot added. This way you can keep filtering more efficiently.

Also I added a value to disable memory pointer sorting at the end since it can be very slow on like 100 million pointers lol.

I did run the search again shortly after with the throttle set to 80, and it does look like it cuts off search when the throttle limit > is reached.... which is unfortunate, because at 80% I had almost 5million less pointers when done scanning.

Yes, it stops collecting more pointers when the RAM limit is reached. However, I now implemented an improvement by re-ordering the filtering operations and also saving RAM by deleting from the list instead of setting up a new one. This means it will run through more smoothly with a RAM limit of 99%. I just finished a pointer search with 117.941.132 pointers in ~ 569.865 second(s) but my laptop has 32 GB (20 GB were temporarily taken by the pointer searcher). However, I'm wondering why you really need that many results. If you want even more, maybe we can do something to cut down on those huge pointer lists before they happen? It would speed everything else up as well. You may try some of the other command line options like the concrete last pointer offsets.

No. You need to give it the start value, a comma and an end value. I added this clarification to the --help output.

Lol. I'm aware of this now, but if I had only the --help, I wouldn't have even thought of adding a comma... I'd have probably tried "1-3" or "3" or a few other things. But a comma wouldn't have been my goto. Adding examples to the --help would help, I bet. If I couldn't figure it out, and I'm used to hacking and pointer searching, a newbie def won't. :P

Initial means "the set of memory dumps to use as the initial memory snapshot", e.g. some systems have more than 1 memory dump which need to be treated as a single input but it only has 1 target address no matter how many memory snapshot files it is divided into. In your case it's only 1 memory snapshot file though. The comparison memory snapshots are all additional memory dumps.

In the video it was I assume the MEM80 and MEM90 dump from Super Mario Galaxy. The comparison memory snapshots are the ones you want to compare/validate the pointers against to sort out the pointers which don't work on all memory dumps.

Ah. Yes, I did noticed one was MEM80 and MEM90. I wasn't aware that Wii had to have them dumped separately, as I don't own a wii. On Vita, we get to choose the range we want to dump, which is usually 81-83, but it can go into the 90s sometimes, but it's usually all one big file... I've had to make cuts sometimes and add 00s in as padding on a few occasions where there's random bits of system mem tucked inside my dumping areas, and the vita crashes if you try to read these protected areas... so having that option to use split files might actually be useful for us as well.... although, it's quite rare that I've had to do that, and I don't recall anyone else mentioning that on vita yet.

Since you asked for better documentation, I wrote a glossary of the terms I'm using in the README.md of the project which should be very helpful. Also I improved some error messages to explain the reasons for the invalid input better.

Nice. I'll give that a read, but I think I understand... most of the terminology, myself. But, some of it seems unique to your wii hacking, and I won't know about it yet. The better error messages will definitely help, though.

Another thing you can do is save pointers to a file and then load that again to validate against another memory snapshot if you don't want to repeat the whole search with another comparison memory snapshot added. This way you can keep filtering more efficiently.

I will give that a try this weekend when I have the free time to sit down and actually do more than one run at a time. :P

Yes, it stops collecting more pointers when the RAM limit is reached. However, I now implemented an improvement by re-ordering the filtering operations and also saving RAM by deleting from the list instead of setting up a new one. This means it will run through more smoothly with a RAM limit of 99%. I just finished a pointer search with 117.941.132 pointers in ~ 569.865 second(s) but my laptop has 32 GB (20 GB were temporarily taken by the pointer searcher). However, I'm wondering why you really need that many results. If you want even more, maybe we can do something to cut down on those huge pointer lists before they happen? It would speed everything else up as well. You may try some of the other command line options like the concrete last pointer offsets.

Vita's been pretty difficult to crack some of the games, lol. sometimes the pointers start WAY in the end of the dumps, but are usually rooted somewhere in the 81-83 areas. I think it's because of the way the ARM processor works, by having 2 segments for every module, so those 2nd segments can sometimes be put really far away, even though the first segment is early in RAM. So, having those extra 5million pointers may be necessary for vita, especially if they're from later in the RAM.

But, then again, the vita ALSO has a weird issue where almost every game just SOMETIMES loads 0x1000000 bytes out of place, and we have no idea why... There's so much going on on the vita that I WISH I knew more about programming it to help explain what's going on, but I'm climbing mount stupid here. I make good codes, and do some really advanced things that only I know how to do.... but when explaining it to people, I can only give my experience, I don't have the book knowledge of the inner workings to explain what's causing these things we see.... And, unfortunately, since the switch came out, most of the technical people have moved to is as the new handheld, so we're a dying console. :(

AND THEN some games, have pointers that aren't static addresses, but dynamically rooted at a static offset from the start of it's memory segment.... so the base address changes every time the memory segment loads or shifts location... Vita's a demon.... don't try hacking it... It will eat your soul away. lmao.

Either way, I'm glad my feedback pushed you to making it more efficient. Good job. That was a pretty quick fix that will probably help a lot. Thank you a lot for that.

If you want even more, maybe we can do something to cut down on those huge pointer lists before they happen? It would speed everything else up as well. You may try some of the other command line options like the concrete last pointer offsets.

Hmm... do you mean something like stopping the pointer search midway and doing the excluding/validating/etc and then resuming where it left off? I don't know what would make it more efficient here. You're the expert on your program, so I'll have to trust your word if you think it'll make it more ram efficient.

Also... Thank you for your hard work. I don't give you the feedback expecting anything to be changed in the code at all. So, seeing you read it, and then immediately work on a patch or think up a new design is greatly appreciated, but not at all expected.

Since I took a while to release this, I wanted to make it absolutely perfect. You inspired me again to improve and fix a couple things. I'm looking forward to your more detailed feedback once you get around to doing more in-depth testing. I don't think any of my terminology is Wii specific, I took many systems into account while designing this. It even works on Windows games since they have modules which need to be dumped individually and they can be loaded by passing the folder of modules as argument. Furthermore, it has been tested and working for the 3DS, Switch, Vita (as you know), the Nintendo Wii and Wii U at least. Essentially the tool's flexibility supports every system in existance using that terminology. One of the reasons I didn't want to make a GUI is because it gets really complex to support all use cases cleanly with a GUI. Usually a GUI allows less flexibility than using the command line interface as well.

Ya, I wouldn't do a GUI, either, I don't think. Working with TempAR, it's original creator made it have a GUI, but that limited it to only comparing 2 dumps at once, which works but there's no reason why you can't compare more than 2 dumps to make it better. I DID add that and it was a pain in the butt (plus my code is sloppy as hell since I didn't know C# at all) and the GUI was looking pretty ugly for a while. But even now that I've added up to 6 dumps, there's no reason why it COULDN'T compare more than 6, and people have asked for it.... but there's no room left on the GUI, even after I made it larger... So, I'm not adding it. XD

Well, in that case you're doing it wrong, you should use something more scalable like a table/list similar to how I did it in the Java Universal Pointer Searcher GUI and not duplicating the items/fields for each additional memory dump. This way you can allow people to add unlimited rows. However, managing this with all options and possibilities I'm providing in the command line tool gets out of hand quickly for GUI construction and it's so much unnecessary work when you can already use the pointer searcher like this, it's only less user friendly. Normally, I'm also a huge fan of maximum usability but command line tools can have their charm as well. This tool isn't intended for the masses of noobs and kids anyway. Without those guys a tool cannot become super popular which is okay for this one, only active game hackers will be interested in this and they should have a few more brain cells to put to work than the noobs and kids who can't work with the command line.

Well, tempAR was a program I only took over from the PSP days. So, I just edited what was already there... Didn't even have the source code, I had to use DotPeek to decompile it. So, adding more dumps meant cut+Pasting bits of code and hooking them up to boxes in the GUI. I would have loved to make a list like in UPS, but the way it displays pointers is a bit flawed to begin with... Adding more dumps just increases the number of pointers, and it never goes through and erases the ones that don't work in later dumps... It needs a rework, but I'm thinking that'd take a complete rebase, and I've only just made my first C# program from scratch a couple days ago. So, I've just added a few extra features to make vita hacking easier, and left it as is with the source available. Insanely fun project that I'm proud of, but it does have it's flaws.

I do agree with the charm of CMD apps. Most of my work is batch-based. :P I still try to make my programs noob friendly, because I make several for my wife who is def a noob with computers,

Well, tempAR was a program I only took over from the PSP days. So, I just edited what was already there... Didn't even have the source code, I had to use DotPeek to decompile it. So, adding more dumps meant cut+Pasting bits of code and hooking them up to boxes in the GUI. I would have loved to make a list like in UPS, but the way it displays pointers is a bit flawed to begin with... Adding more dumps just increases the number of pointers, and it never goes through and erases the ones that don't work in later dumps... It needs a rework, but I'm thinking that'd take a complete rebase, and I've only just made my first C# program from scratch a couple days ago. So, I've just added a few extra features to make vita hacking easier, and left it as is with the source available. Insanely fun project that I'm proud of, but it does have it's flaws.

Well, yeah, it's quite a mess then and would need a rewrite. Working with just dotPeek or decompiling in general for further development is shockingly bad. Also I added the ability to scan for deeper pointers similar to how double-clicking an entry in TempAR works using the Universal Pointer Searcher Engine as well as updated the README. Now this tool should be feature complete. It has taken a long time but I'm glad I put all this effort into it. We just need this tool to receive more wide-spread use and adoption since hardly anyone outside of the geek world even likes command line applications despite the fact that they can very powerful and neat but could have a steep learning curve.

BullyWiiPlaza / Universal-Pointer-Searcher