search

BullyWiiPlaza / Universal-Pointer-Searcher

An application for finding memory pointers
Apache License 2.0
24 stars 1 forks source link

Changing Offset ranges breaks searches #10

Closed Yohoki closed 5 years ago

Yohoki commented 5 years ago

Edit: The first few posts have incorrect info. New info begins at comment 10. https://github.com/BullyWiiPlaza/Universal-Pointer-Searcher/issues/10#issuecomment-527346731

Trying to search for pointers with depths larger than 2 seems to break the searches and displays 0 results. I saw a thread from last year with this same issue, but it seems to be broken again.

Set to search for pointer depths 1-2 image Set to search for pointer depths 1-3 image

BullyWiiPlaza commented 5 years ago

Nice, thanks. Now I just need you to upload your memory dumps and all the information you entered for the files like byte order etc. in order to investigate and fix it shortly. :)

Yohoki commented 5 years ago

Can do. I'll do a picture of the settings screen and give the list of addresses. Is that enough?

The example I used was using little endian, stating address of 81000000 and was the following dumps, in order: d3, d4, d5, d6, d1, d0, d2 https://drive.google.com/open?id=1dEUKVz8GrsKIvVytB74GCYAoIORzjGHk image

Yohoki commented 5 years ago

Also, I just wanted to let you know, aside from 3+ deep pointers not working ATM, this works great for PS Vita dumps. It takes AGES to find pointers in TempAR and to find a list of only ~20 WORKING pointers in seconds is mind blowing.

I'm used to seeing a list of 2000+ pointers that maybe work sometimes if you squint and pray to the right gods and they kinda half like your offerings.

So, Thanks for making this program! :D

BullyWiiPlaza commented 5 years ago

Great to hear. :) Yes, I spent a lot of time on optimizing the pointer search runtime and the (native) search engine is written in C++, so it is very efficient.

Can you also send me the target addresses for each file? I can see them in your screenshot but not for which respective files they are. Thanks.

Yohoki commented 5 years ago

My bad. It's "PCSF00309.psv" just rename to .txt or open in notepad. I forgot that not everyone has those to open in notepad. XD That's the format PS Vita's VitaCheat plugin uses to load codes, but it's really just a .txt file.

PCSF00309.txt

BullyWiiPlaza commented 5 years ago

Alright. The bug was related to negative values. I do not recommend scanning for negative pointer values since those are usually not reliable pointers. Anyway, I fixed the bug so it should work correctly now. :)

Yohoki commented 5 years ago

Vita actually uses negative pointers pretty frequently. So it's pretty necessary to have, at least for our scene. So thank you for looking into it. :D

You're right. Setting the searcher to offsets 0-1000 on v3.6 does let me do deeper searches. It seems the issue has to do with the Offset Range.

So, using the 3.7 version, with settings and dumps set up exactly the same as before, here's what I've noticed: Searching for offsets between 0 - 3998 results in pointers found as expected, but searching for between 0-4000 cuts the search to 0 again. Raising the Maximum past 4000 breaks the searcher.

Searching between -6000 - 1310 finds pointers in the correct range as expected again. But Increasing the max range to 1314 again breaks the searcher and results shows 0.

The 3.6 version still lets me search higher offsets. I went to 10000 successfully, but then I crashed the computer while searching for 0-100000 (probably overkill, but I just wanted to check if it worked XD)

I don't often find pointers that high, but it's not uncommon on vita. God of War uses some in the 0xe000 range and Soul Sacrifice has some in the 0xe5000

To avoid making another post, This part of the Debugging Output looks funny to me:


2019-09-03 02:41:27.918 (   0.593s) [main thread     ]    PointerSearcher.cpp:369   INFO| Pointer map size prediction validation: 4.37 times the actual amount
2019-09-03 02:41:27.918 (   0.594s) [main thread     ]          benchmark.cpp:30    INFO| Parsing pointer map took 0.564 second(s)
2019-09-03 02:41:27.919 (   0.594s) [main thread     ]    PointerSearcher.cpp:179   INFO| 575221 address value pair(s) found
2019-09-03 02:41:27.929 (   0.604s) [main thread     ]    PointerSearcher.cpp:538   INFO| Sorting pointer map by value...
2019-09-03 02:41:27.954 (   0.629s) [main thread     ]          benchmark.cpp:30    INFO| Sorting pointer map took 0.031 second(s)
2019-09-03 02:41:27.956 (   0.631s) [main thread     ]    PointerSearcher.cpp:897   INFO| Using 8 threads with 8 hardware thread contexts...
2019-09-03 02:41:27.956 (   0.632s) [main thread     ]    PointerSearcher.cpp:906   INFO| Finding pointers with depth 1...
2019-09-03 02:41:27.957 (   0.632s) [main thread     ]    PointerSearcher.cpp:913   INFO| 710 memory pointer(s) found
2019-09-03 02:41:27.957 (   0.633s) [main thread     ]          benchmark.cpp:30    INFO| Pointer depth took 0.001 second(s)
2019-09-03 02:41:27.957 (   0.633s) [main thread     ]    PointerSearcher.cpp:906   INFO| Finding pointers with depth 2...
2019-09-03 02:41:27.958 (   0.633s) [main thread     ]    PointerSearcher.cpp:917   INFO| Chunking 710 memory pointers into 8 part(s)...
2019-09-03 02:41:28.068 (   0.743s) [main thread     ]          benchmark.cpp:30    INFO| Pointer depth took 0.11 second(s)
2019-09-03 02:41:28.101 (   0.777s) [main thread     ]    PointerSearcher.cpp:961   INFO| Maximum pointers count of 100000 reached
2019-09-03 02:41:28.102 (   0.777s) [main thread     ]    PointerSearcher.cpp:781   INFO| Strictly removing pointers with bad last offsets and shallow ones...
2019-09-03 02:41:28.125 (   0.801s) [main thread     ]          benchmark.cpp:30    INFO| Removing bad last offset and shallow pointers took 0.023 second(s)
2019-09-03 02:41:28.130 (   0.805s) [main thread     ]    PointerSearcher.cpp:965   INFO| Memory pointer(s) count: 100000
2019-09-03 02:41:28.130 (   0.806s) [main thread     ]    PointerSearcher.cpp:604   INFO| Removing invalid pointers...
2019-09-03 02:41:28.130 (   0.806s) [main thread     ]    PointerSearcher.cpp:606   INFO| Using memory mapped file reading: true
2019-09-03 02:41:28.131 (   0.806s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 2/7
2019-09-03 02:41:28.131 (   0.807s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.965 (   3.640s) [main thread     ]    PointerSearcher.cpp:768   INFO| Updated memory pointers count: 0
2019-09-03 02:41:30.967 (   3.643s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 3/7
2019-09-03 02:41:30.968 (   3.644s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.968 (   3.644s) [main thread     ]    PointerSearcher.cpp:768   INFO| Updated memory pointers count: 0
2019-09-03 02:41:30.969 (   3.644s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 4/7
2019-09-03 02:41:30.969 (   3.645s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.969 (   3.645s) [main thread     ]    PointerSearcher.cpp:768   INFO| Updated memory pointers count: 0
2019-09-03 02:41:30.970 (   3.645s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 5/7
2019-09-03 02:41:30.970 (   3.645s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.970 (   3.646s) [main thread     ]    PointerSearcher.cpp:768   INFO| Updated memory pointers count: 0
2019-09-03 02:41:30.971 (   3.646s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 6/7
2019-09-03 02:41:30.971 (   3.646s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.971 (   3.647s) [main thread     ]    PointerSearcher.cpp:768   INFO| Updated memory pointers count: 0
2019-09-03 02:41:30.972 (   3.647s) [main thread     ]    PointerSearcher.cpp:613   INFO| Processing memory dump 7/7
2019-09-03 02:41:30.972 (   3.648s) [main thread     ]          benchmark.cpp:30    INFO| Reading additional memory dump took 0 second(s)
2019-09-03 02:41:30.972 (   3.648s) [main thread     ]          benchmark.cpp:30    INFO| Removing invalid pointers took 2.842 second(s)
2019-09-03 02:41:30.973 (   3.648s) [main thread     ]    PointerSearcher.cpp:968   INFO| Memory pointer(s) count: 0
2019-09-03 02:41:30.973 (   3.649s) [main thread     ]          benchmark.cpp:30    INFO| Printing pointers output took 0 second(s)
2019-09-03 02:41:30.973 (   3.649s) [main thread     ]    PointerSearcher.cpp:971   INFO| 0 memory pointer(s) found

I don't know the code, and I won't even try to look because you know it MUCH better than I do... But It looks like it's found 50k pointers and something happened and we lost them during the "Strictly removing pointers with bad last offsets and shallow ones..." part.

BullyWiiPlaza commented 5 years ago

After thinking about this for a while, I realized that this isn't a bug. You have to increase the maximum pointers count to get more results since it cuts off when it hits the limit. The limit is based on found pointers in the first memory dump. Starting from the 2nd memory dump, it simply filters the initial list. It is possible to get less results when increasing the offset range because different pointers will then make up the results list which could be more inaccurate.