Pull request an option to have 128mb of memory to main repo?

[ThirteenAG]

Description

It would be useful for this issue: https://github.com/PCSX2/pcsx2/issues/5539

Reason

Custom code can be injected to a memory region that won't be overwritten by the game, which is essential for having plugins support.

Examples

-

[xTVaser]

I've wanted to do this, but it is non-trivial to do at runtime -- all the current code that sets up the memory layout is mostly compile-time. A potential idea that was floated was to always allocate 128mb and then under normal circumstances it would only use 32mb of that. This sounds wasteful to me though.

Additionally, having 128mb of extra memory to use for extra plugin support is not really the use-case? The point of having 128mb of memory is to simulate what the original PS2 devkit has at it's disposal. I feel like adding to the EE memory to support such a feature is an anti-pattern and going to cause issues -- but i have no idea what is intended with these plugins so I may be wrong.

I'd encourage you to make an issue on the upstream repo, nothing will really come from it being made here. I really have no incentive to do this myself as I would still need to ship the custom patch files for the jak games that enable their dev-mode -- and that is something that would never merge in upstream.

Feel free to take my code if you want to PR it yourself as well -- I don't care about credit or anything over such a trivial changeset

[ThirteenAG]

I feel like adding to the EE memory to support such a feature is an anti-pattern and going to cause issues -- but i have no idea what is intended with these plugins so I may be wrong.

I have a concept here: https://github.com/ThirteenAG/PCSX2PluginInjector#experimental-project

Also tested your binaries and redirected a call to >32mb memory region doing some nops and return, it worked. So the idea is, to allocate custom code and data in that memory the game knows nothing about, so that code doesn't get overwritten by game functions.

Feel free to take my code if you want to PR it yourself as well -- I don't care about credit or anything over such a trivial changeset

Can you link me a specific commit (or few) with these changes? With the number of branches and other commits, kinda hard to look it up.

[xTVaser]

This should still be accurate - ignore anything that isn't a cpp file https://github.com/xTVaser/pcsx2-rr/compare/master...128mb-build the rest is related to the build / the Jak games.

[ThirteenAG]

Thanks a ton!

[ThirteenAG]

ICYMI, I made a PR here: https://github.com/PCSX2/pcsx2/pull/5758 Seems like they requested a review from you, would be much appreciated if you could check it out.

[ThirteenAG]

If you don't mind, going to continue conversation here, since this post went pretty much unnoticed with all the useless drama going on.

I was able to boot games with configurations that require 128mb to load properly.

Can you tell more about those games? Which are they, and how come they do not require GetMemorySize() syscall to return more than 32mb? Was that syscall actually used in retail games?

do you think it would be possible to make this an arbitrary amount in the future? Doesn't seem too far off, and I imagine for the use-case you're going for a configurable amount would be more useful than just 4x as much (although maybe that's plenty?)

I would assume so, but the main reason I chose your implementation, because I assumed it was at least tested on some use case and working. The issue I was banned for, that IOP memory offset will be shifted for some cheat engine users, I still believe is out of scope of my PR, since memory layout changed several times before and then some when going to x64 build. Why it was crucially imperative for me to preserve it, I don't know. Plus they mentioned a savestate incompatibility between different memory sizes, again, I'm not a pcsx2 dev, I wouldn't know how that supposed to be handled properly and I understand you didn't think about that either? But applying to my use case, that is to have custom code and data in that memory region, I suppose the sky is the limit. My largest plugin is 1.58 MB at the moment, because it increases some ingame limits, that require new arrays that I preallocate within plugin. I might try to do some tests later to see if more than 128mb works as well, but for now, that amount is indeed plenty, in comparison to original 32mb. Initially I tried to work with that using in-game malloc function, to shield the code I inject to memory, but that was not enough, I think it was less than megabyte I was able to allocate without the game crashing on loading.

I only tested the current wx GUI and not Qt.

On an unrelated note, perhaps you know why it's not possible to set game window size in QT version? I only could resize window with mouse cursor, it's strange.

[xTVaser]

As I said above, I provide the 128mb build along with patch files for all the jak and daxter games to use their "dev mode". This mode assumes you are running on a PS2 TOOL, which has 128mb unlike the release version of the console.

In official sony documentation, it says that games are not allowed to depend on the size of memory somewhere, so that's probably why the game does not detect anything with GetMemorySize(). No citation on this.

I read through some of the back and forth on the PR -- most of it did seem like valid backwards compatibility concerns. So I'm now re-assured that I won't be wasting my time on trying to get it merged into the upstream repo as its way too much work to do it properly with no benefit to me.

Your use-case sounds interesting and valuable to the people that care about it but, to be blunt I don't have any interest in it myself.

As for Qt, it's a work in progress -- not sure what you are referring to exactly as I usually just play on the lowest settings in native.

[ThirteenAG]

I'm mostly curious how memory management is done internally in those games. Suppose someone would want to make a plugin, or an extension if you will, for the game that uses 128mb, even if just to conduct some test or something, would it make sense for me to have for example, 256mb memory limit and only use region after 128mb for custom code allocation? Personally I only plan to have plugins/mods for GTA, splinter cell and burnout 3, unfortunately none of them require more memory. I'll try to get one of the games you mentioned and check it out.

most of it did seem like valid backwards compatibility concerns.

Indeed so, but the main point for some reason was to be compatible with cheat engine, which I tried to debate. Guess not allowed to have an opinion about that.

Your use-case sounds interesting and valuable to the people that care about it but, to be blunt I don't have any interest in it myself.

Understandable, it's no problem to have it in a fork, my reasoning behind making a PR was "it can't hurt to ask", but now that I lost the ability to fork, I guess it can 😄

Ah, and one final question, that PS2 TOOL mode in these games, does that require patching the game to enable it? Or it's specific game builds that have it? Sorry if the answer is obvious, I haven't checked the repo closely yet.

[xTVaser]

Hmm I'm not sure, I think supporting games that do use 128mb is so incredibly niche that I personally wouldn't concern myself with it, especially on a dedicated fork. To my knowledge, no PS2 game could use more than that (there wasn't a bigger dev kit for example). So if you wanted to, 256MB would likely be more safe.

And yeah, it requires patching of the game -- you have to flip some flags in the ELF and mess with the stack pointer a bit. ex - https://github.com/xTVaser/pcsx2-rr/blob/128mb-build/bin/cheats/9184AAF1.pnach