Add compatibility with OpenPS2Loader VirtualMemoryCards

[Nerdy314159265]

I'm using PCSX2 v1.4.0

I'm unsure of what exact differences there are in the file structure. A cursory look with a hex editor shows that they are very similar, however, the PCSX2 file is slightly larger and uses the .ps2 extension instead of .bin.

So I'd recommend either changing the format to match OPL or convincing the devs of OPL to change to match PCSX2 (which might not be possible considering their limited to the original hardware), or if possible adding support for OPL VMCs on top of the current system.

Also I'd like to recommend a new feature probably similar to the folder based memory. The ability to either set game specific memory cards or to have memory cards automatically swap between games possibly based on card names, for example SCUS_97199.bin might be the name of a OPL memory card and I would expect loading Ratchet and Clank would automatically load that memory card.

The reason I'm suggesting this feature is because OPL supports loading games and VMCs over SMB/CIFS. This feature would allow someone to direct PCSX2 and OPL to the same folders to allow someone to save and quit a game on PS2, then pick it up on their laptop or other device.

[vsub]

PCSX2 have a FolderMemory Card feature which makes any game to have it's own "card"(not really a card) All of the saves are placed in a folder and while that option is enabled,pcsx2 only load saves for the game you are playing You cannot modify\delete saves for one game from another like the normal memory card allows

The folder memory card allows you(since they are just folders)to just copy your saves to the PS2 memory card(even the OPL memory card)while using uLaunchELF

[Nerdy314159265]

Are the individual saves the same as the standard 8MB card created by default but with the addition of the auto-swap feature?

If so, then the folder based memory card sounds like it is already half of what I need but that doesn't sound like it takes care of the inherent file incompatibility between OPL and PCSX2. I'm trying to set up a system so that it is not necessary to use any extra software to import the save from PCSX2 to the PS2. If PCSX2 loads a saves to an SMB share, I want to be able to load the save directly from OPL from the SMB share.

OPL has a very specific file and folder structure that it has to follow indicated here. To be able to configure this, I need to be able to have all my saves/memorycards in the VMC folder and a solution to the file incompatibility needs to be found.

[uyjulian]

They are exactly the same format, despite the differing size. You can use third party tools to import/export saves from the memory card file.

[Nerdy314159265]

A third party tool wouldn't allow for what I'm suggesting. Micro-managing converting saves back and forth would be quite tiresome especially if you switch back and forth frequently.

If they are the same format, then does that simplify making them more compatible with each other? At the moment, neither software recognizes the other's memory card as valid.

[uyjulian]

Just add or remove padding depending on which program you want to use it in

[Nerdy314159265]

I'm not sure where I could safely remove padding from and like I said previously, that won't allow for the seamless transition that I'm aiming at. Being able to save a game on PS2 then pick it up on PCSX2 without extra work between the two.

I'm fine with setting up systems to get to there but rigging something to move around, copy and cut chunks out of the file doesn't seem like the best option especially from a stability standpoint. It'd be much more convenient for people with a similar desire to have the two pieces of software become compatible with each other.

[uyjulian]

add it at the end

[KrossX]

What exactly is the difference between VMC and PCSX2 memory card files? If I remember correctly, a PCSX2 file is just a raw dump. If so, maybe OPL should do a simple raw dump as well?

[uyjulian]

Yes, it is a simple raw dump.

[RedPanda4552]

I had a whole wall of text prepared but I will start by asking a simple question first.

What kind of size difference are we talking? Is it, perhaps an exact 256 KB?

[KrossX]

I've read something about missing the checksum section. If it's a per-block thing, it wouldn't be solved by just adding bytes to the end. I'm also too lazy to read their source code.

[RedPanda4552]

That's what I was thinking. If it's an exact 256 KB difference, that means they're discarding ECC bytes. Harmless, since you don't really need ECC unless you're using shitty NAND flash (aka every 2000's era console with memcards), and can just spoof it on the fly if you really need it to complete something.

[KrossX]

They could've just ignore them but still save them and have a simple raw dump while at it. It ain't much of a space saving after all, it makes things simple and adding support would've been trivial. Oh well...

[uyjulian]

If it's so trivial, you can add it yourself! Feel free to send a Pull Request: https://github.com/ifcaro/Open-PS2-Loader

Adding/subtracting padding works fine too and is the simple solution. Import/export on the command line with MyMC is also simple.

[KrossX]

If OPL's format was the same "adding support would've been trivial" indeed, just an extra line around here. We do abuse the file a bit though for the auto-eject function, but I suspect this wouldn't be a problem as the user would want to load from the memory card to begin with and avoid savestates in this case.

[uyjulian]

Ah, yeah, forgot about the file extension.

[Nerdy314159265]

I'm bringing some comments over from the OPL's side.

BatRastard

What's stopping us from adding ECC data to our VMC format? This is something I never understood, but it's exactly the reason why applications such as MCR and MyMC exist in the first place. Every one of these developers at one point got frustrated enough at the incompatibility between OPL and PCSX2 VMC files that they took it upon themselves to become a middle man ........ while muttering through clenched teeth that they're should be any middle men! LOL It's highly inefficient ... ;-)

sp193

It's the same reason. Neither do we need that data. On the other hand, PCSX2 emulates the hardware, so it has to emulate the data that the real hardware would store too. To understand how adding this data will make things very awkward for VMC, you need to do some math. The page with ECC data is 528 bytes, while the sector size of most devices out there is 512. You can only read data in units of 512, at addresses in multiples of 512. Hence reading and writing in multiples of 528 bytes to a device that can only do the same in multiples of 512 bytes, is simply an unnecessary challenge.

Nerdy314159265

Sounds like it might be easier to add a bit of code on PCSX2's side that automatically adds the ECC data when loading an OPL VMC file to memory and discards it when saving.

BatRastard

Actually, after SP193 explained everything, it would be easier for PCSX2's developers to add a tickbox to its memory card manager that enables/disables saving the ECC data entirely.

Since OPL is running on a far more limited device maybe one of the solutions mentioned here would work.

If the ECC incompatibility is taken care of then that would give us compatibility between the formats which would probably be nice for a lot of the people that have to use MyMC at the moment. The only thing left would be my feature request for swapping between cards but that isn't as important as the base compatibility.

[KrossX]

Well, that actually makes sense. I didn't even consider that.

I think it would be easier to convert on load and convert on save, adding empty bytes on load and ignoring them on save. Can you upload some full OPL memory cards?

[Nerdy314159265]

Full as in complete or full as in filled with saves?

[KrossX]

Filled with saves.

[Nerdy314159265]

Don't have time to test these but here's link ...

http://www.filedropper.com/oplvmcarchive

The README should explain everything. The VMC with the most saves is the RPGS - it's about half full. Any issues downloading this, let me know below and I'll see what I can do.

From a BatRastard in the OPL issue.

[KrossX]

I've implemented something basic, it converts the file on load and should convert on save replacing the old file. But, it can take quite some time to show up on the BIOS and there are bad sectors on the tested card. I would assume the checksum code is actually right cuz empty byte didn't work. So I guess error correction on load will be necessary. (edit, but there's no ecc data to do any correction with!)

RPGS.BIN + mymc converted

When viewing this files with a hex editor, some pages are quite different. Some that are full of 0xFF end with 0x7F for example. The mymc converted load quite fine and quickly.

Converted with mymc

Loaded by pcsx2 as .bin

EDIT: Can someone try that RPGS.BIN file and see if the PS2 reports damaged sectors?

[Nerdy314159265]

Sorry for the wait.

I tested RPGS.BIN on my PS2 from the link you posted. I first tried copying the memory card to a spare by mounting it to a VMC on uLaunchELF and copy/pasting. This got stuck on a single file for long enough that I reset my PS2 to escape. This left a corrupted file that I couldn't delete either through the default MC manager or uLaunchELF. I ended up having to find MC Annihilator (I had never used it before) to reformat my MC.

Then using the Restore MC image tool within MC Annihilator I restored the image to my MC. From that I can confirm that I, too, got a corrupted data block.

EDIT: Idk if it's actually relevant but I also tried using RPGS_new.ps2 by renaming it to a .bin and using MC Annihilator. My PS2 got mad and decided it was a new card so I let it reformat it.

[KrossX]

Things seem to be quite weird then, I'll assume that's normal. =P Please check the pull request as there will be automatic builds to test this.

[lightningterror]

Closing as https://github.com/PCSX2/pcsx2/pull/2995 added support.