Soft-patching zipped ROMs? Name of patch file?

[fooness]

Dear community,

the Lakka wiki states:

It is recommended to keep ROMs zipped, […]

… so I wondered, will RetroArch soft-patch ROMs, if they are still zipped?

Does the .ips/.ups patch file need to be named exactly like the .zip file (e.g., with underscores but spaces) or does the patch file need to be named like the ROM file which is stored inside .zip file (e.g. without spaces in case of no-intro).

It’s not always easy to see what the ROMs in the zip archives are named, though.

Thank you very much!

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/43237246-soft-patching-zipped-roms-name-of-patch-file?utm_campaign=plugin&utm_content=tracker%2F296058&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F296058&utm_medium=issues&utm_source=github).

[fooness]

I just found this romhacking FAQ which states:

If you are having auto-patching problems and you're using ZSNES AND you have a separate directory for your saved games, you need to put the patch in that directory, instead of the same one as your ROM. Fortunately, SNES9x finds the patch either way. If your ROM is in a zip file, your IPS filename should match the ROM name inside the zip file, and not the name of the zip file itself.

[ghost]

ips/ups/bps patch should be the name of the zip file, regardless if the rom filename inside,

it has to be name "filename.ips"

[fooness]

@retro-wertz Could you please verify this again?

I just tried that for almost an hour, in my case it only works if the .ips/.ups is called exactly like the .sfc stored in the .zip … I wish that wasn’t the case and you were right!

[fooness]

What do you dislike about soft-patching? It does not change the checksum which is needed for validating as proper no-intro rom, and I think hard-patching does change the checksum?

[ghost]

urgh....................

[hizzlekizzle]

I don't think that means we should kill it, though. For fullpath cores we could copy/hardpatch+cleanup-on-exit. It makes sense for them to not show up in playlists, since they're just patches, but users can copy/rename whichever patch they want to play from the main playlist entry. I do think adding the hardpatch checksums to the databases when possible is a good move either way.

[fooness]

@fr500: […] it doesn't work for playlists (it would always match the original game) […]

Could you please explain that? Because in my opinion, that’s not really a problem … for example, you want to have the original Seiken Densetsu 3 (Japan) in your playlist but you also want this game in English thanks to the English translation patch … wouldn’t you rename your patch (plus the game/rom file) to what it should represent—so, e.g. Seiken Densetsu 3 (USA) or Secret of Mana 2 (USA)? Then there’s no problem with applying the patch to the original game, as you wrote …

The only thing is, but that’s probably dependent on preserving the checksum, that the game-/romtitle in the playlist is still the original one … which is kind of irritating.

@hizzlekizzle: […] I do think adding the hardpatch checksums to the databases when possible is a good move either way.

For this to happen and flawlessly work, we probably first need a verified set of patches. As verified set of roms we’re probably talking of no-intro, but I didn’t find such a set/database for patches. Unfortunately, four out of five of the “original” .ips patches I downloaded from the authors websites or original posts at ~romhacking.net~ did not work with no-intro roms. I had to use .ups patches (credited to the same authors but I doubt they were created by them (they had a very lot younger creation date)) from ~superfamicom.org~ …

If I missed finding such a database for no-intro/verified patches, please let me know.

[i30817]

Removing softpathing altogether from RA would kill my interest in using RA. The first and foremost reason i started using it was the implicit (but false) idea that it would always guarantee softpatching in any supported core, which i got from the command line options and from the common sense idea that it is one of the few things that can be externalized to all cores (it's input). If you're planning to drop it altogether i may as well not bother.

What's really frustrating me is that is that i know that if even a single dev experienced in creating softpatching for emulators took the bull by its horns and implemented a efficient, non-ram based version of it - ie, stream like - in FUSE (file system in userspace) i wouldn't have to worry about emus supporting it ever again, just like i don't bother with compression after externalizing it to filesystems. I understand that 'decompressing' a patch into memory, and overlaying memory ranges and recognizing when it should come from file IO or the patch memory overlay is a hard problem to do efficiently without wasting a gazillion memory or slowing down input unacceptably, but that seems just the kind of thing that is a 'solved' problem somewhere in the OS (the second part).

[i30817]

On a related but separate matter, the idea that you can collect a comprehensive collection of all possible hacks CRCs in order not to have a escape clause to the way that not having a CRC disables a lot of features in RA is... quite unlikely, especially considering how some hacks are designed or amenable to be combined or how romhacking.net is not comprehensive, or how there are new ones and new versions constantly being made.

Making unknown CRCs fail gracefully, unlike now where they don't even appear on playlist after scanning should be the main thing, but that is just my opinion. I avoid the issue altogether using softpatching eh.

[i30817]

Also just another 2 details i thought about which you might find interesting:

if you try to 'identify' softpatching by identifying the CRC of the patch file instead of the rom, there is something to be aware on SNES files and some PSX hacks: they can be modified by tools in order to work without headers (SNES) or because of the psx error correction (PSX) - different final result also applies to hardpatching ofc.

Also in the case of wii and gamecube, dolphin has a scrubber for isos it compresses, which had two versions (replaced 'trash' by FFFF and 0000) so you might have some trouble incorporating crcs into a RA dolphin port.

Who knows what other gotchas are lurking about crcs. Anything that modifies the game files for a valid reason, i doubt i found all the cases just from what i use, but fwiiw, just the idea of combined patches makes it a nightmare imo.

[eadmaster]

what about just adding a menu command to manually load a patch file from the file system after the ROM is loaded?

There is no need to match the filename this way and patches can stay in a separate dir.

[ryz]

Hey guys, I'm also strongly in favor of softpatching, it's a great feature and keeps ROM integrity. I just think it could be handled a bit better.

What I would like (SNES/SFC example): I have Game, stored in game.7z which contains game.sfc and game.ips (Note the same filenames). game.7z can be scanned and finds and adds the original game.sfc. Starting Game would automatically extract game.sfc AND game.ips, softpatching in the process.

Alternatively it would be great to specify a softpatch manually via the playlist, in the lpl file e.g. "softpatch_path": "roms\\game.7z#game.ips" or similar. This would keep things nice and tidy.

Currently the best "workaround" seems to be just keeping the ips file unpacked next to the archive, then it will be recognized and softpatched by retroarch when unpacked (if in the same directory).

[Sanaki]

As I understand it, the way archives are handled by retroarch natively (zip and 7z specifically) means that from the core's perspective, the files are treated as though they're bare, sitting in the directory the archive is located in. Adding support for checking inside the archive for patches is a reasonable request, but I'd hardly call it a workaround to keep the patch file in the same directory the rom effectively resides in. It works the way it's supposed to.

[i30817]

This is all moot. People are still assuming that retroarch softpatching works externally to the cores.

It doesn't. If it did, it wouldn't be impossible to softpatch a N64 game ROM right now, which is between 8 and 256 mb. Or it wouldn't be impossible to softpatch roms for genesis plus dx.

The only controls that retroarch exposes for softpatching are these:

  -U, --ups=FILE        Specifies path for UPS patch that will be applied to content.
      --bps=FILE        Specifies path for BPS patch that will be applied to content.
      --ips=FILE        Specifies path for IPS patch that will be applied to content.
      --no-patch        Disables all forms of content patching.

and they're only modifications of existing core softpatching inputs. Retroarch doesn't have the code to feed cores a ROM that is softpatched 'transparently', either by streaming/FUSE filesystem, in memory or temporary file, particular cores do that when they notice that the ROM they're loading has a sibling file of the right name; or when a -U switch is provided and RA starts a ROM right away and forces that particular core to consider that patch files as a 'sibling', which wouldn't be the case from a zip.

BTW, currently RA doesn't even extract all the files on a zip when trying to find a rom inside one, which often causes it to use the wrong file (a bin instead of a cue for instance). The scanner in RA is really terrible at being 'automated', finding the right metadata, and has no user configuration either.

[ryz]

As I understand it, the way archives are handled by retroarch natively (zip and 7z specifically) means that from the core's perspective, the files are treated as though they're bare, sitting in the directory the archive is located in. Adding support for checking inside the archive for patches is a reasonable request, but I'd hardly call it a workaround to keep the patch file in the same directory the rom effectively resides in. It works the way it's supposed to.

Thanks for the comment. That's why I put "workaround" in quotes - I'm aware that this is the recommended way, I just think it could be more elegantly handled.

This is all moot. People are still assuming that retroarch softpatching works externally to the cores. It doesn't. If it did, it wouldn't be impossible to softpatch a N64 game ROM right now, which is between 8 and 256 mb. Or it wouldn't be impossible to softpatch roms for genesis plus dx.

I didn't assume that. I'm just asking for better handling and/or options for softpatching (such as manually being able to specify a softpatch in the playlist) to keep things tidy. Maybe it's just better archive scanning/extraction. I never assumed that any added functionality will magically make every patch in existence work on any system.

and they're only modifications of existing core softpatching inputs. Retroarch doesn't have the code to feed cores a ROM that is softpatched 'transparently', either by streaming/FUSE filesystem, in memory or temporary file, particular cores do that when they notice that the ROM they're loading has a sibling file of the right name; or when a -U switch is provided and RA starts a ROM right away and forces that particular core to consider that patch files as a 'sibling', which wouldn't be the case from a zip.

Understood.

BTW, currently RA doesn't even extract all the files on a zip when trying to find a rom inside one, which often causes it to use the wrong file (a bin instead of a cue for instance). The scanner in RA is really terrible at being 'automated', finding the right metadata, and has no user configuration either.

Improving the scanner would probably solve a few issues and provide an alternative solution to this issue. I don't think it should just extract every file blindly though - some archives can be pretty messy.

Funny that you mention bin/cue files - that was literally why I added my comment. Because that "problem" has been solved with CHD file support (also not externally and on a per-core basis) - a single, compressed file in a immediately playable state, which makes handling and storing disc-based games SO much simpler.

[i30817]

What i usually do for softpatched games (when the softpatch actually works with the core) is just duplicate the rom with hardlink in another directory (or with a different name if you don't want to be that fussy). This is ok, if and only if, the ROM is actually read-only, because a hardlink is the same file and modifications to one reflect in the 'other'. If you end up trying to hardpatch the file, you'll probably regret using hardlinks when the 'other' softpatch gets weirdly broken.

This is annoying for cd games, because those need to be hardpatched to work in RA, because the cores never allow softpatching of cd up (N64 up really), so the worst cases need to most wasted space.

As for decompressing the whole archive, i think the only reason it's not a per core option yet is because the code would need refactoring.