search

MiSTer-devel / NeoGeo_MiSTer

NeoGeo for MiSTer
GNU General Public License v2.0
149 stars 77 forks source link

Add romsets.xml for some games sold on gog.com . #71

Closed vpelletier closed 4 years ago

vpelletier commented 4 years ago

As suggested in #70 .

Here is my attempt. altnames are probably not cannon. For example the "Turbo" in "Metal Slug 2 Turbo" is earlier than normal, but that makes it immediately visible in MiSTer menu rather than having to wait for the horizontal scroll.

There is one thing which is bugging me: game variants share the vast majority of their roms, and they have to be duplicated if one wants to have more than one variant. Is there a way to have multitple variants listed, with their specific file-to-memory mappings, when a single folder containing all variants is present ?

sorgelig commented 4 years ago

If i remember right, parts support the paths. You can have one common folder with base variant and additional folders where some parts will be taken from base rom.

ThreepwoodLeBrush commented 4 years ago

It seems the gog ROMs are the same as the ones from the Classic Complete Collection on humblebundle. I would like to help complete the romset.xml for this collection, but I would need a hint on how to determine the correct index values.

vpelletier commented 4 years ago

I would need a hint on how to determine the correct index values.

I have a very limited understanding of what the index values really mean. I understand they are related to the memory map (if not the NeoGeo's, at least te core's), that different rom types reside at addresses starting at a certain fixed offset, and that the next offset for multi-rom types is a function of the previous rom's size. Also, graphics have "independent" sequences for even and odd indexes.

Also, at least in GOG games there is a neogeo.dat file, which describes how all (?) games are to be loaded (not just the one installed), where file lines seem to contain:

Type to base offset:

The first 4 points come from xml_load_files in Main_MiSTer/support/neogeo/loader.cpp, the last one from non-packed roms already present in the pre-existing releases.xml.

To determine the next offset I check an existing working example and scale using its file size and the file size of the rom I'm working on. So far I did not encounter roms with gaps in their address ranges, but I'm obviously very new to all this.

mslug had the extra trick of requiring the 68k program low and high halves to be swapped.

About completing the file, I should get a 128MB ram module next week (waiting for the PCB to arrive, just received the components) assuming I do not botch my soldering, at which point I will purchase (part of) the rest of GOG's NeoGeo catalog and work on those (probably not most fighters though, but certainly all Metal Slugs).

charstring1 commented 4 years ago

@vpelletier fwiw, I recently bought the GOG Metal Slug 1 and was disappointed to find that it's the DotEMU release and I couldn't find the original ROM files anywhere in that package. Not sure if it was changed at some point or what. :-(

ThreepwoodLeBrush commented 4 years ago

@vpelletier Thanks a lot, that is very informative. In a few days I can give it a try to make a romsets.xml for the "NeoGeo Classic Complete Collection" from Humble Bundle. I should be able to reuse the entries you already made for the GoG set and same should apply vice versa.

Here the proms that need splitting and reversing are: mslug, twinspri, ironclad, samsho2, turfmast

@charstring1 I have the same issue with Metal Slug 2, there were no ROMs included on GoG. Blazing Star was nearly the same as the humble bundle collection, though, only the neogeo.zip contains more bios files on GoG.

vpelletier commented 4 years ago

@charstring1 I used the linux installers fetched very recently (purchased & downloaded about 4 weeks ago) from GoG. Here are the versions I used, with their md5sum:

8b1041877d3636c853de6c1eff448a1b  gog_fatal_fury_special_2.0.0.1.sh
a2e66cad8bdf8fdba21a3180c16a104a  gog_king_of_the_monsters_2.0.0.1.sh
f01e47a817ae43aa735066f75f3049df  gog_metal_slug_2.0.0.2.sh
9ac873de91b2374965df4475b32c5489  gog_metal_slug_2_2.0.0.2.sh
0280f171ba946f6dd6e3146aed67a765  gog_samurai_shodown_2_2.0.0.1.sh
88a977742c7c9132a9f51c6423d44c25  gog_twinkle_star_sprites_2.0.0.1.sh
5edd3c48898f59a7d118fe7b262adc29  ironclad_en_gog_1_15377.sh

I think the roms should also be somewhere inside the other versions, but may be tricky to locate and extract (non-standard archive format, obfuscation, ... I never had a look myself). Here are some relevant threads from GoG forums, although they are light on technical details on how to get the roms (the last thread is about the osx installer, which seems quite similar to the linux version):

Another idea would be to install cygwin to get a minimal linux environment within Windows and try to run the linux version of the installer from it. Or maybe the unix subsystem which AFAIK exists in win10, but I have never tried it (I have used cygwin, just not for GoG installers). And you can even try to go with a full-blown linux install in a VM (qemu, virtualbox, ...). Either way will require some familiarity with linux/unix and none is trivial, but that's the best I have so far.

vpelletier commented 4 years ago

I installed GoG Windows version of Metal Slug (using wine).

At a glance, resources/game seems to be the place to look in, so I ran MAME's romcmp on it and the linux game files. Here is what matches:

mslug_game_z80          201-m1.m1               IDENTICAL
mslug_adpcm  [1/2]      201-v1.v1               IDENTICAL
mslug_adpcm  [2/2]      201-v2.v2               IDENTICAL
mslug_game_m68k [1/2]   201-p1.p1    [2/2]      IDENTICAL
mslug_game_m68k [2/2]   201-p1.p1    [1/2]      IDENTICAL
mslug_spr_usage         201-s1.s1    [4/4]      61.538696%
mslug_game_sfix [1/4]   201-s1.s1    [1/4]      44.317627%
mslug_game_sfix [3/4]   201-s1.s1    [3/4]      40.802002%
mslug_game_sfix [2/4]   201-s1.s1    [2/4]      40.139771%
mslug_tiles  [odd 2/2]  201-c4.c4               32.994343%
mslug_tiles  [even 2/2] 201-c3.c3               31.464100%
mslug_tiles  [odd 1/2]  201-c2.c2               30.272888%
mslug_tiles  [even 1/2] 201-c1.c1               26.998091%

Which I understand as:

So it's a mixed bag: it's not directly usable, but it's not completely hopeless either. It should be possible to write a relatively simple tool to generate "normal" rom images from these files, and maybe even integrate that into MiSTer Main so only a gog-win-romwets.xml (or so) would be needed... Once the swapping rule is understood, of course. And hopefully it can then work for all similar titles.

On the BIOS side, there are a few matches which seem likely to be relevant - there are several more, but which seem far-fetched enough to be false positives:

mslug_bios_m68k            usa_2slt.bin            IDENTICAL
mslug_zoom_table            000-lo.lo               IDENTICAL
mslug_bios_sfix [2/4]      sfix.sfix    [2/4]      64.645386%
mslug_bios_sfix [3/4]      sfix.sfix    [3/4]      59.417725%
mslug_bios_sfix [1/4]      sfix.sfix    [1/4]      47.528076%
ThreepwoodLeBrush commented 4 years ago

I finished going through my NeoGeo Classic Complete Collection from humble bundle. I got most working and all the main sets you provided were working fine (I did not test alternative sets). I only added main sets so far, no additional sets for which enough files are present and added the link to the complete collection in the header. Here the results of a first attempt and the romsets.xml attached: gog+hb-ccc_romsets.xml.txt

Working: aof2: CRC = OK. Working blazstar: CRC = OK,OK,OK. Working fatfursp: CRC = OK,OK. Working irconclad: CRC = OK,OK. Working kotm: CRC = OK. Working lastblad: CRC = OK,OK,OK,OK,OK. Working mslug: CRC = OK,OK. Working mslug2: CRC = OK,OK,OK. Working pulstar: CRC = OK,OK,OK. Working rbff2: CRC = OK,OK,OK,OK,OK. Working samsho2: CRC = OK,OK. Working shocktr2: CRC = OK,OK,OK,OK,OK. Working shocktro: CRC = OK,OK,OK,OK,OK. Working turfmast: CRC = OK,OK. Working twinspri: CRC = OK,OK. Working

Not Working: bstars2: CRC = OK. graphics corrupted mslug3: Not working, has SMA mslugx: CRC = OK,OK,OK,OK,OK. Not Working (Copy Protection Warning)

vpelletier commented 4 years ago

bstars2: CRC = OK. graphics corrupted

        <file name="../bstars2/041-s1.s1" index="6"/>

I think this index should be "8". I purchased this from GoG and with this fix it works for me.

mslug3: Not working, has SMA mslugx: CRC = OK,OK,OK,OK,OK. Not Working (Copy Protection Warning)

I do not see anything wrong with your XML. I should receive my xsd ram PCBs this week, so I will hopefully be able to start testing soon.

ThreepwoodLeBrush commented 4 years ago

bstars2: CRC = OK. graphics corrupted

      <file name="../bstars2/041-s1.s1" index="6"/>

I think this index should be "8". I purchased this from GoG and with this fix it works for me.

Indeed, that fixes it. Only two games not working then (not counting the few encrypted games I did not list or test included in the collection!)

vpelletier commented 4 years ago

And I just found out the xml format is already documented in DEV_NOTES.md.

No direct informarion about why mslug3 woukd not work, but I learned that the 2 extra flags are about security chips, and one has cryptography mentionned on neogeo dev page. So maybe (part of the) the rom needs to be somehow decrypted before it can be used...

sorgelig commented 4 years ago

encrypted roms aren't supported, so it must be decrypted first.

vpelletier commented 4 years ago

Yeah, I saw that. The actual unknown is rather whether the rom is encrypted.

vpelletier commented 4 years ago

I bought mslug3 and checked the linux install content, and it contains files which strongly suggest it is indeed encrypted:

I'll have to check the windows install too.

About the neogeo core, is there a special reason not to support encryption ?

sorgelig commented 4 years ago

Encryptions don't grow on trees. So to support it, they need to be implemented. There are 3rd party decryptors if it's really required. For example there is a tool which decrypts and creates .neo container which is supported by MiSTer.

Btw, Metal Slug 3 is the one and only game. gog won't give anything other than all other packs.

ThreepwoodLeBrush commented 4 years ago

mslug3 is indeed encrypted. garou, kof2000, kof2002, samsh5sp, sengoku3 are encrypted, too.

mslugx is not encrypted and "just" acting up at this moment. I will give it another try.

ThreepwoodLeBrush commented 4 years ago

I tested mslugx a bit more and my purchased rom, that I concatenated following a guide from arcade-projects.com to work out of the box without index-values, does not work either. Maybe it never worked as I tested it when I had only 32MB RAM, assuming it would work with 128MB.

It is definitely not encrypted, but maybe how the NeoGeo-Core handles it triggers the copy protection.

vpelletier commented 4 years ago

Encryptions don't grow on trees. So to support it, they need to be implemented.

Good: it's not a priori a resource limitation reason (fpga elements or latency) then, "only" that more developer time needs to be poured in. This was the intent of my question.

I tested mslugx a bit more and my purchased rom, that I concatenated following a guide from arcade-projects.com to work out of the box without index-values, does not work either.

Some indexes have special handling in the loading code, reordering data on the fly (to optimise SDRAM burst-load, from comments in the code) which I do not think is triggered if a lower rom chunk steps into that offset area, so the rom must still be in a few separate chunks.

vpelletier commented 4 years ago

(probably not most fighters though, but certainly all Metal Slugs)

And GoG decided to put several NeoGeo games on sale right now, so I think I got all they have now. Still need the ram PCBs for any >32MB game test though.

sorgelig commented 4 years ago

Adding encryption chips into core gives absolutely nothing, as all games are available in decrypted form. Core still have some bugs - fixing of those will give more benefit.