Idea: Genesis Everdrive <-> MiSTer save conversion

[birdybro]

https://github.com/superg/srmtools - Source code for an existing converter in c++

Source code from there. Would be nice for us in the community to link people to your site when they ask about this. :)

[euan-forrester]

Thanks for this!

It's definitely nice to have source code, and it looks really simple (just packing/unpacking uint8s as uint16s).

I'm not sure how it fits into the site right now: so far it's been about making available saves found around the Internet by putting them into raw form, which is then loadable by many emulators and/or writable to original hardware.

I'm wary of an explosion in formats if it starts including formats for every specific emulator/flash cart/mister core without having a plan to manage the complexity (which I don't have at the moment!) and make a nice UI. Already the nav is pretty goofy looking. I've had some ideas about using a sample save to automatically add/remove a header (which I'm guessing is a common difference between these sorts of formats), but that wouldn't cover this case.

I've been maintaining a list of emulators which are incompatible with each other in the readme (https://github.com/euan-forrester/save-file-converter#emulators-with-incompatible-save-formats), to serve as food for thought and examples when it comes time for this. I'll add this one too.

So it's an interesting usecase for sure. I'm just not sure what to do about it right now.

[euan-forrester]

@birdybro I've received other requests for this, so I've put together a preliminary page: https://savefileconverter.com/#/mister and it does conversions just for the Genesis.

It's not linked from the main nav because it's just in a preliminary stage and not ready for prime time yet.

Can I get your help improving it?

I don't have a MiSTer: can you tell me if any conversion is required for saves the following platforms:

• NES
• SNES
• N64
• GameCube
• Game Boy
• Game Boy Color
• Game Boy Advance
• Game Gear
• Sega Master System
• Sega Saturn
• Dreamcast
• Turbografx-16
• PlayStation 1
• PlayStation 2

And, what is the correct file extension for the MiSTer for saves for each of those platforms? Are they all .sav, or are some different?

These are the main platforms I'm interested in for the site right now, so I'd like to get them all filled in correctly. If there's others that require conversion, then we can talk about adding them for sure.

Thanks again for making an issue here to let me know about this need, and especially for linking to the code! That made my job a lot easier :).

[euan-forrester]

Also, I've noticed that Genesis games that use EEPROM to save do not need this conversion. However, what about the Genesis games that use FRAM to save? Is there someone who can test this for me on a MiSTer?

Edit: This appears to just be Sonic 3. In an emulator, it requires the same byte expansion thing as SRAM saves, but I'm not sure about a MiSTer.

[birdybro]

Sorry about the delayed response!

EEPROM's are currently not implemented in the Genesis MiSTer core (doing EEPROM in FPGA takes a bit of time to do apparently and it's not been a major priority). There are SRAM hacks available for the relevant few EEPROM Genesis games so they should be compatible with SRAM conversion if that is a concern.

Regarding FRAM, I don't believe Sonic 3 is treated any differently in the Genesis core, usually for that it would need a special quirk defined with the game. I believe all of the quirks are just to allow the games to boot. There is only one SRAM quirk I am aware of that has to do with how to treat the SRAM data itself (which also allows the game to run correctly), which is just for one Sonic 1 hack that is using an incorrect SRAM initialization (0x00 instead of 0xFF which was sega's unusual spec).

https://github.com/MiSTer-devel/Genesis_MiSTer/blob/e0b5e2ddc4d41d0a06661a190a8fc35ebb203742/Genesis.sv#L1003

and

https://github.com/MiSTer-devel/Genesis_MiSTer/blob/e0b5e2ddc4d41d0a06661a190a8fc35ebb203742/rtl/system.sv#L922-L924

Other than that, Sonic 3 just seems to work, so I think the core must be similar to original hardware on this front where it doesn't worry about the unused sectors of the FeRAM in Sonic 3. This system doesn't use cartridges if that helps your understanding. :)

For the list from above, most of the systems are on the platform and the others will not be capable of being emulated on the platform due to hardware limitations, so that keeps it easy!:

NES - https://github.com/MiSTer-devel/NES_MiSTer SNES - https://github.com/MiSTer-devel/SNES_MiSTer N64 (won't ever come to the platform, beyond what the hardware can do) GameCube(won't ever come to the platform, far beyond what the hardware can do) Game Boy - https://github.com/MiSTer-devel/Gameboy_MiSTer Game Boy Color - https://github.com/MiSTer-devel/Gameboy_MiSTer Game Boy Advance - https://github.com/MiSTer-devel/GBA_MiSTer Game Gear - https://github.com/MiSTer-devel/SMS_MiSTer Sega Master System - https://github.com/MiSTer-devel/SMS_MiSTer Sega Saturn (Work in Progress, coming soon, wishing for Sergey's safety in Ukraine...) Dreamcast(won't ever come to the platform, beyond what the hardware can do) Turbografx-16 - https://github.com/MiSTer-devel/TurboGrafx16_MiSTer PlayStation 1 - https://github.com/MiSTer-devel/PSX_MiSTer (almost complete, already being used by players) PlayStation 2 (won't ever come to the platform, far beyond what the hardware can do)

That should cover it for the list. Full list of current cores here --> https://birdybro.github.io/MkDocs_MiSTer/cores/console/

The only ones I see missing from the list are MegaCD (not sure what would need to be converted), WonderSwan, Atari Lynx, and apparently a few fringe homebrew for Atari 2600 and 7800 have save functionality but that's just hiscore weirdness and I doubt any conversion would ever be needed for that, and probably not much demand :P

I will provide you files of saves when I get home from work tonight later on, sorry about the delays again!

As for incompatibilities off the top of my head from other cores and famous emulators, I believe there were only 2 games that had weird sram copy protection checks for some reason that made it so the sram size was incorrect on save, but that was fixed a couple years ago.

[euan-forrester]

No worries about the delay!

Thanks for the info about Genesis EEPROM and FRAM saves -- that's super helpful! I'm definitely a bit lost in the MiSTer world, so I really appreciate this help.

For Genesis specifically, for SRAM saves I have someone testing for me and they found that apparently the saves have to be padded out with 0xFF to 64kB and won't be loaded otherwise. I'm curious because the code you linked above for converting Genesis saves doesn't seem to do that. Any insight there? I updated this tool to do that padding (TBD is to undo the padding for converting from MiSTer, once it's verified that the padding works).

And thank you for the info about the various cores. I'm glad there's lots of overlap with my list and the actual list :). And good that there's not many on the actual list beyond what I was thinking. Edit: I updated the list in the tool

Do you have any info about what conversion, if any, is required for transferring saves to each of those cores? What file extension does each one use? Are they all .sav?

Edit: Sorry I missed that you said you were going to provide files this evening!

[birdybro]

Yes 0xFF is based upon Sega's standard. Genesis-Plus-GX uses it and a few other Genesis emulators that I know of. There were a few emulators that don't that are older. The genesis MiSTer core should work fine with incorrectly padded saves though last I checked. I helped make the SRAM00_Quirk, which required switching to 0xFF padding by default for the whole core, and then adding the quirk to switch to 0x00 padding only for Sonic 1 Remastered (which only need it initialized to 0x00 to work funnily enough), and it didn't impact anybody's saves retroactively as far as I have been aware. :).

The conversion to MiSTer ought to be 0xFF padding by default as the target I think, just because that is how original hardware was supposed to be. The only exception known is Sonic 1 Remastered, a weird hack :P

Thanks!

[euan-forrester]

Ah I see -- it seems like there was a miscommunication which resulted in the wrong file being used and thus the incorrect conclusion that padding was necessary. Thanks for clarifying that!

And yeah, I'm happy enough to leave the padding in there for converting to mister, even if it's not required. I'll add the code to un-pad a mister file before converting it back to emulator format, because naively byte expanding all the padding would result in a strange file.

I really appreciate all this help -- this stuff is a bit confusing and definitely moreso when I can't test for myself so there's extra opportunity for mistakes and misunderstandings.

I'm looking forward to grabbing those other files and taking a look through them to compare with emulator files and see what the differences (if any) are! Thank you!

[birdybro]

Yeah, there were no games that failed to work with the incorrect padding other than Star Trek TNG that we found, but this was only when the save file didn't exist yet and you had the game try to look for a save. When you change the sram initialization to 0xFF, it fixes it. It seems like the Genesis SRAM stuff is not very picky at all on padding of existing saves.

[birdybro]

PSX = No different than the other standard emulators. It's just the Bleem/MCD format, and I think there is only one old format that was different.

https://github.com/MiSTer-devel/PSX_MiSTer/raw/main/memcard/empty.mcd - blank psx mcd so you can see default formatting, this is identical to how Memcardrex, mednafen, and pcsx-r treat it, so you shouldn't need anything for PSX.

SNES saves worked when I pulled them off a retrode and interchanged them with bsnes before, so I don't think there is any problem there.

saves.zip - here's just a big zip of all my saves on my MiSTer for you now so you can see.

[euan-forrester]

Oh nice, so you've even tested a bunch of genesis games with respect to padding -- that's awesome to hear!

Yeah, looks like the psx file is the same format as emulators, just with a .mcd extension rather than .mcr like I've seen elsewhere. I'll make the tool just pass the file through untouched but change the extension.

And I'll take a spin through that zip file -- thank you. I'll let you know if there's a platform that's missing or if I need more info about something. Thank you!

[birdybro]

Yeah MCD is the format Bleem!, memcardpro, and a couple other things use. No problem! :)

[euan-forrester]

Yeah one thing I need to do with this tool is to do a pass through the whole thing and make it more relaxed about extensions -- there's just too many cases where different extensions mean the same thing.

[Kitrinx]

the genesis "padding" is what logically happens if you read the 16 bit bus from an 8 bit ram chip. The chip itself is storing and returning 8 bit values, but the bus has 8 data bus lines (presumably) pulled up to 0xFF. I think this is where the original padding came from in save files. Maybe the emulator authors who did this were anticipating some theoretical use of the other half of the data bus that never happened. RAM prices being what they are, it's safe to say that games of the day weren't using sram twice as large to hold half empty space.

[birdybro]

It was in an official Sega technical bulletin that said cartridges manufactured should (in most cases) have their SRAM initialized to 0xff is all I meant. :)

There was only one game where not doing this caused a problem.

https://github.com/MiSTer-devel/Genesis_MiSTer/issues/170

[euan-forrester]

@Kitrinx heh yeah I figured it was 16 bit read of 8 bit data :). Gotta use all that 16 BIT POWER :).

@all man I was hoping for an easy win with NES saves, but I looked at a couple of games that I'm familiar with and see that their 8k saves on a real cart are expanded to 32k on the mister, but the remaining data is garbage rather than padding. The garbage is virtually indistinguishable from the real data (assuming that the real data is actually the first 8k of the file). Any thoughts here? I guess I'll just try padding them out to 32k.

From eyeballing a couple of SNES saves I'm familiar with, it looks like the mister ones are the same size as the ones from a cart so no padding or transformation needed there.

[birdybro]

Yeah, I've been able to use SNES saves no problem from my Retrode2 dumps, and ones from bsnes as well.

NES I've never had a problem using the saves. There are some 8kb saves in there, those are nes2.0 headered roms. It's possible most of my saves are iNES headered rom's being used. I know the ginga eiyuu densetsu was a rom i had to repair the header for (with kitrinx's help) since it was incorrect in the NRS nes 2.0 database, but it got all fixed. Once it was fixed that save is 8kb. So maybe NES 2.0 headered roms leave accurate saves and iNES does not...

[birdybro]

NES.zip

Yup I used NES 2.0 Headered roms and forced a save, and it created 8KB files this time.

[euan-forrester]

Hm that's interesting -- thanks for figuring that out!

Do you think I should pad all the NES saves out to 32k, since I don't know what type of ROM will be used? Does the core accept 8kB or 32kB saves regardless of which type of ROM is being used?

For now I have it padding the save, and you can check it out here if you want to test: https://savefileconverter.com/#/mister

I also added PS1 and TG16 support -- those were just plain raw files. I'm still making my way through the list, but that's where I got to today :).

[Kitrinx]

the core won't start mirroring the addresses until 32kb, so any file with size up to that is fine, however if it's smaller it will cause no harm, the remaining ram will just be uninitialized data which is generally harmless

[euan-forrester]

Gotcha -- thank you for the info! I'll leave it as padding out to 32k then.

I'll work through the remaining platforms in the next few days, but in the meantime I'd love any help you could give me with testing what's there: I don't have a MiSTer of my own to verify that this works.

You can find it at https://savefileconverter.com/#/mister and so far I've done NES, SNES, Genesis, TG16, and PS1

[euan-forrester]

Okay I looked at Gameboy and GBA today:

For Gameboy: I noticed that many saves were 8k, but there were also a lot of 32k files. Almost all 32k files were for Super Gameboy enhanced games, but not all SGB games had 32k files. Many of the 32k files had strange alternating padding of 0x00 then 0xFF and back again.

Also I saw that for Final Fantasy Adventure, the mister save was 1k but mine off my cart (via a Retron5) is 0.5k: Final Fantasy Adventure (U) [!].srm.zip

Can someone please test that file on their mister?

I'm just going to code the GB/GBC "conversion" to pass the file straight through in both directions for now -- please let me know if they should be padded differently or do something else different!

For GBA:

The Castlevania Aria of Sorrow save in that dir is virtually empty, but it's only 32k whereas mine of my cart (via a Retron 5) is 64k. But, in mine there's data more than halfway through the file Castlevania - Aria of Sorrow (USA) (1).srm.zip

Similar for Metroid Fusion, where my save (via a Retron 5) has data all the way to the end of its 64k whereas the mister one is 32k: Metroid Fusion (USA, Australia) (1).srm.zip

I certainly didn't expect to see files that were "too short" from the mister! Any thoughts here? Can you try them out too on a mister?

Again for now I've just made the tool pass the file through in each direction. As always, you can check out the latest version at https://savefileconverter.com/#/mister

[birdybro]

FF Adventure - That save shows up as just 512 bytes of zeros when I unzip it:

The MiSTer sav when I make a new one is 1024 bytes of 0xFF.

For GBA... I don't know if the emulator supports 64k/128k FRAM yet.

[Kitrinx]

Oh.. by the way, some systems save real time clock data at the end of the save data. They'll repeat it over and over. I think gb and gba do this.

[euan-forrester]

Re FF Adventure: Oh! That's strange! I just tried both downloading the one attached here, and the original .zip file I made and both unzipped into a file with data in it. Um, can you maybe try again? Very strange.

Re GBA: Castlevania is battery-backed SRAM (one of the few) and I'm not sure about Metroid other than there's no battery :).

Interesting info about the RTC -- I'll keep an eye out for that -- thanks!

[birdybro]

Now I see it :)

[euan-forrester]

Okay cool! Have you had a chance to test it out on the MiSTer? I hope it works, otherwise I guess we pad the files out to a minimum of 1k? I think that Final Fantasy Legend 1 also had a 512B save file if I remember right.

[euan-forrester]

Sorry for the radio silence -- we've been moving so I've been swamped in box land :). I've had a chance to look at some more systems:

Sega CD: The emulator makes a file called cart.brm that's 524B and has a footer at the end. I tried the Sonic CD save from the .zip file and it's 8kB and has a similar (but longer) footer. But it doesn't seem to work straight in the emulator after renaming (does that save actually contain gameplay data, like completed levels, etc?). Any thoughts here?

Sega Master System: I made the tool pad the file out to 32kB with 0x00, since that's how the example files looked

Sega 32X: We didn't mention this system earlier, but it seems worth adding. There's no example mister saves for it -- can someone please make one (preferably with actual game data after playing for a little while). Here's a list of 32X games that can save: https://segaretro.org/Battery_backup#32X

Lynx & Jaguar: The directories for these platforms are empty -- can someone please supply a save or two here?

Sega Game Gear: There weren't any example files provided, so I just made the tool copy the file straight over. Can someone please provide an example Game Gear file (again, with actual gameplay saved)? Here's a list of Game Gear games that can save: https://segaretro.org/Battery_backup#Game_Gear

[birdybro]

Sorry as well, been kinda distracted with getting our new documentation site rolled out for the MiSTer --> https://mister-devel.github.io/MkDocs_MiSTer/

Sega CD: I have no idea how the saves work in that one. Which software emulator did you try the save with? The Sega CD core was kind of a very difficult core to even make function on the MiSTer so it's possible it may need refining for things like saves. EDIT: https://segaretro.org/CD_BackUp_RAM_Cart#Maximum_capacity - since each game will create a new save, the saves could theoretically be as large as 512KB, which is hilarious.

Sega 32x: There is no 32x MiSTer core currently. ;) The Sega Saturn core is being worked on by srg320 who lives in the Donbas region in Ukraine unfortunately :( He's okay the last we heard though. If anyone would make the 32x core, it would likely be him. When the Saturn core is finished, it's possible that work could be done on a 32x core as that means a hardware model for an SH-2 processor would be finished.

Jaguar: This is a very early stage core, I'm not even sure if saves are supported yet. @Kitrinx could probably comment on that, as she just updated the core quite a bit to actually be functional.

Lynx: There were no commercial releases for Atari Lynx with save functionality. There are a tiny handful of homebrew games with it. Robert Peip never implemented the save functionality from what I can tell, as a result.

Sega Game Gear: Here ya go, I just started a save in Shining Force The Sword of Hajya and had it autosave. Shining Force - The Sword of Hajya (USA).zip This is also in the SMS core :P It's an SMS/SG1000/GG combo core.

[euan-forrester]

Nice -- I like that documentation. Very friendly!

Thanks for all the info about the 32x/saturn/jaguar/lynx cores. I've gone and added some notes to my file that contains the list of platforms so I won't forget!

Sega CD: I'm using the GenesisPlus emulator on my Mac -- it's the default for OpenEmu. Here's the file it made: cart-original.brm.zip I'm feeling at a loss here. Should we give up on the platform, or is there somewhere we can go for more info?

Game Gear: Thank you for the file! It loads fine in the GenesisPlus emulator. Curiously, the emulator makes a 26kB file initially, which is than padded out to 64kB after the first save, rather than the 32kB file from the mister. Both the mister and emulator files have data up until about the same offset though. Can you please give my save a try? Shining Force - The Sword of Hajya (USA).sav.zip

SMS: Have you had a chance to test the code I added for this one?

FF Adventure: Have you had a chance to test the save I attached above in the gameboy mister core?

GBA: So it sounds like the core here isn't complete, and these 64k saves (Metroid Fusion, Castlevania: Aria of Sorrow) just aren't supported at the moment?

[euan-forrester]

Oh, and WonderSwan: is there any direction you can give me here? It looks like there's lots of different and strange file sizes in the directory you gave me: 2560B, 8704B, 33280B, 131584B, 262656B, 524800B -- always 0x200 more than a power of 2? I know nothing about the WonderSwan

[birdybro]

Yeah there are apparently NVRAM, EEPROM, and SRAM save games for the WonderSwan. This is probably the same behavior as the GBA core, as the same developer of that core made the WonderSwan core as well, so I would assume that software emulators (if they are all the same size) are padding, while the core is not. (and the PSX core, and tons of work on other cores, and savestates, etc... he's an absolute machine)

[euan-forrester]

Okay, thanks -- do you have an example save of each size that contains actual gameplay saved that I can test with? There's a ton of files in that directory, and I'm not sure how to sort through them.

Also, I had a few questions above as well -- sorry for the multiple posts, which I think makes it harder to see what I've replied.

[euan-forrester]

Hey, would it be possible to link to this converter in the docs you are working on?

I took a look though, and maybe a short section in The Basics called something like Save Files, detailing where save files go and how to name them and a link to this converter for if someone wants to bring over their saves from somewhere else?

[birdybro]

Sorry was busy learning how to port a core to MiSTer for the first time this last week. I've just been not paying attention to much else LOL!

I like the idea I'll add it now! :)

EDIT: Done! https://mister-devel.github.io/MkDocs_MiSTer/basics/saves/

[euan-forrester]

Nice -- that looks great! Thank you!

I totally understand about being busy -- no worries! I have a few questions above and just let me know when you have a sec to look at them.

[birdybro]

Sega CD: I'm using the GenesisPlus emulator on my Mac -- it's the default for OpenEmu. Here's the file it made: cart-original.brm.zip I'm feeling at a loss here. Should we give up on the platform, or is there somewhere we can go for more info?

I'll just explain some behavior of the Sega CD core as I test it out, with what I'm seeing. I am using Shining Force CD for this test. If I boot SFCD with Internal + Backup Cart RAM it gives me an error about initializing it, and instantly the core creates the save file that is 520KB. From 0x1FC0-->0x1FFF there is a header for the backup RAM cartridge. I'm assuming that the core goes from 0x0000 to 0x1FB0 for the internal memory (Sega CD had memory built into it for saves) and then it expands to the largest backup RAM cart size from there, which is 0x2000 --> 0x81FFF

Here's the header from 0x1FC0-->0x1FFF:

5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 00 00 00 00 40 00 7D 00 7D 00 7D 00 7D 00 00 00 00 00 00 00 00 53 45 47 41 5F 43 44 5F 52 4F 4D 00 01 00 00 00 52 41 4D 5F 43 41 52 54 52 49 44 47 45 5F 5F 5F
___________    @ } } } }        SEGA_CD_ROM 
   RAM_CARTRIDGE___

Now when I switch the MiSTer MegaCD core to only use "Internal" backup battery RAM it creates a file that is 8KB large, and spans 0x0 --> 0x1FFF. That same header of "RAM Cartridge" is at the bottom, so it appears that the core is mistakingly putting a header in the wrong place maybe? Not sure what's happening there. When I do the same thing with the backup cart, I see that header gets probably repeated at 0x81FC0 --> 0x81FFF ? Not sure.

Files attached for reference. megacd.zip

So the 8KB is just the internal backup ram that the sega cd had, and the 512KB is the internal backup RAM + the external optional backup cartridge you could buy.

The original Sega CD had 8KB of internal memory:

https://segaretro.org/Sega_Mega-CD/Technical_specifications#Memory

Internal Backup RAM: 64 Kbit (8 KB)

And the backup cart had 1 Mbit (128KB):

https://segaretro.org/CD_BackUp_RAM_Cart

the only officially-released RAM cartridge for the Sega CD has a capacity of 1 Mbit (128 KB; 2045 blocks)

But the Sega CD system itself supports up to 512KB! Which is crazy back then :P

https://segaretro.org/CD_BackUp_RAM_Cart#Maximum_capacity

So the reason for the 512KB is that the MiSTer core is operating at the theoretical total limit, however this could probably break compatibility with games since it's not really "cart + internal" it's more like "high level emulated storage that wasn't achieved on real hardware" so I think I will probably change how it appears in the OSD or see what we can do about that on the core side.

Been a busy week again, decided I'd answer some of the Sega CD stuff. Sorry for late replies. I'll take a look at the other stuff and test and update when I can get back to you.

[birdybro]

Game Gear: Thank you for the file! It loads fine in the GenesisPlus emulator. Curiously, the emulator makes a 26kB file initially, which is than padded out to 64kB after the first save, rather than the 32kB file from the mister. Both the mister and emulator files have data up until about the same offset though. Can you please give my save a try? Shining Force - The Sword of Hajya (USA).sav.zip

It loaded just fine for me, so Game Gear seems fine. MiSTer doesn't have much of an easy way to pad out a save file procedurally after writing it, it has to create a save file when the game wants it internally in some kind of memory, then you open the on screen display and it knows to write it to storage. That's probably why it's got a pretty set padding. The SMS core (which plays GG games too) funnily looked at that save and pushed it out to 64 KB :)

Shining Force - The Sword of Hajya (USA).zip

This is the after-effect.

[birdybro]

SMS: Have you had a chance to test the code I added for this one?

I just did and it doesn't seem to change the save file at all, they have identical hashes going in and coming out.

MiSTer.zip

Converted from MiSTer to Emu-raw.zip

[birdybro]

Also I saw that for Final Fantasy Adventure, the mister save was 1k but mine off my cart (via a Retron5) is 0.5k: Final Fantasy Adventure (U) [!].srm.zip

Can someone please test that file on their mister?

It works fine!

[birdybro]

Similar for Metroid Fusion, where my save (via a Retron 5) has data all the way to the end of its 64k whereas the mister one is 32k:

Yup works great!

https://user-images.githubusercontent.com/16388068/160756938-a5882884-fdfe-473a-9910-f370b6e57d47.mp4

The Castlevania Aria of Sorrow save in that dir is virtually empty, but it's only 32k whereas mine of my cart (via a Retron 5) is 64k. But, in mine there's data more than halfway through the file

Works great too!

https://user-images.githubusercontent.com/16388068/160757289-a1921c10-b59f-48a8-a510-768b152e1684.mp4

GBA: So it sounds like the core here isn't complete, and these 64k saves (Metroid Fusion, Castlevania: Aria of Sorrow) just aren't supported at the moment?

So to answer that question, they work fine! :)

[euan-forrester]

Thank you for this, and sorry about the delay. It looks like GBA, GB, GG, SMS are all fine, then -- thank you for that! It seems like there's some possible strangeness with the sizes, but if everything works I'm happy to leave well enough alone.

I asked on the WonderSwan github page about saves there, and the extra 0x200 bytes are RTC data that's appended to the save. I've asked if it's possible for me to set that information, since it'd be nice if the user didn't have to set the clock after converting a save.

I remember @Kitrinx mentioning that GB and GBA also set RTC data, and so I have the same question there about what the format is and whether it's possible for me to set it when converting the file.

For Sega CD, that's very interesting. Thanks for digging in and finding out that info about how it's creating the file. I made an issue on that github repo as well but haven't heard back there.

So it sounds there like there's a footer in the file at the expected place for if the device was operating with the actual largest memory, but then also at the end of the file which is sized for the theoretical largest memory? I guess from my naive perspective that sounds like a bug.

It sounds like I should wait for you to look at this more on the core side, then?

One thing to mention is that this project has had some exposure on RetroRGB and My Life in Gaming recently, and MiSTer conversions are now the most popular feature on the site day after day, with between 6 and 11 people using it each day. I honestly didn't think there would be such demand for this feature.

[birdybro]

Here's an interesting one...

Neo Geo. Can you produce a NeoGeo save from an emulator and share it here? I think I might have found a weird issue. For some reason the MiSTer Neo Geo core's memory card files are 72KB which is a lot larger than the maximum size a memory card would occupy.

Nevermind, I forgot these save files also retain per-game UniBios settings. These will probably never be compatible both ways.

[euan-forrester]

Heh I didn't see the edit until just now, so here's a file I created: saveram.zip

It's 64kB long, but I can't attest to much about it. I don't have any experience with NeoGeo, but from googling I don't think it's possible to run MAME in AES (home console) mode using OpenEmu, so it's from Metal Slug but in MVS (arcade cabinet) mode. So who knows what it contains. I got to level 2 and then copied the file. In AES mode the game is supposed to save your progress after losing all your lives so you can restart at the last level you were on.

[birdybro]

64KB makes sense, as that would be the MVS backup RAM internally. MiSTer is expanding to 72KB I think so it can retain the UNIBIOS settings as well as the DIPS, game saves (which MVS doesn't have), etc...

[birdybro]

Heh I didn't see the edit until just now, so here's a file I created: saveram.zip

It's 64kB long, but I can't attest to much about it. I don't have any experience with NeoGeo, but from googling I don't think it's possible to run MAME in AES (home console) mode using OpenEmu, so it's from Metal Slug but in MVS (arcade cabinet) mode. So who knows what it contains. I got to level 2 and then copied the file. In AES mode the game is supposed to save your progress after losing all your lives so you can restart at the last level you were on.

Okay, looking at it and it appears that MiSTer initializes the Save RAM to 0xFF. It is 72KB vs the provided example being 64KB. At 0x10000 (meaning just after the first 64KB) that's where the BIOS settings appear to be stored. This save all I did was change UNIBIOS from Europe+Arcade to USA+Arcade, then it creates the save file for the game:

MiSTer Blank Save with BIOS settings saved only

Then I just set the game to easy, changed to 5 lives cuz I'm a wuss, and went into the first level and died pretty early on (totally on purpose) to game over, it asked me if I wanted to save and I said yes, it had to format the memory card, and here is that file:

MiSTer AES Save

There was no difference for the first 64KB, then a bunch of differences after that:

So the 8KB AES save starts at 0x10000, if you need the offset. The first 64KB is for MVS I'm assuming so I'll now switch to MVS. I then had to turn on the DIP Settings option and reset the game... I turned on Blood, and then set the code number to 1337 because. Then I changed the time to 3:22 because that's what it is right now... But it didn't let me for some reason (unibios maybe? i dunno). I hit exit in the menu, it tells me to turn off dip 1 so I do... and the system reboots, but the save doesn't get updated interestingly enough. It offered to load the game despite being in MVS and Arcade mode, so I did. I died, but chose not to save this time, opened the on screen display to see if it would save, and it did. Odd. Might be high score or bookkeeping figures saved I dunno. Here's that file:

AES save but loaded on MVS and soft dip settings, maybe not saved

Now in HxD I can see that the first 64KB are populated by lots of data. So the first 64KB must be MVS and the next 8KB are AES.

[euan-forrester]

@birdybro hey, I'm looking at Sega CD saves and had some questions about the mister. I looked through the saves you gave me earlier, and the ones for sega cd games had just one file and it was 8kB in size and was named after the ROM.

On other platforms, there's 2 files: one for the internal memory on the sega cd itself, and another for a ram cartridge. The former is always 8kB but the latter can be different sizes.

So I'm a little confused about the mister. What happens if you make both types of save for the same game? What are the files named, and how large are they? I found that Popful Mail is a good one to use because you can select where you want to save in the main menu, and make a save quickly just by going to the first area of the game then opening the menu. Would it be possible to please send me both types of save from that game?

I also found that you had a couple of saves for Shining Force CD (3R), and the files were 512kB in size but the 'footer' in the file was only after 8kB, making the file actually able to only store 8kB of data (the rest was padded with 0s). This seems very strange to me! Any thoughts here?

Thanks again for all your help! I've got a proof of concept working for moving sega cd saves between mister/emulator/megaeverdrivepro so I'm excited to get all the details figured out and get it onto the site!

[birdybro]

Thanks for all your work on this as always!

For the explanation as to why there are two size of files, this is due to a toggle in the on-screen display for the core:

https://github.com/MiSTer-devel/MegaCD_MiSTer/blob/7ff9ebc90bd97368ec751ee1322f79234bf2d876/MegaCD.sv#L329

When it's set to "Internal" then it's 8KB. When it's set to "Internal+Cart" then it's a 520KB size save file of 8KB internal combined with a 512KB external backup cartridge (because this is the theoretical limit of an original Sega CD in terms of addressing assigned, even though no Backup RAM cart released contemporaneously allowed for this). Example video of the toggle just to give you perspective:

https://user-images.githubusercontent.com/16388068/185029291-1d0fac5f-6bb5-45d4-8f8d-42e7a2f50fcc.mp4

Basically on MiSTer h0000 --> h1FFF is the 8KB internal RAM, the remaining is reserved space for the theoretical addressable limit of 512KB for the backup cartridge option, totally 520KB for a combined save that can see both.

To test your 8KB --> 520KB save theory, I started Popful Mail using the 8KB save. Then I switched the option in the OSD to use Internal+Cart (520KB resulting file size). I go to Continue and it says no save data exists. The game also didn't ask me to initialize so it must be seeing the first file. It doesn't recognize save data unless I do something and then save I guess -_-

The reasoning behind both the files being combined is because we use a reserved index number for our save system that is standardized across the cores and it only allows for 1 save per game in order to be minimal in resource utilization.

Here's the requested saves to compare:

Popful Mail (USA) (RE).zip

I don't remember what happened with all the shining force stuff, so my testing methodology could have been incorrect and I've learned a lot more since then, so we'll just ignore that :P

EDIT: oh and the differences... At the footer of a MegaCD 8KB internal RAM section there is the following:

At the end of the 512KB section (end of file as well) is the following:

I can only guess all the values in these files are populated by the BIOS + Game, ekeeke of gpgx also thought the same in the issue you raised there when discussing how MiSTer does it, and I think he's right.

[euan-forrester]

Gotcha! That makes total sense now, and it's a nice solution to put them both into the same file.

Yeah, to resize the saves so they can transfer from emulators <-> megaeverdrivepro I had to fiddle with the bits he mentioned, and it worked perfectly.

I'm a little bummed I haven't figured out how to parse the actual files themselves yet, to pull them apart if there's multiple saves in them, but at least I can get them to be recognized by different devices.

Thanks for all that info about the mister -- I'll be able to add support for creating and pulling apart those files no problem.

Thank you for such a quick and super helpful answer!

[euan-forrester]

This is now up and running: https://savefileconverter.com/#/mister

Thank you again! Next up is making the UI for converting sega cd saves for the 2 genesis flash carts

[euan-forrester]

@birdybro I saw that the Saturn core now supports saving! I know nothing about Saturn save files, but I'd love to add support for it to the tool.

Any chance you can provide me with a few save files, and ideally give me any info you know about differences (if any) between them and save files made by emulators?

BTW, I'm not sure if you saw, but a while back I added full support for parsing and creating Sega CD save files. I'm guessing that the Saturn is going to be somewhat similar: being something of a file system containing multiple potential saves.