farmerbb / RED-Project

ROM Extraction Documentation Project
181 stars 8 forks source link

TurboGrafx-16/ PC Engine/ CoreGrafx Mini ROM dumping #112

Open kjetil-f opened 10 months ago

kjetil-f commented 10 months ago

Anyone got it to work yet?

Maybe it's possible to use a similar method as the Sega Genesis 2 Mini to achieve this?

According to this video it seems doable.

DrAzathoth commented 10 months ago

Supposedly, Hakchi2CE allows you to dump it like the other mini consoles it supports, and like the Genesis Mini, you’ll have to unpack M2’s archive. By coincidence, I ordered myself a CoreGrafx Mini, so I will be able to test this myself in a week or so.

kjetil-f commented 10 months ago

I had my suspicions. Please report back if you are able to test it. Can't seem to find much information on the matter.

DrAzathoth commented 10 months ago

The CoreGrafx mini arrived today, but I have no idea how to FTP the console. Unless someone can teach me how or the Hakchi2 CE update gets released, I have no way of dumping it at the moment.

Anyone can feel free to help me out here.

kjetil-f commented 10 months ago

Maybe the Sega Genesis 2 Mini method work. But maybe you don't need/should use the player 2 port in this case, but do as he says in the video.

DrAzathoth commented 10 months ago

I already tried that, but unfortunately it didn’t work.

kjetil-f commented 10 months ago

That's a bummer. Probably have to wait for a compatible Hakchi release then (if that ever comes out).

Levine91 commented 9 months ago

at this point I don't think a hakchi release for the TG-16 is ever coming. However, the pieces are clearly all there for someone to do it. Despite at least 3 different groups teasing it over the course of 3 years with nothing ever actually being released hack wise.

1.) This person dumped the exclusive games somehow. https://archive.org/details/pcemini

2.) There is more than one video on YouTube of someone playing on a modded TG-16/PC Engine mini https://www.youtube.com/watch?v=4kIvOiuui_E - 4 months ago, showing off various emulators https://www.youtube.com/watch?v=o8dzJr6KwA4 - PoC of swapping games out ONE MONTH after it released in 2020.

DrAzathoth commented 9 months ago

I would contact KMFDManiac via email or DM, but I can’t seem to find any contact info. I guess it is about time that I comment on one of his recent videos asking for a public release of the Hakchi fork for the purpose of dumping the ROMs from the Mini.

Wow. That was quite the run-on sentence.

Levine91 commented 9 months ago

I would contact KMFDManiac via email or DM, but I can’t seem to find any contact info. I guess it is about time that I comment on one of his recent videos asking for a public release of the Hakchi fork for the purpose of dumping the ROMs from the Mini.

He is still pretty active on Reddit. Try there. https://www.reddit.com/user/MDFMKanic/

stuckinstyx commented 5 months ago

SUCCESS!!

I was able to dump the ROMS from my PC Engine Mini using a method very similar to the Genesis 2 Mini. The key is to get it into FEL mode before running the Hakchi dumper. You can do this using a tool found in the Astro City Mini Batocera installer here:

https://github.com/acm-cfw/acm-cfw.github.io/releases

Scroll down the the ACM alpha 0.1, click assets and download acm_install20221203.zip In the data directory of that zip folder you'll find sunxi-fel-handshake.exe Run this and plug in the mini console into a USB port using a non-crippled USB A - Micro USB cable. Turn on the mini console and wait for the tool to tell you it is in FEL mode. This might take a few tries.

Then you can run the hakchi dumper tool linked in the Genesis Mini 2 guide https://github.com/farmerbb/RED-Project/discussions/56

It will immediately start working.

Once you have the dump, mount it using

guestmount -a /path/to/nand.bin -m /dev/sda9 --ro /path/to/mount

Note the different path from the Genesis Mini 2 guide.

The three important files (alldata.bin, alldata.psb.m, and m2engage) will be in the root folder of the mounted drive.

DO THIS AT YOUR OWN RISK.

Notes: 1) I couldn't get the sunxi-fel-handshake tool to work on my Win11 laptop. Might have been a power issue. It worked fine on a Win10 desktop and a Linux desktop 2) I couldn't get the guestmount tool to work on my Win11 laptop usig WSL. The tool wanted a valid kernel image in the right place. I couldn't be bothered to troubleshoot it, I just copied the nand file over to my Linux desktop and mounted it there. 3) The USB drivers come with the hakchi dumper tool, so you may need to install these before the FEL tool.

kjetil-f commented 5 months ago

Amazing. Well done! I will test it out when time allows. So basically run sunxi-fel-handshake.exe -> plug in and turn on your mini -> wait for FEL mode -> run hakchi.exe from the Genesis 2 tutorial -> save NAND -> mount the saved NAND and extract ROMs

DrAzathoth commented 5 months ago

Holy crap! I applaud you, @stuckinstyx!

All that we need now is a way to uncompress the CD games for the TG16 and the Genesis 2.

jd213 commented 5 months ago

Thanks a lot, can confirm that it works perfectly, and m2-unpacker was able to get all the roms/images! Too bad that Salamander wasn't hidden on the PCE Mini, still don't understand why it was an exclusive to the TG-16 Mini when it was never released as a TG-16 game back in the day, but Konami's gonna Konami I guess. Hopefully someone figures out how to convert the CD images as well.

The three important files (alldata.bin, alldata.psb.m, and m2engage) will be in the root folder of the mounted drive.

Glossed over this at first, took me a couple minutes to figure out that one needs to copy from /home/[user]/mnt/ instead of /home/[user]/mnt/usr/games/ like on the Genesis 2 Mini.

DrAzathoth commented 5 months ago

I cannot get my CG mini into FEL mode using the tool. Does this only work with the PCE mini?

stuckinstyx commented 5 months ago

@DrAzathoth , try a different cable. Some micro USB cables are power only.

Also, you don't need to use a second cable like the Genesis Mini 2, a single cable to the power port is fine.

Have you installed the drivers? They are included with the Hakchi dumper if you need them.

Also make sure not to use a USB hub. And try it on a desktop computer if you can. I never got it to work on my laptop. Perhaps the USB ports don't provide enough power.

jd213 commented 5 months ago

It took me a couple tries to get it to dump, try launching both of the tools and then plugging in the mini while its switch is turned on. It also worked fine on my laptop that doesn't have a VPN installed, but not on one that does (even with all of the VPN processes ended in Process Manager).

Levine91 commented 5 months ago

Incredible work! My question now is, how do we swap a game out? (Since I assume we can't just add more right now)

DrAzathoth commented 5 months ago

I’ve tried a wide variety of cables that I know work with other devices, but it still doesn’t work for me. I have been able to dump the Genesis Mini 1 and 2 so I am certain I have the drivers installed properly (PCE Mini is extremely similar to the Gen Mini 1’s hardware). The only feedback I receive is the message “No Allwinner devices in FEL mode detected”. The handshake tool terminates after 20 attempts.

My exact process is this:

kjetil-f commented 5 months ago

I managed to dump the nand from my CoreGrafx mini, but my Linux skills isn't exactly great. Been trying for hours now. Any concrete examples on how to mount this thing?

jd213 commented 5 months ago

I used sudo guestmount -o uid=500,gid=users,nounix -a /mnt/c/Users/[user]/Desktop/nand.bin -m /dev/sda9 ~/mnt Then individually extracted the files as follows: sudo cp /home/[user]/mnt/alldata.bin /mnt/c/Users/[user]/Desktop sudo cp /home/[user]/mnt/alldata.psb.m /mnt/c/Users/[user]/Desktop sudo cp /home/[user]/mnt/m2engage /mnt/c/Users/[user]/Desktop

kjetil-f commented 5 months ago

Thank you. Unfortunately I only get this error message: "libguestfs: error /usr/bin/supermin exited with error status 1"

ghoost82 commented 5 months ago

Like for others the guestmount command failed with a libguestfs error. But the following worked for me:

sudo losetup -Prf /path/to/nand.bin
sudo mount -o ro /dev/loop0p9 /path/to/mount
kjetil-f commented 5 months ago

Thanks. I have tried that as well, but gets "losetup: cannot find an unused loop device".

jd213 commented 5 months ago

Hmm, I know I had similar errors when trying to mount the Genesis 2 Mini nand, but I was able to eventually mount it by googling the errors and trying various things, so hope you're able to figure it out.

kjetil-f commented 5 months ago

Finally got it to work with my Core Grafx mini.

First I tried the same methods on another computer, but got the same errors. What worked for me in the end was:

cp /mnt/c/Users/[username]/Desktop/nand.bin . sudo losetup -Prf nand.bin sudo mount -o ro /dev/loop0p9 rootfs

Then I just used the regular Windows File Explorer to navigate to the "rootfs folder" and copied over the 3 required files back to Windows and used the m2-unpacker.

I couldn't find any bios files on here. Do any of you know if it's included with the system?

DrAzathoth commented 5 months ago

@kjetil-f, I still haven’t had luck dumping my CG mini so I can’t say this with certainty, but surely the BIOSes have to be included? Holding the select button when you start a CD game on the mini console emulates running the game on an older BIOS (for example, getting the Peke easter egg instead of Rondo). Perhaps the BIOSes are hidden in the .cd archives just like how they are with the Genesis Mini 2? Yes I know the BIOSes are loose on that console, but it has been confirmed they are inside the archives as well. (Source)

kjetil-f commented 5 months ago

Cool. How did he do that?

But I think you might be right. I took a quick peak with some of the games using a hex editor, and searched for "bios". Found a reference for it it within the games at least.

18 hits within alldata.bin.

RealRelativeEase commented 4 months ago

None of the commands to mount the nand file have worked for me so far. Are you using WSL2 or custom builds/actual Linux systems?

jd213 commented 4 months ago

I used WSL2, yeah. Did you run an update in it already? I believe the Genesis Mini 2 thread shows how, as well as some other tips.

RealRelativeEase commented 4 months ago

I used WSL2, yeah. Did you run an update in it already? I believe the Genesis Mini 2 thread shows how, as well as some other tips.

WSL2 is version 2.0.14.0, guestfs and guestmount are also the current versions as well. I usually get an error message related to fuse:

fuse: device not found, try 'modprobe fuse' first libguestfs: error: fuse_mount failed: /home/red/mnt, see error messages above

I tried to follow the instructions in the Genesis Mini 2 thread, but none of them worked for me.

RealRelativeEase commented 4 months ago

cp /mnt/c/Users/[username]/Desktop/nand.bin . sudo losetup -Prf nand.bin sudo mount -o ro /dev/loop0p9 rootfs

I'm also getting an error message on this one:

losetup: cannot find an unused loop device mount: /home/red/rootfs: unknown filesystem type.

Notes:

1. I couldn't get the sunxi-fel-handshake tool to work on my Win11 laptop.  Might have been a power issue.  It worked fine on a Win10 desktop and a Linux desktop

I was able to dump the NAND using my Win11 laptop using an external power supply for the laptop.

kjetil-f commented 4 months ago

Too bad. How about "sudo cp /mnt/c/Users/[username]/Desktop/nand.bin ." do this part work?

RealRelativeEase commented 4 months ago

Too bad. How about "sudo cp /mnt/c/Users/[username]/Desktop/nand.bin ." do this part work?

That one seems to work, same as the first line in the commands you posted earlier. But I'm still getting an error message on the other commands: losetup: cannot find an unused loop device

Just to make sure, you're using an official version of WSL with a Linux distribution from the store? My WSL seems to be missing modules, and from what I've read on Google so far, I would need to compile my own version to use modules.

kjetil-f commented 4 months ago

Got the same error on my first computer. For what it's worth, I used this guide to install WSL2: https://youtu.be/_fntjriRe48?si=BOxk1yiyHJrwBffr

Check if WSL 2 are set to default.

PS I used the version that's just called Ubuntu from the Windows store.

RealRelativeEase commented 4 months ago

Got the same error on my first computer. For what it's worth, I used this guide to install WSL2: https://youtu.be/_fntjriRe48?si=BOxk1yiyHJrwBffr

Check if WSL 2 are set to default.

PS I used the version that's just called Ubuntu from the Windows store.

Thanks! I followed the steps in the video, enabled VM functionality, installed the distribution simply named "Ubuntu" and set WSL 2 as my default. Also installed guestmount and libguestfs-tools in WSL 2.

I'm getting a different error message now: mount: rootfs: mount point does not exist.

Did you do any other preparations, other than running the commands you've posted?

RealRelativeEase commented 4 months ago

Okay, I figured it out. I created a folder on my desktop named "PCEMini", then I used sudo losetup -Prf /mnt/c/Users/[user]/Desktop/nand.bin sudo mount -o ro /dev/loop0p9 /mnt/c/Users/[user]/Desktop/PCEMini/ to mount the image to that folder. Then I used cd /mnt/c/Users/[user]/Desktop/PCEMini/ to enter the mount folder, and then sudo cp /mnt/c/Users/[user]/Desktop/PCEMini/alldata.bin /mnt/c/Users/[user]/Desktop sudo cp /mnt/c/Users/[user]/Desktop/PCEMini/alldata.psb.m /mnt/c/Users/[user]/Desktop sudo cp /mnt/c/Users/[user]/Desktop/PCEMini/m2engage /mnt/c/Users/[user]/Desktop to copy the files.

kjetil-f commented 4 months ago

Finally! Congratulations :-) I know how frustrating it is when things like this just doesn't work.

Rot-gut commented 4 months ago

Appreciate these guides! I was able to dump my nand.bin and convert everything. Is there steps on decrypting PCD and MCD files?

Also I checked the hash data on several PCE roms against no-intro's hash data and nothing appears to match. Were all of these roms altered in some way for the Turbografx-16 mini?

Levine91 commented 4 months ago

I don't believe they were all altered, atleast not in a significant way. You're better off comparing the 2 roms using a hex editor. See how many bytes differ.

I'm still also wondering if its possible to swap a game's rom with a different game or not.

jd213 commented 4 months ago

It's also possible to apply romhacks to some of the PCEmini roms, so the code for the ones that work should be the same (at least for the parts that get patched). But Splatterhouse Chrome didn't work for me after patching, so I wonder what was changed for Splatterhouse.

DrAzathoth commented 4 months ago

It's also possible to apply romhacks to some of the PCEmini roms, so the code for the ones that work should be the same (at least for the parts that get patched). But Splatterhouse Chrome didn't work for me after patching, so I wonder what was changed for Splatterhouse.

Are you sure you patched the right region ROM? According to the romhacking.net page, it needs the Japanese ROM, which is only on the PC Engine mini—not the TG16 mini or the CG mini.

Edit: The page also mentions “You have to add 512 octets (= 512 bytes / 4096 bits) at the begining of the ROM's file before patching.” Did you try that?

jd213 commented 4 months ago

Ah, thanks. I have the PCE Mini but I missed the part about the octets, so I used Advanced_SNES_ROM_Utility to add a SNES header (which are also 512 bytes), and the patched rom worked after that (at least in Ootake).

I forget if there were other patches that didn't work, but good to know that the roms are seemingly all unaltered.

DrAzathoth commented 2 months ago

I have good news and bad news.

Good News: I have found a tool that extracts both the pcd and mcd formats perfectly!

Bad News: I am not allowed to distribute it.

stuckinstyx commented 2 months ago

Are you allowed to give a hint of where to find it?

On Thu, 4 Apr 2024 at 21:58, DrAzathoth @.***> wrote:

I have good news and bad news.

Good News: I have found a tool that extracts both the pcd and mcd formats perfectly!

Bad News: I am not allowed to distribute it.

— Reply to this email directly, view it on GitHub https://github.com/farmerbb/RED-Project/issues/112#issuecomment-2038559079, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFDXE2H7QOFPWRLONZJPX73Y3XZLFAVCNFSM6AAAAAA3TZYK6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZYGU2TSMBXHE . You are receiving this because you were mentioned.Message ID: @.***>

DrAzathoth commented 2 months ago

I traced it to the No Intro forums, but I had to ask someone if I could have it. He told me not to distribute, so I’ll respect his wishes.

DrAzathoth commented 2 months ago

After looking at the tool itself, it appears to have been made by Cyanic from MMC. Possibly an overhauled version of this. Regardless, if you guys want this tool publicly released, then you’ll need to contact Cyanic, not the unrelated No-Intro forum member.

RealRelativeEase commented 2 months ago

Bad News: I am not allowed to distribute it.

Sigh...

What's the output format for the uncompressed disc images? And does the tool also support compression of disc images in other formats to potentially make them playable in M2's stock emulators?

DrAzathoth commented 2 months ago

Bad News: I am not allowed to distribute it.

Sigh...

What's the output format for the uncompressed disc images? And does the tool also support compression of disc images in other formats to potentially make them playable in M2's stock emulators?

It seems like a WIP development tool used for R&D for the PCE mini hack, thus it only has extraction and debug log support. I still haven’t been able to dump the nand for my CoreGrafx mini, so the results might only be true for the Genesis Mini 2.

What it does is extract the mcd archives into an iso/cue format with uncompressed .wav files for the audio tracks. Optionally, you can choose to extract the BIOS packed with it in the archive. Curiously, the BIOSes have different checksums than the ones easily found loose. After compressing the iso/cues into a chd files, some games were recognized by RetroAchievements, so most, if not all of them, are probably clean dumps.

The only problems I encountered were both the US and Japanese versions of Ecco: The Tides of Time erroring before it extracted the last audio track. Also, the iso for the European version of Robo Aleste was unable to the mounted by me, but ran fine in an emulator.

RealRelativeEase commented 2 months ago

After looking at the tool itself, it appears to have been made by Cyanic from MMC. Possibly an overhauled version of this. Regardless, if you guys want this tool publicly released, then you’ll need to contact Cyanic, not the unrelated No-Intro forum member.

Does anyone have their contact info? I've found a GBATemp profile which had the same profile picture as their Nuget profile, but they haven't logged in in about four years. I'm not sure if they're maintaining an online presence at this point, so we might be out of luck.

mjBrickman commented 2 months ago

I believe Cyanic's GitHub is https://github.com/GMMan. I found this through using a reverse image search on their profile picture and checking to see if any of their repositories correlate to the mini modding community, which some do. They appear to be still actively using their GitHub account as well.