[WiiU] PicoDrive Core crashes unless picodrive_drc is disabled in the PicoDrive.opt

[Ploggy]

Hi there, first off thank you for getting the PicoDrive working for WiiU and finally getting 32X on the WiiU :)

WiiU needs to have the Core option picodrive_drc = "enabled" set to "disabled" in the PicoDrive.opt for the Core to load a Rom but by default it's set to enabled. Easy enough to edit the PicoDrive.opt file manually but for a user who doesn't know they would just assume the Core is broken.

Performance results for Core on WiiU:

Megadrive Roms work fine from what Roms I tested 32X games run but sadly they are a little slow. Mega CD Roms don't seem to work, they try to boot to Bios but show a pixelated Screen..

https://pasteboard.co/JXunukQ.jpg

Thanks again for bringing the PicoDrive Core to the WiiU :)

[irixxxx]

Could you possibly make me a log of starting a 32X image with drc enabled? This can be enabled with the "log to file" option in then logging section.

[Ploggy]

Sure here's the log file:

retroarch.log

And here is the crash report the WiiU throws out when the Core crashes. (when loading a 32X Rom)

EDIT: Added a more detailed logfile :)

[irixxxx]

Not much in the logfile. It's however clear from the crash that it somehow crashes in the drc-compiled code. However, without seeing the compiled instructions around the crash I'm unable to do much about it. If I wrap up a small patch to output a hexdump of that in the core, would you be able to compile and run that?

[Ploggy]

It can't compile it, but I'll test anything you provide :)

[irixxxx]

Just a test. If it really produces a log files before crashing, please send it. retroarch_wii.dol.gz

[Ploggy]

That's a Wii app? (dol) we need a WiiU (rpx)

[Ploggy]

I setup vWii again just to test that dol.. It crashed once I tried to load a 32X game This is the log I got, but it shows nothing interesting and seems to end abruptly? retroarch.log

[inactive123]

Honestly it might be best just to wait for @irixxxx to give you a WiiU build. I don't think adding even more variables like vWii support right now is what we need as it adds even more variables and unknowns. We could look into vWii support or whatever once things actually work reliably well on a real Wii, right now the focus should be on real Wii and WiiU.

[Ploggy]

He posted the dol file tho? I never mentioned vWii until it was uploaded.. :P I just tested it.

[irixxxx]

well, it doesn't matter... both wiii and wiiu crash with 32x, and I think there's a common reason. @twinaphex there seem to be log data lost in the crash. Is there a possibility to flush the log buffer?

[irixxxx]

Here's another image, this time for Wii U. Please be aware that while I can test the Wii images with dolphin, I can't do so with Wii U, so this may run, or may not run at all. Whatever, I found an offset bug in the EABI support which may well lead to a crash when returning from drc-compiled code. If it starts at all, please check if this still crashes with 32X. retroarch_wiiu.rpx.gz

[Ploggy]

Your compiled Core does run :) But it still crashes in the same place when loading a 32X game. Logfile shows more info though, hopefully its helpful :) retroarch.log

[Cee123]

The core works beautifully on the Wii, with mega drive, master system and game gear. But crashes with 32X. Don't know if this will help. This is when trying to load Knuckles Chaotix.

Probably something to do with limited memory on the Wii.

[Ploggy]

@Cee123 exactly the same situation with WiiU, The crash report pic I posted is from Knuckles Chaotix too lol

[irixxxx]

I might have found a way to flush out the log data cached in the filesystem. Could you try this, please, and see if the log contains more data? In that case, send me the log and the crash data.

retroarch_wiiu.rpx.gz

[Ploggy]

Well, whatever you did.. there's alot more info in the logfile now, filesize has gone from 4kb to 13kb :) retroarch.log

[irixxxx]

Could please send me the crash log as well?

[Ploggy]

Sure..

[irixxxx]

ATM I don't know if it is possible to make a DRC on Wii U. It has memory protection and no apparent possibility to map memory for code execution, which is absolutely needed.

[Ploggy]

There is a WiiU specific Dev Discord Channel where you could maybe get some more specific info. It's called 4TU https://discord.com/invite/F2PKpEj They could probably tell you for definite if its possible or not? I'm sorry I cant be more help.

Or perhaps ask Aliaspider, (on the Retroarch Discord) he made the initial WiiU Retroarch port.

[Ploggy]

@irixxxx that logfile fix you made, can that be used across all cores or is it something only for Picodrive? I ask because a few Cores on WiiU have weird crashes that show nothing relevant in the logfile and that fix may help debug those Cores too?

[irixxxx]

I'm now doing not only fflush on the log, but also an fsync for the underlying file. That should only be necessary if the low level file I/O is part of the executed program, which is normally only the case if there is no operating system. So, I wouldn't recommend this for general use. It also makes logging abysmally slow, since every single bit of output produced in the log will lead to some sectors written to the underlying media.

Whatever, the basic change to picodrive was to redirect printf to the libretro logging callback. The fsync change is in Retroarch. If this can be somehow integrated there, every emulator using the libretro logging can inherit this by just building it with a newer Retroarch version.

[Ploggy]

It would be very handy if it were added to main RA. WiiU core crashes can be a pain to figure out :P Thanks for the info :)

[irixxxx]

I think DRC on Wii U might not happen for some time, if ever. I can't find any OSS software using a dynarec/JIT on Wii U which I can use to see how I can get access to JIT capable memory. I would appreciate a note if anyone has more info on how to do this. For now my only option is to disable the DRC on Wii U :-(

[Ploggy]

PPSSPP on WIIU by Aliaspider uses a JIT :) https://github.com/aliaspider/ppsspp Aliaispider is still on the Retroarch Discord from time to time.

[irixxxx]

Are you absolutely sure about that? I can't find any powerpc code emitter in that. Besides, It's only a handful of commits ahead of the ppsspp origin repo, and the commit name don't suggest anything related to athe dynarec.

Besides, I've tried to contact aliaspider via mail. Got no response :-(

[Ploggy]

Heh he pops in every now and again https://github.com/aliaspider/ppsspp/commit/c2731382ab439c0e74996ef49c0231d190d73592 I cant find the exact commit,

https://github.com/aliaspider/ppsspp/commit/b9bf37036a77fa38a5d166a21cd0b59ec6fc8461 perhaps this?

[irixxxx]

ah ok, I just got the zip that contained the master branch.

[Ploggy]

Sorry I should have said to check the branches :)

[irixxxx]

disabled the drc on wiiu for now, unless I understand why that ppsspp port works. I was told memory permissions can only have type W^X... however @aliaspider is using extended heap memory as RWX. I'm pretty astonished this is working.

[Ploggy]

I saw the commit you made :) hopefully Aliaspider can come to the rescue :P Thank you for taking the time to really look into this, WiiU doesn't get much love these days, so any devs wiling to add new stuff to it are appreciated.. thanks!

[crystalct]

@Ploggy Picodrive core works on WiiU (big endian machine)?

I reply to myself.... i think yes because i managed to compile it and get it to work for the PS3. ^_^

[irixxxx]

Yes. For 1.98 I added big endian support, as well as 32x DRC support for PowerPC. Alas, the latter isn't working on the WiiU, since I'm no WiiU programmer and have no clue how to prepare memory in a way that I can write code and execute that.

[crystalct]

@irixxxx there are 2 zlib folders inside project... is there a specific reason?

[irixxxx]

the one in libchdr isn't used. It's just there because it came with the submodule.

[crystalct]

using old gcc i'm having "error: redefinition of typedef" problems....

[irixxxx]

What's the message?

[crystalct]

In file included from zlib/zlib.h:34,
                 from pico/cd/libchdr/src/libchdr_chd.c:53:
zlib/zconf.h:265: error: redefinition of typedef 'Byte'
pico/cd/libchdr/deps/lzma-19.00/include/7zTypes.h:66: error: previous declaration of 'Byte' was here

i solved with this workaround:

#if !defined(__7Z_TYPES_H)
typedef unsigned char  Byte;  /* 8 bits */
#endif

inside first zconf.h and adding -DMACTYPES as CFLAG

[irixxxx]

hmm. wouldn't a #define __MACTYPES__ before the zlib.h include in libchdr_chd.c be sufficient?

[crystalct]

#if defined(__PS3__) || defined(__PSL1GHT__)
#define __MACTYPES__
#endif
#include "zlib.h"

Simpler... yes.. worked.

[irixxxx]

Would you please make a PR to rtissera/libchdr for this. I'm going to remove my libchdr repo in the near future since the important things are upstream anyway.

[crystalct]

Ok... i supposed irixxxx/libchdr-picodrive ....

[irixxxx]

Thanks anyway! I can't (and wont btw) do any checking for anything I don't have the hardware for. That would be rather pointless. I must rely on feedback for such things.

[crystalct]

Done.

[crystalct]

https://www.psx-place.com/threads/retroarch-psx-place-community-edition-beta-3.32272/page-15#post-292130 https://twitter.com/greatdiamondCT/status/1387709891788279808

[Ploggy]

Posting this since it's related.. :) A more detailed explanation on JIT for WiiU

https://github.com/libretro/RetroArch/issues/4852#issuecomment-828919857

The Wii U's a touch awkward to JIT on since not only does it enforce W^X (any mappings you have need to toggle between "can write" and "can execute"), it also will only allow executable code at all in a special memory area - so you can't mprotect random pages like you can on Linux - and that area has to be enabled in the first place in the app.xml. As far as I know it isn't enabled through a traditional browser exploit/CBHC into Mii Maker and HBL v1.x setup like most people have, though I'm pretty sure it is enabled for the HBL channel (v2.x). Haven't checked the Aroma beta but I would be very surprised if it didn't just grant all permissions. ppsspp gets around this somewhat awkwardly by writing code into where HBL ELFs are usually kept, which I guess works but very much isn't ideal, probably breaks RA's already shaky argc/argv handling, can only work for people running the rpx builds, and definitely won't work under Aroma (tbf I'm pretty sure RA doesn't work under aroma anyway). There isn't a super easy answer tbh. Can either implement something a bit hacky and will break soonish, or update the whole RA toolchain (or port it to wut) to get things working under Aroma and work from there. Modifying gnu-lightning doesn't seem too bad, though I'm having trouble understanding some bits (what's the intent of line 2050-2079?). It at least looks like it can be coaxed into working on a W^X setup, and from there it's just toggling between r-x and rw- where needed. Wonder if some of this code would be better placed into the libretro API or libretro-common, dealing with JIT'd code across platforms seems like a good place to stick some abstractions in place of the #ifdefs everyone seems to use now...

[irixxxx]

Aha... I'm apparently not the only one having been confused by what ppsspp does ;-)

[irixxxx]

Hey @crystalct, just for my record, could you check if 32x works with the drc enabled? And, since it's a problem on the wii, could you also do a check if chd images work for mega cd?

[crystalct]

CHD works (just picodrive_libretro.info must be updated to recognize chd ext). There is no drc menu in the options for PS3. Sega 32x works but slow.... and it's strange...for PS2 i know they works well..... is drc enabled for PS2?

[irixxxx]

Maybe the compiler isn't reporting a compatible architecture. Try adding ARCH=powerpc to the ps3 branch(es) in Makefile.libretro