UWP/Xbox Series S/X - Some games that work on standalone and pc retroarch current mame core do not work on series x/s retroarch.

[joepogo]

When using the mame-current core in retroarch, there are a small number of games that will not load and some crash retroarch. The roms have been verified to work in the current standalone mame and verified with the proper mame current dat so it is not the rom.

As stated these exact same roms were tested in the current standalone mame and load with no issue at all.

The games in question are:

Galaga Pole Position 1 and 2 Gyruss Donkey Kong Donkey Kong Jr. Moon Patrol Burgertime

Currently, the only way to get these to load at all is to use the 2015 mame core. These exact same games ALL crash even when trying to load them using the Mame 2016 core.

Can someone shed some light as to what is going on with this in retroarch mame? Most mame games on the current core work well within retroarch and have no issue at all but this is a really strange one!

[riggles1]

They do work, either something is wrong with your settings, or your romset. MAME Current (247) and romset 247 (merged and non merged).

Turn on the RA error log function to see what the issue is and post it here.

[joepogo]

Thanks riggles! :) Will do! I know it definitely isn't the romset as i use the exact same one on mame pc 247 and retroarch 247 (current core).

I am using/testing these on series/x but that shouldn't matter. When i get home in a little bit ill turn on logging and post the log file here as soon as i get it.

[joepogo]

OK, i attached the log file for this. Thanks for the patience riggles.

[joepogo]

For some reason i can't edit my comment or attach the file so i uploaded it to tempsend.

https://tempsend.com/gxmcs

[riggles1]

[libretro INFO] Starting game:D:\Arcade Games\galaga.zip [libretro INFO] Game name: galaga, Game description: Galaga (Namco rev. B) [libretro ERROR] System not found: Arcade Games [libretro WARN] Driver Arcade Games not found -1 [libretro INFO] Creating frontend for game: galaga [libretro INFO] Softlists: 1 [libretro WARN] Invalid video value opengl; reverting to software [libretro INFO] SOURCE FILE: ../../../../../src/mame/namco/galaga.cpp

Show me your MAME options screen in the RA quick menu. https://imgur.com/mlnOFr8

(also enable mame.ini paths at the end)

Otherwise it's not finding the system, there's an issue with your directory paths. You'll want to make a mame.ini (enable read configuration for the core), should go in the root of your RA. It's having issues because your MAME romset isn't in root/roms/mame/ which is where mame is expecting it to be (since mame in RA is still like regular mame but with RA functionality, wrapped), regular mame uses a mame.ini.

Here's an example mame.ini, edit with the paths you want to use https://pastebin.com/raw/aFzuM8U6

[joepogo]

Thanks Riggles! When i get home i will try this out. Just started my shift so it might be a little bit. When you say root, do you mean in localstate/system/mame? In that area?

I am running all my games off of a usb stick because space on the xbox internal drive is limited to a certain amount. Most of the games run just fine without needing to do anything but i guess that doesnt mean they all will.

Is there a way to point the ini to the usb stick in D:?

[riggles1]

Oooh, yeah no idea how it works with xbox, I'm only knowledgeable with how it is in windows. Try ask people in the discord and you'll find someone with a similar setup to you. But with root I mean where the retroarch.exe itself is, but that's how it is in Windows.

It's something with the paths going on here, so ask in there and close this issue (and reopen this issue with a new name speficying xbox if it turns out to be some xbox specific bug)

[joepogo]

It's looking more and more like an XBox specific bug so i edited this issue report to reflect it.

I turned on mame ini paths and read configuration in the menu. I then edited the mame.ini file to point the ini to the D:drive where my roms are located on the USB Stick and those games still refuse to load for some reason.

ALl of the other games i run on mame retroarch xbox for some reason load just fine with no issues at all and mame finds the roms correctly. But when i try to load any of those 7 or 8 games it just crashes back to the dashboard.

I'll try a couple more things but it makes no sense really... :/

They DO happen to work just fine in the mame 2015 core, but also crash on the mame 2016 core.

[greenchili2]

I did some experimenting with some of these games and lookin at the log it always bombs out when initializing the discrete sound driver. I also noticed that the games listed by Joe Pogo are all games that used to use sound samples and now use the discrete sound drive. So more likely than not that is the issue.

Something must have changed in the code that the xbone does not like.

[joepogo]

Crusin usa, ibara, and gradius 4 can now also be added to this list as well.

Crusin usa won't load in any core I've tried at all.

[joepogo]

You can add ibara, espgaluda ii, iron horse, time crisis and winding heat to this list. It's incredibly frustrating to have so many random games crashing the RetroArch uwp mame current core. I'm not sure what the issue is as mame 2015 core runs these just fine but the core and emulation is old. What has changed in the mame current core for uwp that is now broken?

[barbudreadmon]

It appears to me that every game affected by that issue is using MAME's threading implementation in one way or another :

• threading is used for TTL logic (dkong, gyruss, ...)
• it's used for 3D rendering (crusin usa, ...)
• it's used in cv1k games (ibara, espgal2, ...)

source

So i think it is extremely likely that MAME's threading implementation is the culprit for those crashes on xbox, since threading often requires platform-dependent quirks.

[joepogo]

Thanks so much @barbudreadmon for this helpful reply.

Hopefully it is something we can work around as the series x has the grunt to run all these games with ease.

I guess this explains why these same games work just fine in FBANEO on xbox AND the mame 2015 core? It's incredibly frustrating to have RETROarch only to be able not play classics like galaga and donkey kong. :)

I am assuming that mame must have changed something in after the 2015 core since these games worked just fine until then. The only game thats never worked at all has been ridge racer.

The vast majority of mame current games run great and just fine with no issue.

Is there anything we can do to get these working on our series x consoles?

[barbudreadmon]

I guess this explains why these same games work just fine in FBANEO

They work with FBNeo because even if they didn't, we (i'm a FBNeo developper) would look into this since we care about cross-platform compatibility.

Yes, i'd suppose many things changed in mame over the past 8 years, it could be that they changed something in their threading code, or even that those games weren't using threading at all back in 2015.

[joepogo]

Thanks again for your responses as always @barbudreadmon . It is very appreciated!

I guess the best thing (and maybe the only thing) to do is make a request asking the mame devs to possibly look into it or hope someone here with know how sees this and is willing to make a possible fix then and look into it.

Luckily it is not a huge list of games, and the majority seem to be the discrete sound core games. I hope eventually someone can take a look at this.

I am hoping with all the changes in .255 maybe, just maybe this will work.

@sonninnos sorry to ping you, just wondering since you are currently updating mame on retroarch to .255, have you noticed by chance any changes in the threading code since .251?

[sonninnos]

have you noticed by chance any changes in the threading code since .251

No. I've only looked at what failed to compile and work on my machine, and none of that has been related to threading.

[barbudreadmon]

I guess the best thing (and maybe the only thing) to do is make a request asking the mame devs to possibly look into it

It's probably better to forget about that but good luck if you decide to try anyway.

[joepogo]

Heh, yea, i'm not sure how my report will be met but at least in the meantime i do have FBANEO as well which is just as good for those games. Thank you for all the great work you and others do @barbudreadmon

Not that I'm worried but I'm not sure if ill be met with hostility or friendly action if/when i ask. :P Either way, it's good to start getting a better idea of what the issue could be here.

This will or could all be moot if xbox can ever move off of UWP, which i think is the real issue here.

[barbudreadmon]

Not that I'm worried but I'm not sure if ill be met with hostility or friendly action if/when i ask. :P

I remember one of their main devs specifically explaining that emulators being available on modern consoles is detrimental and hurts the emulation scene, because it's competing with the game remasters available on those consoles, so i'd be surprised if you didn't encounter some hostility. The MAME team is not monolithic though, and some MAME devs are very open minded.

[TheElecPlay]

Running mame-current on Series X and I've rebuilt my set of MAME roms (mk4.zip, tekken3ja etc) using clrmamepro with the 0.251.dat. Games (along with their bios) will run on MAME 2010-MAME 2015. However, running these games on mame-current (0.251) will crash and boot me back to the Xbox dashboard. Please find attached my logs - perhaps whoever is working on the mame cores could possibly look into a workaround or fix for mame on RetroArch Xbox UWP retroarch2023_06_1417_41_36.log

Thanks

[grant2258]

@sonninnos if this is threaded issues on the xbox it can easily be resolved by fixing a define for the thread handling in osdsync.cpp or the threading can be disabled via a define. Right now it handles windows linux/bsd and mac not sure whats defined in the xbox build.

https://github.com/libretro/mame/blob/5e04369f1154f5458431fd9e01b7c70cf14ff932/src/osd/osdsync.cpp#L89-L93 and https://github.com/libretro/mame/blob/5e04369f1154f5458431fd9e01b7c70cf14ff932/src/osd/osdsync.cpp#L278-L281

is where you would put your define in if you choose to disable it threading and multi core cpu for xbox. Can rule that out as an issue.

[joepogo]

Oh wow! Thank you for digging into this @grant2258 !

Hopefully @sonninnos will be gracious enough to disable this for us and once the pull request is committed we can give it a test to make sure.

Running through dev mode we can update our mame current core easily once a commit has been pushed.

Once again, thank all of you guys who are even responding on this issue ticket. It's much appreciated.

[joepogo]

Hey @grant2258 if sonninnos can't get to adding the define is it possible to trouble you to get you to make the pull request?

I'm in way over my head with code and you seem to know the ins and outs very well.

I've got dev mode on series x so any commit made to the main mame branch can immediately be tested.

This crashing issue has been an issue since we started getting apps on series consoles a few years back.

Anyways, I thank you for your time and everyone else's as well.

[grant2258]

If you can compile here is a patch. Just copy the text and put it in a file the mame source directory where the makefile is then git apply filename your just removing the #if #endif in them two places i posted earlier. That will be enough for you to test if if its works.

diff --git a/src/osd/osdsync.cpp b/src/osd/osdsync.cpp
index bd9a82b0f9f..c81b914aa4e 100644
--- a/src/osd/osdsync.cpp
+++ b/src/osd/osdsync.cpp
@@ -88,14 +88,14 @@ static void spin_while_not(const volatile _AtomType * volatile atom, const _Main

 int osd_get_num_processors(bool heavy_mt)
 {
-#if defined(SDLMAME_EMSCRIPTEN)
-       // multithreading is not supported at this time
+
+       // test for xbox
        return 1;
-#else
+
        unsigned int threads = std::thread::hardware_concurrency();
        // max out at 4 for now since scaling above that seems to do poorly
        return heavy_mt ? threads : std::min(std::thread::hardware_concurrency(), 4U);
-#endif
+
 }

 //============================================================
@@ -276,10 +276,8 @@ osd_work_queue *osd_work_queue_alloc(int flags)
        if (osdworkqueuemaxthreads != nullptr && sscanf(osdworkqueuemaxthreads, "%d", &osdthreadnum) == 1 && threadnum > osdthreadnum)
                threadnum = osdthreadnum;

-#if defined(SDLMAME_EMSCRIPTEN)
-       // threads are not supported at all
+       // threads are not supported at all test for xbox
        threadnum = 0;
-#endif

        // clamp to the maximum
        queue->threads = std::min(threadnum, WORK_MAX_THREADS);

[joepogo]

Hey @grant2258 we tried compiling this change but it gives errors

Are we missing something?

[grant2258]

post the error message your getting it could be the compiler being picky about the second return no being reached. No problems compiing in gcc or clang so would really need to see the error to be sure.

[joepogo]

Thanks for continuing to help us troubleshoot and get through this hurdle @grant2258 .

He said there was so many errors the command prompt wouldn't let him see them all.

He said he is using GCC 13.1.0-6 for the compiler. Since we're on Xbox we're on uwp so I'm assuming he's building it for that.

If you compile this by chance please let me know if you have any issues. He tried compiling twice and both times it failed.

Anyways, thanks for continuing to look at this and hopefully it'll be sorted soon.

If we can get it to compile I can easily test it on Xbox to make sure it fixes the crashing issues.

[grant2258]

Its compiles fine in Linux with no issues with gcc 13.1.1. Not sure if mingw is used for the cores with its limitations as its doesnt implement the C++/WinRT api. Someone at libretro may be able to advise on that I would assume they are using msvc2019 or perhaps msvc 2017.

[joepogo]

Is it possible for you to compile the mame current core for uwp and post/message me the file in the meantime?

I'm very limited in what I can do myself due to running a very old desktop PC. I need to upgrade sometime but I'm stuck with what I have for now.

I can ftp to the Xbox and easily replace the file thereafter.

The only other thing I can think of is to make a pull request and then get it committed so I can update the core through my Xbox and test it it there.

Thanks again for looking at this with us @grant2258 . I'm def keen to get this long standing issue sorted as I'm sure many are.

Very appreciative of any further help I may receive from you.

[grant2258]

Had a quick look seems like its using mingw64 or mingw32 for 64/32 bit. Just compile that with msys2->mingw64 from the start menu the

make -f makefile.libretro  SUBTARGET=mame SOURCES=src/mame/pacman/pacman.cpp 

I had no issues with this compile no idea why your friend is having issues. Tell him to try the above is a small compile.

[joepogo]

Thanks @grant2258, i passed this information along and hopefully this will work regarding a compile.

If not, ill just make a pull request for an experimental fix to disable threading and then test it out once the fix is committed to this repository to make sure it works. We can always revert if it does not.

Thanks again and will update soon.

[joepogo]

Hey @grant2258 I pushed a pull request for this but it looks like I need to fix some #ifdefs so it's only affecting the Xbox/uwp versions.

I'm not sure what to put where. Hopefully you can help a noob out.

[grant2258]

Well I found gcc 13 doesnt compile the master branch tell you friend to make sure the he does a if he isint on the mame0255 branch already.

git reset --hard
git checkout libretro/mame-0.255 
git clean -xfd
make -f makefile.libretro  SUBTARGET=mame SOURCES=src/mame/pacman/pacman.cpp 

see if that fixes his compile problems as new installs of mingw will have gcc 13. This is a very small compile the pacman driver if he can compile this he can apply any patches he wants and just test with the dkong driver.

[joepogo]

I passed this info along to him, thank you grant .

I'll update once he attempts to do it.

And I'll update this thread once your pull request gets approved and we can test.

Thank you again for your time and help thorough out all of this so far.

[grant2258]

Hope it all works out for you guys in the end this here https://github.com/libretro/mame/blob/5e04369f1154f5458431fd9e01b7c70cf14ff932/src/osd/libretro/retroprefix.h#LL14C1-L14C42 makes no sense for a uwp target

[joepogo]

Still waiting for sonninos to merge your pull request..

We did get a compile too though!

Hopefully we can test this out and put the crashes to bed.

[grant2258]

Im sure he will get to it am slow at times myself being busy with other stuff. Its certainly worth a try if people think its an issue.

[joepogo]

Yea, no worries I'm sure he'll get it soon.

Again I can't stress enough how grateful I am to be getting your help grant!

Regarding the file you linked to above is something wrong with it?

[grant2258]

Its defining the api as windows xp for mingw I can see why they are doing that so it works on xp but not so great on uwp. Thats a libretro thing though they can decide what to do with it. Im guessing it was done when an xbox core wasnt available.

[greenchili2]

I was able to try a test build that someone was able to compile ( Gabo? ) on the Xbox Series X from pull request https://github.com/libretro/mame/pull/380 and this did the trick! Every game with discrete sound that I tried worked. I tried 59 of 176 parent rom sets. A few I had to update the romsets due to the changes in the jump to 0.255.

Suffice it to say "I'm getting it on like Donkey Kong".

It would be nice if there was someway to just disable this for the Xbox One systems anyways. Unless it is also an issue with other "UWP" devices. I have none to test so I couldn't say.

I do have a question for @barbudreadmon is there an easy way to tell from the source which titles use threading?

[greenchili2]

I just tried cruis'n USA and it crashes... I did a search and I noticed the SDLMAME_EMSCRIPTEN defines are checked in the VIDEO.CPP file.. as well as some OSD source files related to retroarch. So more investigation may be needed to find a better way to do this. In the meantime if we could try something with the threading turned off in the video code as well. That would be great!

[barbudreadmon]

this did the trick!

I'm glad my guess helped solve the issue

It would be nice if there was someway to just disable this for the Xbox One systems anyways.

I'm sure someone knowledgeable will figure out how to properly implement the correct if defined macros.

is there an easy way to tell from the source which titles use threading?

You could try searching for references to osd_work_queue_alloc in their source code, this is what i found :

src/devices/machine/laserdsc.cpp:85:        m_work_queue(osd_work_queue_alloc(WORK_QUEUE_FLAG_IO)),
src/devices/video/poly.h:474:       m_queue = osd_work_queue_alloc(WORK_QUEUE_FLAG_MULTI | WORK_QUEUE_FLAG_HIGH_FREQ);
src/devices/video/epic12.cpp:108:   m_work_queue = osd_work_queue_alloc(WORK_QUEUE_FLAG_HIGH_FREQ);
src/devices/sound/discrete.cpp:893: m_queue = osd_work_queue_alloc(WORK_QUEUE_FLAG_MULTI | WORK_QUEUE_FLAG_HIGH_FREQ);
src/mame/sega/coolridr.cpp:3188:    m_work_queue[0] = osd_work_queue_alloc(WORK_QUEUE_FLAG_HIGH_FREQ);
src/mame/sega/coolridr.cpp:3189:    m_work_queue[1] = osd_work_queue_alloc(WORK_QUEUE_FLAG_HIGH_FREQ);
src/lib/util/chd.cpp:2787:  m_read_queue = osd_work_queue_alloc(WORK_QUEUE_FLAG_IO);
src/lib/util/chd.cpp:2788:  m_work_queue = osd_work_queue_alloc(WORK_QUEUE_FLAG_MULTI);

poly.h is for polygons (meaning 3D) discrete.cpp is for TTL epic12.cpp is cv1k's vdp chd.cpp is for chd laserdsc.cpp is probably about laserdiscs coolridr.cpp is probably about sega's cool riders arcade game

[grant2258]

@greenchili2 do a grep -rnI threadid and grep -rnI osd_work_queue_alloc (which should work now) its the former that still will have issues. As I mentioned to @joepogo there is a problematic setting of the windows api to xp this was probably done before the upw core for the xbox though.

scripts/src/netlist.lua:21: "_WIN32_WINNT=0x0501", scripts/src/osd/retro_cfg.lua:57: "_WIN32_WINNT=0x0501", src/osd/libretro/retroprefix.h:14:#define _WIN32_WINNT 0x0501 // Windows XP

0601 0603 0x0A00 (Windows 10)

you will probably want to implement this lib to get things working. https://github.com/neosmart/TaskThreads or change the code to use task threads.

[joepogo]

I'm still testing but the list of crashing games is nowhere near as many as before. I'll post my full test list soon here when I'm done.

Cruisin USA is def still crashing.

Pinksweets, espgaluda II and others work now but ibara is crashing (but still need to look into it) but most of the shooters on that driver are working.

Thank you so much this far:

@barbudreadmon @grant2258 @greenchili2 @sonninnos

Football you've done so far. It's pretty awesome to see your theory was correct @barbudreadmon . 😁

I'll be back soon with testing results.

[greenchili2]

@greenchili2 do a grep -rnI threadid and grep -rnI osd_work_queue_alloc (which should work now) its the former that still will have issues. As I mentioned to @joepogo there is a problematic setting of the windows api to xp this was probably done before the upw core for the xbox though.

scripts/src/netlist.lua:21: "_WIN32_WINNT=0x0501", scripts/src/osd/retro_cfg.lua:57: "_WIN32_WINNT=0x0501", src/osd/libretro/retroprefix.h:14:#define _WIN32_WINNT 0x0501 // Windows XP

0601 0603 0x0A00 (Windows 10)

you will probably want to implement this lib to get things working. https://github.com/neosmart/TaskThreads or change the code to use task threads.

Thanks for the info. But that kind of stuff is waaay out of my wheelhouse. I think for now just having the option to turn off threading for audio and adding it to work for the video in games like Cruis'n USA will be the best bet.

I know for Mednafen PCE threading was added to the CD audio code. On the OG XBOX it wasn't working simply because it was expecting the system to auto assign the thread a handle name ( according to doc it should ). I ended up having to make up a name and then it worked. Oddly enough. Anywho.. I'll give that article a read. Maybe I'll pick up a thing or too.

But my familiarity with retroarch code is very limited at the moment. But if you could add something into video.cpp (in MAME) similar to what you had in the osdsync.ccp file I think that will cover most everything until a more equitable system can be done.

[GABO1423]

I was able to try a test build that someone was able to compile ( Gabo? ) on the Xbox Series X from pull request #380 and this did the trick!

Just chiming in to say that yeah, I was the friend who tried compiling this lol. Glad to have been of help.

[joepogo]

Can confirm now.

ALL mame games now work with this toggle available!! This includes cruisin USA/world and all other games I had in my folder that were crashing including chd games. Even ridge racer which I never was able to get working on any core.

Obviously some games that were benefiting from the 3D threading on PC took a small fps nose dive on series x so in the future if Xbox friendly threading can be added, I think those games can be sped back up most likely to full speed.

Firstly, thank you to @barbudreadmon for your very helpful input and insight looking into what could be the problem. Your guess was correct! And I can't thank you enough.

Secondly, thank you @grant2258 for your patches and code and pull request to get the toggle in and committed to mame.

Thank you to @sonninnos for merging the pull request and thank you to @GABO1423 for compiling the mame .255 core for Xbox and putting up with my noob shit on discord lol. 🤣

Thank you to my buddy @greenchili2 for testing as well.

If you @barbudreadmon or @grant2258 want a small challenge in the future maybe you can give some insight as to why maybe shaders are broken on flycast in retroarch on Xbox.

Luckily there's not too many issues plaguing retroarch on Xbox. The mame crashing and flycast issue were my only 2 biggest ones.

I'll probably put up a bounty source for that one. Hoping it's easy.

I'll close this issue soon as the initial problem is now solved. Hoping we can somehow get some Xbox friendly threading in and wash hands of this problem.

Thanks to all of you for your time, input, and patience.

[grant2258]

glad it worked out for you there is some conflicting information about crusnusa the report before said it didnt work.

[joepogo]

@greenchili2 had the wrong rom. 🤣 It runs for him now too.

[TheElecPlay]

Can confirm now.

ALL mame games now work with this toggle available!! This includes cruisin USA/world and all other games I had in my folder that were crashing including chd games. Even ridge racer which I never was able to get working on any core.

Obviously some games that were benefiting from the 3D threading on PC took a small fps nose dive on series x so in the future if Xbox friendly threading can be added, I think those games can be sped back up most likely to full speed.

Firstly, thank you to @barbudreadmon for your very helpful input and insight looking into what could be the problem. Your guess was correct! And I can't thank you enough.

Secondly, thank you @grant2258 for your patches and code and pull request to get the toggle in and committed to mame.

Thank you to @sonninnos for merging the pull request and thank you to @GABO1423 for compiling the mame .255 core for Xbox and putting up with my noob shit on discord lol. 🤣

Thank you to my buddy @greenchili2 for testing as well.

If you @barbudreadmon or @grant2258 want a small challenge in the future maybe you can give some insight as to why maybe shaders are broken on flycast in retroarch on Xbox.

Luckily there's not too many issues plaguing retroarch on Xbox. The mame crashing and flycast issue were my only 2 biggest ones.

I'll probably put up a bounty source for that one. Hoping it's easy.

I'll close this issue soon as the initial problem is now solved. Hoping we can somehow get some Xbox friendly threading in and wash hands of this problem.

Thanks to all of you for your time, input, and patience.

Good to hear that most of the games work. Are the latest commits for mame on retroarch series X available via the online updater or are they still being tested?