search

libretro / flycast

Flycast is a multiplatform Sega Dreamcast emulator. NOTE: No longer actively developed, use upstream repo for libretro from now on - https://github.com/flyinghead/flycast
http://reicast.com
GNU General Public License v2.0
152 stars 77 forks source link

Make libretro reicast work on ARM (iOS/Android) and 32bit PC [$220 awarded] #12

Closed inactive123 closed 5 years ago

inactive123 commented 7 years ago

Related to these issues -

https://github.com/libretro/reicast-emulator/issues/11

https://github.com/libretro/reicast-emulator/issues/6

What we expect in order for this bounty to be considered fulfilled:

  1. It has to work on ARM Linux with the dynarec.
  2. Make it work for 32bit PC. Making it work on either Windows or Linux is enough and would be considered enough for this requirement to be fulfilled..
--- The **[$220 bounty](https://www.bountysource.com/issues/43656021-make-libretro-reicast-work-on-arm-ios-android-and-32bit-pc?utm_campaign=plugin&utm_content=tracker%2F49543223&utm_medium=issues&utm_source=github)** on this issue has been claimed at [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F49543223&utm_medium=issues&utm_source=github).
MastaG commented 7 years ago

I'm adding in an extra 50 euros just for the libretro ARM dynarec port. With the only conditions being that audio is working and we don't need to configure our input controls separately just for reicast.

Keep in mind reicast can already be built for ARM Linux as a standalone binary supporting everything (including sound and rumble support).

FTR, recalbox is already shipping reicast for RetroArch using wrapper scripts. See: https://github.com/recalbox/recalbox-buildroot/tree/master/package/reicast

inactive123 commented 7 years ago

@MastaG Hi there, cool to see your support.

Let me know when the 50 Euros has been added, I will then retweet it and draw some further attention to it.

MastaG commented 7 years ago

Done :) Its 70 bucks now. I hope somebody finds the time. This would bring dreamcast emulation to a lot of devices. Also keep in mind reicast is running very very well on ARM already.

hizzlekizzle commented 7 years ago

I made a forum thread to discuss this issue and bounty, as well: https://forums.libretro.com/t/make-reicast-work-on-arm-and-32-bit-pc/9969

tabnk commented 7 years ago

Nice :)

alegag commented 7 years ago

Did some fixes related to ARM this weekend (see PR #14), but not working on my ARMv7.

I've added to the bounty too, looking forward playing my DC games on my XU4 ;-)

tabnk commented 7 years ago

Hopefully $220 can attract developers to fix 32bit build for PC :)

gizmo98 commented 6 years ago

I added a rpi patch https://github.com/libretro/reicast-emulator/pull/18 but i have no clue why i get this error message:

Applying dynarec type hack.
Using Recompiler
Sh4 Reset
Get MemPtr unsupported area : addr=0xA0000000
SIGSEGV @ 0x762a00a4 (signal_handler + 0x0x234ef44) ... 0x762a00a4 -> was not in vram
[libretro INFO] Fatal error : segfault
 in signal_handler -> core/libretro/common.cpp : 389 
DEBUGBREAK!

I also added platform specific versions of ngen_AllocateBlock with arm DynaRBI::Relink() https://github.com/gizmo98/reicast-emulator/commit/11eca7ce75519cbf83130166b70ec773df4e38eb but the problem persists.

bassdr commented 6 years ago

I have a very similar issue on linux x86_64. SEGFAULT when loading at 0xA0000000 ... should it word on x64 PCs before porting it to arm/x86?

alegag commented 6 years ago

I've seen issues on the main reicast branch about x86_64 problems, including speed, you should try to compile in 32bits, it's suppose to work fine (unless your OS is not multilib and don't run 32bits code).

On Sep 12, 2017 21:42, "David Racine" notifications@github.com wrote:

I have a very similar issue on linux x86_64. SEGFAULT when loading at 0xA0000000 ... should it word on x64 PCs before porting it to arm/x86?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/libretro/reicast-emulator/issues/12#issuecomment-329032364, or mute the thread https://github.com/notifications/unsubscribe-auth/AUP7h0fQuY9YI9_m0VIPk57P-Z_XRxvCks5shzL5gaJpZM4Mvl8o .

bassdr commented 6 years ago

Thanks @alegag for the info. My OS does support multilib, I had to compile both retroarch and reicast_libretro in 32bits... and I did a few tests:

This is the same crash again. bm_GetCode() seems to be the reason of the segmentation fault... and comes from upstream reicast-emulator. Someone reported something similar, see https://github.com/reicast/reicast-emulator/issues/865

gizmo98 commented 6 years ago

Can someone test this branch with a 32bit PC? https://github.com/gizmo98/reicast-emulator/commits/patch-6

bassdr commented 6 years ago

@gizmo98 You branch does not link on x86 (x86_64 building with -m32 everywhere).

core/rec.o:(.bss+0x4): multiple definition of `ngen_FailedToFindBlock'
core/rec-x86/rec_x86_asm.o:(.data.rel+0x0): first defined here
/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/../../../../x86_64-pc-linux-gnu/bin/ld: core/rec-x86/rec_lin86_asm.o: warning: relocation in readonly section `.text'
/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: creating a DT_TEXTREL in a shared object.
collect2: error: ld returned 1 exit status
make: *** [Makefile:561: reicast_libretro.so] Error 1
gizmo98 commented 6 years ago

@bassdr please pull latest update and try again.

bassdr commented 6 years ago

@gizmo98 It compiles and link, still crashes when loading. BTW my test game is 'Soul Calibur (USA)[DCCM].cdi'.

Linux paging: 00001000 00001000 00000FFF
MAP 00800000 w/ 25165824
MAP 20000000 w/ 25165824
MAP 04000000 w/ 16777216
MAP 06000000 w/ 16777216
MAP 0C000000 w/ 0
A8
vmem reserve: base: 0xb8700000, aram: 0xd8700000, vram: 0xbc700000, ram: 0xc4700000
Resetting mem
*** Error in `/home/david/projects/libretro/retroarch/retroarch': free(): invalid pointer: 0xd8884008 ***
(gdb) bt
#0  0xf7fd8ae9 in __kernel_vsyscall ()
#1  0xf5d33749 in __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#2  0xf5d34c07 in __GI_abort () at abort.c:89
#3  0xf5d6e380 in __libc_message (do_abort=2, fmt=0xf5e5fb3c "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#4  0xf5d74357 in malloc_printerr (action=<optimized out>, str=0xf5e5c6ff "free(): invalid pointer", ptr=<optimized out>, 
    ar_ptr=0xf5ead760 <main_arena>) at malloc.c:5004
#5  0xf5d74b11 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:3865
#6  0x081605a5 in png_reverse_filter_adam7_iterate (data_=<optimized out>, ihdr=<optimized out>, pngp=<optimized out>)
    at libretro-common/formats/png/rpng.c:721
#7  png_reverse_filter_adam7 (data_=<optimized out>, ihdr=<optimized out>, pngp=<optimized out>) at libretro-common/formats/png/rpng.c:735
#8  png_reverse_filter_iterate (rpng=<optimized out>, data=<optimized out>) at libretro-common/formats/png/rpng.c:765
#9  rpng_process_image (rpng=<optimized out>, _data=<optimized out>, size=<optimized out>, width=<optimized out>, height=<optimized out>)
    at libretro-common/formats/png/rpng.c:1106
#10 0x08174634 in image_transfer_process (data=<optimized out>, type=<optimized out>, buf=<optimized out>, len=24538, width=0xf307f248, 
    height=0xf307f244) at libretro-common/formats/image_transfer.c:218
#11 0x0806ab8b in task_image_process (image=<optimized out>, width=<optimized out>, height=<optimized out>) at tasks/task_image.c:93
#12 task_image_iterate_process_transfer (image=<optimized out>) at tasks/task_image.c:136
#13 task_image_load_handler (task=<optimized out>) at tasks/task_image.c:227
#14 0x0806a9ed in task_file_load_handler (task=0x89416c0) at tasks/task_file_transfer.c:103
#15 0x0806453d in threaded_worker (userdata=<optimized out>) at libretro-common/queues/task_queue.c:459
#16 0x08132a84 in thread_wrap (data_=0x83cd350) at libretro-common/rthreads/rthreads.c:142
#17 0xf7c9d22a in start_thread (arg=0xf307fb40) at pthread_create.c:333
#18 0xf5dec4ce in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
(gdb) exit
kivutar commented 6 years ago

Any news on this?

rcaridade145 commented 6 years ago

A lot of crashes seem to mention 0xA0000000. That is the address to there the bios is loaded. Or better where one of the mirror for the bios must sit https://github.com/p1pkin/demul/blob/master/Memory.c

rcaridade145 commented 6 years ago

@twinaphex Is there any CI to build this ? i mean that because the log @bassdr posted pointed to some problems at libretro-common. On the commit https://github.com/libretro/libretro-common/commit/a2e5a51fae55a62eee186af9987a16496341a251#diff-3cdf6c2957b9f9c33e12d5b84e35253a from 2018/02 those issues seem to me taken care of. Can anyone please try again @bassdr @gizmo98

7oxicshadow commented 6 years ago

I don't want anyone to get excited by this as it is still very early days but I have successfully built a 32bit version of Reicast on Linux (yes linux only at this moment in time).

I need someone with linux experience to test a .so file for me.

I have tried it in a virtual machine and it gets in game but wont show any picture (sound is fine). I think this will be due to a lack of drivers and the library is working as expected.

@twinaphex are you a linux user?

Can anyone recommend an easy place to upload the so file?

7oxicshadow commented 6 years ago

reicast_libretro.zip

Create a backup of your original file and replace it with the one above. Launch a Retroarch and cross your fingers :)

inactive123 commented 6 years ago

Hey, I didn't read this before @7oxicshadow - please forgive me.

I have a Linux machine but it's x86 x64 though. Let me see if I can get a 32bit Linux machine setup in the next few days - is that OK.

rcaridade145 commented 6 years ago

@7oxicshadow what distro and hw are you running?

@twinaphex Not sure if you're familiar with https://www.osboxes.org/virtualbox-images/ not sure about 3d support

rcaridade145 commented 6 years ago

@twinaphex updated the Makefile from my branch (https://github.com/rcaridade145/reicast-emulator/blob/master/Makefile), with some changes namely on Linux, that were present on reicast Makefile and not on LibRetro.

blackman91 commented 6 years ago

Wow so does 7oxicshadow core works?

7oxicshadow commented 6 years ago

To be honest I am not sure in what I have achieved so far (if anything).

What I can tell you is that it compiles fine and definitely runs in the sense that I have game audio but I get a black screen, When the black screen is up I can't even get the retroarch menu to appear anymore. It is as if anything to do with the retroarch backend stops responding.

Given the dates of some of the reports in this thread I can't tell if I have actually fixed anything, or its just been that long since someone tried it that actually other fixes (from other contributors) have already made it get this far....

The core that I put up a little while back was built on fedora 27 but it might work on other distros.

ghost commented 5 years ago

@7oxicshadow I tried your core in a 32-bit qemu VM with virgl. RetroArch otherwise works fine by itself in this VM with normal XMB working with the OpenGL driver under X. My virgl supports 3.0 compat and 3.3 core profiles, as well as ES3... I compiled RA with normal desktop GL support. Here is the log from running a DC game:

http://dpaste.com/05DW456

Basically I get a black screen always. I can hear the swirl bios boot, and then nothing. No input/sound after that, but the CPU stays very active the whole time. Trying to boot a naomi game segfaults before I even hear the bios boot.

rcaridade145 commented 5 years ago

@bparker06 can you please try to build the current core on your qemu VM. There are been fixes to the Makefile and @flyinghead has been working like crazy on the gfx part so maybe your problem is already overcome.

7oxicshadow commented 5 years ago

Good news. I have a working (proof of concept) 32bit Linux Reicast core. (Linux only at this moment in time...)

Could either @bparker06 or @twinaphex test the following .so for me: Reicast_libretro_working_250718

I have tried it in a 32bit virtual machine but it should work fine.

inactive123 commented 5 years ago

Great to hear - let me try asking again.

7oxicshadow2 commented 5 years ago

Ok. I have been working on an older build of libretro/reicast and have just attempted to merge my changes into the latest branch.

I have had to make a couple of trivial changes to get it to build but it appears that some new GL changes causes the core to sigsegv long before the dynarec gets to do anything :(

I have proven that it gets as far as the following function call:

glsm.c -> glsm_state_setup -> glGenVertexArrays(1, &gl_state.vao); //line 2613

After this I am a bit lost as GL stuff is new to me. Could @flyinghead or @twinaphex take a look to see if there is an obvious reason for this to occur?

The log shows:

Sh4 Reset
recSh4 Init
Freeing fpcb
SIGSEGV @ (nil) (signal_handler + 0x0x56a546a0) ... (inl) -> was not in vram
DEBUGBREAK!

My updated branch is here: Github

flyinghead commented 5 years ago

glGenVertexArrays doesn't exist in GLES2, so it's probably a null pointer hence the SIGSEGV.

All the calls to glGenVertexArrays, glBindVertexArrays and glDeleteVertexArrays should be conditionally compiled with #ifndef HAVE_OPENGLES

7oxicshadow2 commented 5 years ago

@flyinghead. Thanks for the advice. I did manage to get further but I suspect there are other GL calls giving me problems. Almost all of the GL calls in gl_create_resources() had to be commented out before it would progress further.

It now loads the rom but crashes again with sigsegv. I have tried disabling both calls to dc_run() so that just the retroarch UI is being serviced but it still crashes even with that. (I know this normally works because I was exploiting that yesterday to do some debugging).

Speaking of debugging. Is there a guide on how to debug reicast anywhere? From what I have read it exploits the SIGSEGV mechanism for optimisation but as a result makes debugging with GDB incredibly difficult...

flyinghead commented 5 years ago

This is the libretro GL initialization so I suppose it has some issues with GLES2.

Regarding debugging just execute this, or put it in your .gdbinit file: handle SIGSEGV nostop noprint

inactive123 commented 5 years ago

I haven't really paid attention yet to see if Reicast actually works with GLES2. I suppose I should do that one of these days. I know Reicast OIT definitely won't work, so you will need to use the non-OIT renderer, and then make sure HAVE_GL3 is not defined, and CORE is not defined.

Since this dynarec is meant for 32bit x86, can you just try using desktop GL instead since the main focus is on getting the 32bit dynarec to work? What was the last snapshot in time then when you actually had the 32bit dynarec working? Any github commit on your fork to point to?

flyinghead commented 5 years ago

Actually I thought you were building on ARM. Didn't read the issue title up to the end... On linux x86 you shouldn't define HAVE_OPENGLES and you should probably define HAVE_GL3.

7oxicshadow2 commented 5 years ago

The version that I have done all my testing on is the following:

github

It works fine with default build settings on that build and I assume that is using GLES2?

The commit on my branch is: my_commit

Where does CORE get defined? I have seen HAVE_CORE?

These are the settings I have in the make file. Looking at them I suspect it is already set to use desktop GL?

DEBUG         := 0
NO_REND       := 0
HAVE_GL       := 1
HAVE_OIT      := 0
HAVE_CORE     := 0
NO_THREADS    := 1
NO_EXCEPTIONS := 0
NO_NVMEM      := 0
NO_VERIFY     := 1
HAVE_GENERIC_JIT   := 1
HAVE_GL3      := 0
FORCE_GLES    := 0
STATIC_LINKING:= 0
HAVE_TEXUPSCALE := 1
7oxicshadow2 commented 5 years ago

This zip file is the complete source to my working version. Just run make in a 32bit linux environment and the resulting ".so" file should work as expected:

32bit_source

Note: The lin86.s assembly file is slightly different to the one in my branch as I had to make it fit with the current code at the time of writing.

7oxicshadow2 commented 5 years ago

In order for people to understand why we have been having problems getting 32bit builds in Libretro I will try to give a brief overview.

The non libretro build of reicast effectively starts the dynarec in an infinity loop. Once it is running there is no way of stopping (well there is but it is stubbed out). This is usable in most cases as all you have to do is close the window and execution is forced to break. It will result in a SIGSEGV error but this doesn't really cause any problems.

The Libretro build is different. It has been modified to interrupt the dynarec, process all of the retroarch stuff and then jump back in again in a controlled loop. So if it already works on 64bit why doesn't it work on 32bit? For legacy reasons the 32bit stuff has been done in assembly and is completely independent of the 64 bit and to make things worse there are 3 assembly files all doing the same thing but in a different language for each. 1 for ARM, 1 for Linux and 1 for Windows.

As I am a linux user, I have concentrated on the linux assembly first. The reality is that not only have I had to try and get my head around the assembly language, but I have also had to learn how the libretro api works so i can debug it.

So to the nitty gritty, what did I have to do to make it work?

As you may have noticed, when starting the core the entire interface just goes black and stops responding with only audio doing something. This is because the 32bit libretro build was starting the dynarec and never leaving. As it never leaves, it doesn't process any of the retroarch stuff....

I have modified the assembly so we can leave the dynarec, store any relevent PC variables and then when returning, restore everything and carry on. Just the same way the 64bit process does.

In reality, all I need to do now is modify the windows and ARM builds and it "should" work for all 32bit builds.....

blackman91 commented 5 years ago

Genius! go collect that bounty my man.

7oxicshadow commented 5 years ago

Bit too soon for anything like that but we made a good start. There appears to be some issues with the latest code on 32bit at the moment to iron out.

7oxicshadow2 commented 5 years ago

@twinaphex How are you building under a windows environment? Visual Studio? Is there a project file anywhere?

Thanks

inactive123 commented 5 years ago

Would this fix your GLES issues?

https://github.com/libretro/reicast-emulator/issues/188

inactive123 commented 5 years ago

@7oxicshadow I use mingw/msys2.

https://docs.libretro.com/compilation/windows/

7oxicshadow commented 5 years ago

@twinaphex Thanks for that.

I have created a windows build to test that is available here: win_32

Note: This is built against my older build that works. I have not had time to experiment with the latest code yet. I have not been able to test it myself because retroarch just CTD everytime I open it on a windows virtual machine.

If anyone tests it, can you report back if it works or not.

Thanks.

blackman91 commented 5 years ago

I tested it and it won't load the core when selecting it in RetroArch load core option, on windows 7 64 bit pc though. Has anyone else tested it?

inactive123 commented 5 years ago

@blackman91 @7oxicshadow

Tested it in 32bit Retroarch on Windows, I get this when starting a game with the core -

reios: Hardware ID is: SEGA SEGAKATANA reios: Maker ID is: SEGA ENTERPRISES reios: Device info is: 5B21 GD-ROM1/1 reios: Area symbols is: U reios: Peripherals is: 07BBA10 reios: Product number is: T1401N reios: Product version is: V1.000 reios: Release date is: 19990730 reios: Boot filename is: 1ST_READ.BIN reios: Software company is: SEGA LC-T-14 reios: Software name is: SOULCALIBUR
reios: loading ip.bin from FAD: 45150 reios: bootfile is '1ST_READ.BIN ' reios: iso9660 PVD found reios: iso9660 root_directory, FAD: 45170, len: 6144 Found 1ST_READ.BIN at 000093 filename len: 14 file LBA: 547203 file LEN: 3679460 [libretro INFO] [LUT]: Product number: T1401N . left init Segmentation fault

blackman91 commented 5 years ago

@twinaphex

Have you tested the linux 32 bit build? http://www.mediafire.com/file/4nbz088fpi11b7w/reicast_libretro_working_32bit.zip/file

inactive123 commented 5 years ago

I don't have a 32bit Linux configuration setup.

7oxicshadow commented 5 years ago

the 32 bit wimdows build isnt working. i have managed to get a working test environment. There is a function called bm_getcode(). Within it is a reasonalby simple memory read. When ever it attempts read from the variable it crashes.

It is as if it is an unintialised pointer. I am not sure but i suspect that it is related tothe vram stuff. All of the vram initialisation stuff uses a specific windows functions and i think it works differently.

inactive123 commented 5 years ago

I pushed some commits courtesy of @flyinghead. While I cannot get x86 compiling for 32bit Windows right now, he says it should work for Linux at least. Can anybody try that?