[BOUNTY] Write an ARM64 DYNAREC

[natinusala]

Writing an ARM64 DYNAREC (JIT compiler) would allow games to run at full speed on the growing number of ARM64 platforms :

• Android phones through RetroArch64
• iOS devices running on ARM64 (starting from the iPhone 5S if I'm not mistaken)
• Nintendo Switch through the Switch OS
• Some of the newest boards Lakka supports such as RK3328 and RK3399-based boards, as well as the Raspberry Pi 3B+

Link to the associated bounty : https://www.bountysource.com/issues/63766562-write-an-arm64-dynarec

[tiliarou]

contributed, any people interested tackling this task ?

[iOS4all]

@natinusala If we get a fully working ARM64 JIT can we use it on other emulators like dolphin and ppsspp? Also can we get it to work on horizon os as well? Thanks a lot.

[natinusala]

Dolphin and PPSSPP already have an ARM64 JIT. Actually, PPSSPP uses Dolphin's.

And yet we can get it to work on Horizon with some porting work to map the executable memory (libnx takes care of that).

[iOS4all]

Thanks for replying because some of devs including the creator of ppsspp is waiting to get ARM64JIT support in order to start to port ppsspp on horizon OS. Also I noticed pcsx on horizon is lack of JIT as well. So we hope to could port it to other emulators on horizon os once we get it implemented.

Thanks again.

[natinusala]

If I'm not mistaken the N64 has the same architecture as the PSX : MIPS. The N64 JIT could be used as a base for PCSX reARMed.

[iOS4all]

Sounds amazing indeed...

[pgarba]

Would it be also ok to lift the jitted MIPS code to x86 and then lift this x86 code into LLVM-IR with Remill ?!

This code can then be compiled with LLVM to AArch64 and would have the benefit to allow strong optimizations. It would also be a nice shortcut to get JIT for all targets supported by LLVM.

[dmiller423]

They are very different mips, if you're making a dynarec that'll handle 32b,64b for both endian: you might as well write it for a full IR and retarget it to many architectures... That's a TON of work though, and the n64 has a full mmu.

@pgarba you want to translate to x86, then pull the code and translate to llvm ir then back to arm? at runtime?

[pgarba]

Yes, as this project already has support for x86 its easy to lift the instructions to LLVM IR with Remill. If the x86 JIT is working I don't see a problem with endianess or the MMU as we just lift the x86 opcode behavior to LLVM IR. I understand that doing the lifting during runtime could be expensive but the translation could be done once and stored inside a cache or maybe even AOT translation could be possible like Android with ART is doing it.

[dmiller423]

Even if you succeed it would be impossible to debug, at the best.

[pgarba]

Remill is very stable and will be used in combination with mc sema to recompile whole applications.

https://github.com/trailofbits/remill/

But it depends on the quality of the x86 translation.

[inactive123]

There are one or two main concerns I have about taking the LLVM approach -

1) Dependencies - libllvm would have to be baked into the core repository. We don't want any external dependencies. 2) We tried the LLVM approach in Parallel N64 already for an RSP dynarec. I think a native non-LLVM dynarec would still be a lot faster, and the current issue we have is that since @TinyTiger wrote the LLVM dynarec, the LLVM API has already been ABI broken a bunch of times.

So my two main concerns here are from a dependency perspective, and from a long-term maintenance perspective where the LLVM API gets ABI broken so that we have to continually update the dynarec. To a lesser extent I am also concerned about the performance we would leave on the table as a result of opting for the LLVM approach.

Can my concerns be assuaged to a degree? I think it would help with the LLVM approach if we first tried to fix the RSP dynarec situation - for instance, trying to bake in libllvm into the core so that there are no more dynamic library dependencies at runtime.

[pgarba]

I just read the blog entry about the LLVM approach and it looks like he generated C code first and then compiled it into the LLVM IR. I think he already solved a lot of things so maybe it would be a good start.

I think updating the LLVM API is not the difficult. I will look into his code and try to update it to LLVM 7.0. But honestly once the update is done there is no reason to update to newer versions all the time as long everything is working.

[inactive123]

OK, remember what I said earlier ago about the dependency issue. I think a good way to test if a LLVM dynarec is the way forward here is to first get the dependency issue sorted. So we basically need to look at integrating the libllvm source into the core itself. I know RPCS3 itself does something similar to this for their standalone binary so it must be possible to do this. If we can do this, we can finally have the RSP dynarec working again for the Windows and linux builds as well.

[pgarba]

To get the libLLVM as static lib it has to be compiled from the llvm/clang sources which takes some time for compilation and I think they only support cmake to build it

[inactive123]

We definitely cannot rely on cmake for the core, so we need to find a solution either way for it, or move off the LLVM solution altogether. But we have to be able to use our static Makefile, and we cannot have dynamic library dependencies. Those are the two main rules that we cannot diverge from.

[NightlyFox]

so.... i know nothing about this stuff, whatsoever. but! this guy wrote a arm64 JIT compiler for LUA a while back... https://github.com/LuaJIT/LuaJIT/issues/26 https://github.com/akopytov/LuaJIT/commit/3fa3827ed4b0e27d2c4460556a6335c79db09dc4 maybe we can get him to do one for us if we offer him up this bounty haha. unless we can just use an lua to c converter lololololol.... if that exists... probably wont work. i can dream, i wish it was that easy... Edit: found one o.o https://github.com/davidm/lua2c

i hope this helps...

Apparently they are already porting/ported(someone got the 32bit version working) the luaJIT library to the switch.. https://github.com/LuaJIT/LuaJIT/issues/347

[dmiller423]

Even if that were useful, it doesn't help translating all of the MIPS64 instructions.

[NightlyFox]

I was hoping that it had code inside of it that could be copied over possibly, so that if someone does work on this. They dont have to work from scratch. There is a lot of info in that github that i was hoping was useful. Also, maybe reaching out to Mike Paul and seeing if he is willing to write a ARM64 JIT for a quick $1000+ bonus wouldnt hurt lol

[firerooks]

Hi, I would like to work on this. Is it ok if I port the current 32bit arm dynarec to 64bit?

[inactive123]

@firerooks Definitely, no issues there.

[dmiller423]

A quick 1000$? there is nothing quick about it, this is months worth of work ...

[firerooks]

Great! I'll get to it then

[Gillou68310]

Hi, I would like to work on this. Is it ok if I port the current 32bit arm dynarec to 64bit?

@firerooks I worked on this back in 2016, it's wip so not everything is working but at least it's a good start. Also a lot of things changed in the core so it will need a huge rebase.

https://github.com/Gillou68310/mupen64plus-core/tree/new_dynarec/wip/arm64

[firerooks]

@Gillou68310 Thanks for your contribution! I looked at it and found it helpful

[Gillou68310]

@firerooks Did you already started working on this? I'm considering finishing what I started but I don't want to work on this if you're already working on it!

[firerooks]

@Gillou68310 Yes, I already started. I'm also configuring a way to cross compile to ARM by using visual studio so that I could debug the code

[dmiller423]

Configuring a build and using a debugger within visual studio are two very different things, one you can do fairly easily esp on recent versions with linux/ssh support ( even to WSL ). The other is a royal pain, and you're better off using a normal gdb client but : the GDB extension is avail. now as well. (unless you're running windows on the target and using the VS ARM compiler)

[firerooks]

I'm aware that a normal GDB client is easier to set up, but VS is just a preference of mine and I figured it was worth attempting to set up

[Gillou68310]

ok don't hesitate to ask if you have any questions cause I'm pretty familiar with the new dynarec's code ;-)

[billy-acuna]

Hi @firerooks Any chances to see the progress on a repo og yours? If not it is OK :)

[firerooks]

@billy-acuna Not yet, I'd like to finish more before posting

[Ploggy]

One month check in :) hows things progressing @firerooks? everything going well?

[firerooks]

@Ploggy I've been working on it steadily, but progress was limited due to a busy schedule irl. Now I should be able to devote more time to it. I haven't encountered any major issues thus far.

[Ploggy]

Thanks for the update @firerooks.. It's ok I understand good thing's take time ;) I'm glad things are progressing hitch free :)

[Noah670]

Switch support soon.

[IntelMiner]

Commenting so I remember to keep an eye on this

Gonna tackle ARM64 Pi support once this comes together

[warlockv2]

Same. Leaving my comment here to follow. Anticipating this alot. Good luck with it @firerooks

[yshui]

You can just click the "Subscribe" button on the right side. You don't have to comment. @IntelMiner @warlockv2

[fzurita]

@Gillou68310 I'm surprised you haven't claimed the reward already. You were pretty much done and you were working only on optimization. What is missing in this branch of yours?

https://github.com/Gillou68310/mupen64plus-core/tree/new_dynarec/wip/arm64_optim

It seems to work pretty well on my arm64 Android device. Although, there is that bug in CBFD.

[warlockv2]

Yah thia aeems like its not really going anywhere lately. Id reccomend not waiting and get your reward. We havnt even seen anything yet

[inactive123]

If @Gillou68310 has a working dynarec already that works quite well for the majority of games, I am completely fine with just backporting it to this repo and claiming the reward.

What are the current outstanding issues with that dynarec? I hear Conker's Bad Fur Day mentioned, anything else?

[natinusala]

Could it be ported to both mupen64plus and parallel-n64 ? It was implicit in the bounty, I think it's just a copy paste job

[tabnk]

Let see how well it perform if it work on Nintendo Switch.

[inactive123]

Yeah, let's make it a precondition that it would get backported to both mupen64plus libretro and parallel n64. That way both codebases benefit at the same time.

[fzurita]

I believe @Gillou68310 was only working on optimization so that there was a speed advantage to using a arm64 dynarec. I believe he only made a 64 bit port to get it working in 64 bit, so there is no performance benefit in his implementation compared to 32 bit arm.

[warlockv2]

Yah i mean dude has it pretty much done. Its been over 2 months now. I think he should be able to get a claim on this. We havnt seen no progress from anyone so.

[fzurita]

Porting this implementation to parallel-n64 should almost be copy and paste. The structure looks almost the same in this area to Mupen64plus: https://github.com/libretro/parallel-n64/tree/master/mupen64plus-core/src/r4300/new_dynarec compared to https://github.com/Gillou68310/mupen64plus-core/tree/new_dynarec/wip/arm64_optim/src/r4300/new_dynarec

[Gillou68310]

I'm honestly expecting more from this bounty then just a port to 64bit. Allocating 2 registers on 64bit in order to emulate 64bit mips instructions is just nonsense. Plus we are inheriting all new_dynarec issues specific to the 32bit platform. I'm ok to work on those tasks, I actually already started by porting the new_dynarec to x64 in order to facilitate future development. But honestly I won't waste my time if this bounty is just for doing a lazy port to arm64.

[fzurita]

It's still a lot better than interpreter for 64 bit arm devices that can't run the 32 bit arm dynarec.