N64

[humbertodias]

Implement missing libretro method that give support for mupen64, paralle64 cores. A good and functional c# wrapper: https://gitlab.com/aftnet/LibRetriX/-/commit/665458527d4941b93cd4068c4734c2e26c00177a

[Skurdt]

Are you able to run latest mupen64 builds with this? From what I can tell, RetriX doesn't support hw accelerated N64. Their cores also look like they are wrapper inside of an extra layer (and possibly using modified core sources to begin with) so technically the frontend only supports "Retrix Cores".

Cores that need hardware accelerated support are not working in any of the frontend implementations I found so far (c, c++ or c# alike) except for retroarch.

For Unity it's tricky... It's bound to what platform the build has been targeted for (gl for linux, gles for mobiles, d3d for windows, etc...) On windows, the editor and builds are running on direct3d, there's a way to access the device and context but so far, it crashes before the environment call needed to set it up for the core (in my case that's ppsspp) and I'm still trying to figure that out.

For OpenGL support (Mupen only supports OpenGL), there's no real option other than creating a context and texture outside of Unity and sending it back. I don't think there's any way at all to access the gl context used by unity for a particular texture without using a plugin, but I'd be happy to be wrong here :)

In short, I've been trying to implement hw acceleration for the past few days and, so far, I haven't had much luck...

[humbertodias]

Unfortunatly, no. I couldn't run muppen with the latest version. And your idea of using glfw seems to work.

I've just tested to draw some vertices there. https://drive.google.com/file/d/1qsDgIOZxsh3zyJeWFJ-BNjB3qigQuC0R/view?usp=sharing

Now, I'm trying to do the reverse way. Grab from gl framebuffer and plot it on unity's render. https://github.com/humbertodias/LibretroUnityFE/commit/bcefe8f1b9ad0378cab3581beaa7fe8b80f32b6a

[Skurdt]

The creation of the context and framebuffer seems to work fine. Passing any data happens in RETRO_ENVIRONMENT_GET_HW_RENDER_INTERFACE.

The problem right now is that I can't get to the point of where the core calls this without a crash.

And I still can't figure out how to compile a core successfully... (I want a debug version I can step into)

[humbertodias]

I see. on linux, I usually do

make DEBUG=1

using parallel 64 https://github.com/libretro/parallel-n64/blob/master/Makefile

[Skurdt]

Well stepping works now. It crashes because the context is empty. It looks like I do have to init the core's gl context somehow in RETRO_ENVIRONMENT_SET_HW_RENDER. Retroarch doesn't pass any references there so, and I'm just guessing at this point, maybe they are doing a context sharing of some kind using the current gl context.

[humbertodias]

I found a faster way for testing n64 core using libretro + glwf. https://github.com/humbertodias/nanoarch/tree/feature/n64 I'm testing it outside unity, because every time that crashes I had to kill the process and start all over again.

[Skurdt]

I'm using my wrapper outside of unity as well, I switch to unity to port the changes once I get the first retro_run to not crash.

My findings so far:

• Without calling update, closing the main window shows this:

  Assertion failed: 0, file libretro-common/libco/amd64.c, line 127

• When calling update, I get this:

  AccessViolationException >> Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

This happens in "libretro-common\glsm\glsm.c" when calling "glsm_state_bind" (basically the gl_state is empty)

• If I force a context reset in RETRO_ENVIRONMENT_SET_HW_RENDER using:

  Marshal.GetDelegateForFunctionPointer(inHwRenderCallback->context_reset).Invoke();

I get an AccessViolationException in "mupen64plus-libretro-nx\mupen64plus-core\src\main\main.c" at "main_run" when calling "gfx.romOpen()".

Now I'm just thinking libco and threading overall is involved in the crashes, but again, I'm just guessing...

[Skurdt]

I'm pretty sure I'm seeing a Nintendo logo ;)

[humbertodias]

haha..

I think, I understood.

Take a look at :1237 calling https://gitee.com/simon_haha/RetroArch/blob/master/dynamic.c

and here at :1968 https://gitee.com/simon_haha/RetroArch/blob/master/gfx/video_driver.c

a video_driver_context_lock

there is a mutex that must be avoiding the AccessViolationException

[supneo]

Yes is a nintendo logo :-)

[Skurdt]

the mutexes and negotiation interfaces are for vulkan only I think. opengl doesn't use threads (except in the bigger implementations with multiple contexts and a lot of abstraction...

the threading used for libretro are, from what I can tell, only used to switch to some game thread for the emulators, not sure though...

[humbertodias]

Finally! Outside of unity. Now, I'm gonna try to integrate that

./sdlarch ./mupen64plus_next_libretro.so Super\ Mario\ 64\ \(U\)\ \[\!\].z64

https://github.com/heuripedes/sdlarch

[humbertodias]

Seems that they had faced the same issue https://github.com/heuripedes/nanoarch/issues/5

[Skurdt]

Their issue is quite different in my opinion, it's written in c and uses the same libraries and libretro-common headers as retroarch. So they basically lack development time or effort...

I'm trying to do everything in c#, inside of Unity, running direct3d, start an opengl context to render a texture, and get it back to direct3d to use on a 3d model in the scene. Well that's the goal anyway....

But...........

I've got video and sound in the external window, when launching from unity now.

Some input buttons work but some don't (I don't have analog support yet and some of the input code is not that great... I don't think it has anything to do with the interop interface though).

Some major issues:

• In the editor it only works if I attach a debugger to Unity (don't ask me why...)
• In a build, the main camera seems to have a fov of 1 for some reason... (but the games run and don't crash. (I think it still crash when quitting but I do a full exit when pressing escape so I don't know yet)

Still that's progress...

duke nukem 64 and 007 seem to run ok, ocarina of time has very minor sound glitches once in a while, and mario is running slower.

now I have no idea how to get the output back in the scene so I can hide the temp window... :p

[Hokage3211]

So I apologize if I'm asking bad questions, but I'm a bit new to this area of emulation and I'm not exactly sure what to do next to make it work after adding the set_hw_render callback, using retroUnity as a base, I've been adding some good stuff onto it now using many different examples I've found as bases (mainly only care about making it work on windows for my current needs), and I'd like to get n64 (and maybe other cores by proxy?) working next. What do I need to look into next now that I've got the hw_render_callback stored after the core passes it to my code?

[Skurdt]

The basic rundown is this:

In RETRO_ENVIRONMENT_SET_HW_RENDER:

• Provide the get_current_framebuffer and get_proc_address function pointers through the retro_hw_render_callback (the data parameter).
• Store that same callback on your side (the frontend side)

After retro_load_game and retro_get_system_av_info (so you have the width and height of the content):

• Get or create your OpenGL/Direct3D/Vulkan context
• Call your stored retro_hw_render_callback's context_reset function pointer (which effectively calls the context_reset function on the core.

In the video refresh callback:

• Handle the case where data is not valid for software rendering (ie. data is equal to RETRO_HW_FRAME_BUFFER_VALID or null). null is a valid value, where you should handle frame duping if you support that. Cores should properly send RETRO_HW_FRAME_BUFFER_VALID but that's not a real problem to get this to work... Calling swapbuffer on a window was enough in my case but you can do whatever you want in this callback.

For unity specifically, you don't create your own context and you don't have an easy way to access it. Mainly because the rendering in Unity is multithreaded and for the case of opengl, it uses multiple contexts. Your resources must be created in the same thread that will render them, so needless to say that it will involve a lot of hacks to get this working... I've been able to render the games in a separate window in my hw-acceleration branch, but just because the context is mine and not linked to unity's (also unity is running in dx11 so that's an other issue...). Input and sound are working properly though so that's a win... I guess :p

[Hokage3211]

Hm I hooked up the openGL stuff as best I can to get it all into my codebase, copy-pasting all relevant stuff that I can find, I think, it seems to work fine and creates a new window no errors

but still crashes and this is the stack trace that I can find, it doesn't make it to any retro_refresh calls, seems to just crash outright.

[Hokage3211]

That is to say, that's all I've been able to hook up on top of my codebase, is there any other environment commands that I need to implement to get it to work, that come to mind?

[Skurdt]

as far as environment goes, you don't need anything else.

copy-pasting stuff is fine but, since nobody has been able to do what we want (at least on the open source front), you will have a hard time getting anything to work without doing the extra steps ^^ I've been at it for a week now and it's still not working for me :)

[Hokage3211]

really? How has humbert gotten it to work? Also curious what you mean by "extra steps"?

[Skurdt]

He didn't. I mean using debug builds of hw accelerated cores and step through their c or c++ code to figure out where the crash occurs and why. And also figure out ways to go around the unity limitations regarding context queries and mapping our resources to it.

[humbertodias]

That's true. Libretro wrapper with mupen only works for me outside of Unity. Right now, I'm experimenting two other wrappers that can expose the opengl context:

https://www.codeproject.com/Articles/1216876/Unity-Graphics-Emulator-for-Native-Plugin-Developm and https://gitlab.com/3dheart_public/vtktounity

The flow using plugin to grab the context here makes sense

[Skurdt]

So far I can:

• Create a framebuffer in a plugin
• Send the texture and bind it to the framebuffer's color attachment
• Tell the core to use this framebuffer for rendering

I know that doing operations when the framebuffer is bound works, since I can call basic functions like glClear with a custom color and it changes properly in Unity.

Going into the plugin from unity and back is pretty straightforward. The problem is that when calling core functions (retro_init, retro_load_game, retro_run, etc...), they assign threads themselves to the current one and at the same time issue graphics commands like gl initialization, resources creation and the like.

So basically, while in a render thread, it takes it as the main thread, and just invalidates all its states. At least that's what I think its doing, I'm hoping I'm wrong and it's an issue that can be resolved...

[Skurdt]

Pretty crazy way of doing it but I sort of got a result... This works only inside of a build (crashes the editor every time). And even in the build, exiting freezes the player and it needs a forced "end task" to be able to exit.

It's on a different unity project currently so I'm gonna attempt at porting the modifications to this version of the wrapper and see if that still works... and also make the changes public.

[Skurdt]

https://youtu.be/0uwgENTOxHs

[Hokage3211]

Oooh nice! Looking forward to seeing how it works!

[Skurdt]

How it works is barely and sometimes :) I'm pretty sure it only worked today because it's a full moon.

[Skurdt]

Pushed the changes as-is, I don't know when I'll refactor to revert back the bad class access attributes and make the api proper... If anyone can fix anything in the meantime... :)

[Skurdt]

Now with analog input: https://youtu.be/euec6832wNA

mupen64plus_next is somewhat supported now but other gl cores probably aren't. Not sure if I should close this issue and move towards fixing stuff in the other posted issues from now on (since parallel_n64 still doesn't work, as pointed out in the first message).

(Ps. to supports other rdp/rsp stuff in mupen, they need a vulkan backend, the native plugin has the blueprint to get started but I don't know anything about vulkan ^^)

[Skurdt]

Closing this now, improvements on this relates to add a more robust hardware rendering path and/or implement the vulkan backend. Also the code for this moved to an other repo.