VR controller issues with VR Template

[AltoRetrato]

I'm still very new to Unreal Engine, so sorry if this is a newbie issue.

The VR controllers don't work when I use the plugin with a project based on the UE VR Template.

I can build the plugin and run it on UE 5.11 and 5.2 on Windows 11 in a blank project, and in that case the VR controllers work fine in LibretroMap, but I'm looking to bring it to a VR project already in progress (and based on the default VR template, with all the VR optimizations already set up). If I build the plugin inside the VR Template project or copy the compiled plugin folder from a blank project, it seems to run mostly fine, except that the VR controllers don't work anymore (even in the LibretroMap).

Any tips about how to solve this?

Thanks!

[N7Alpha]

If I build the plugin inside the VR Template project or copy the compiled plugin folder from a blank project, it seems to run mostly fine, except that the VR controllers don't work anymore (even in the LibretroMap)

I’m not sure what’s going on here. Maybe the bindings in my Input.ini aren’t being loaded for some reason in your current project? You should be able to check this in project settings I think. You could also check world settings to make sure its still using LibretroGameMode maybe?

I think you can just ignore this issue although I would like to get it sorted out if you find the cause. Assuming you’re basing your code on the UE5 and later VRTemplate then porting the arcade machine and arcade machine gun actors should be a matter of replacing the libretro grab components with the grab components from the VRTemplate and rewiring the events (onGrabbed, onTrigger, etc.) to reference the grab component from the template. This will consist of deleting the red event nodes and then replacing them with ones referencing the template grab component. I believe there are 5 in total to replace between the arcade machine actor and arcade gun actor.

Let me know if that works for you or you’ve found the cause of the first issue I’ve discussed

[AltoRetrato]

To reproduce the issue:

1. Create a new project based on the VR Template
2. Copy the Plugins folder to your new project
3. Open the project and the LibretroMap
4. Preview the map in VR Mode and try using the controllers

In Project Settings, all Libretro Action Mappings and Axis Mappings appear the same both in the VR Template project as well as the blank project, and World Settings > GameMode Override is also LibretroGameMode. I had already checked this before.

Yet, I noticed that Config\DefaultInput.ini in both projects were considerably different in size. Using the DefaultInput.ini of the blank project in the VR Template project seems to solve the issue.

I'm not sure, but the VR template was originally using the EnhancedInput, and maybe it has priority and consume the VR controller input first?

[N7Alpha]

I tried to figure this out, but I ran out of time for tonight. It seems since the normal Input system is being deprecated probably the correct solution is to handle the different systems in an #if in C++, but I might play chicken with Epic since I don't think they'll remove the old system for awhile. It's weird that the First Person Character controller bindings all still work though.

[N7Alpha]

I found the root cause using the information you gave. The offending line in the config was

[/Script/OpenXRInput.OpenXRInputSettings]
MappableInputConfig=/Game/VRTemplate/Input/PMI_VRTemplate.PMI_VRTemplate

Since this is a global variable there wasn't really a workaround I could think of that wouldn't require changing the user's project settings. There might still be one I just couldn't figure it out. Instead of messing with the global settings I just added a warning box that should display at least in the editor that explains the problem.

[AltoRetrato]

Awesome! As always, thank you for all the effort put on this project! I really appreciate it! 😄

[AltoRetrato]

Sorry to bother with this again, but I'm still having some issues.

Even though I can move around and use the triggers in the controllers as you have programmed, I'd like to use other buttons and controller functions as well, but I can't seem to trigger the required events.

I have an actor implementing the Libretro interface, but instead of a gun it is a maraca (to play Samba de Amigo for the Dreamcast). What I'd like to do is:

• Be able to use the other buttons (e.g., Oculus Touch (R) A Press) for core Libretro inputs (e.g., Joypad Start, via Set Input Digital).
• Send the X and Y coordinates of each controllers (or each "maraca") as core inputs (e.g., Analog Left X/Y) (via Event Tick? Or is there a "core event" for each new core frame / update?).

The interface events (Trigger Pressed) work, but other action mappings (LibretroMenuToggleLeft) don't.

Do I need to change LibretroVRInteractionInterface.h to add more events for this to work? Or am I doing something wrong?

[N7Alpha]

Hmm I don’t know why LibretroMenuToggleLeft wouldn’t work. I’ve never tested it myself. You could try just handling the OculusTouch_Left_Menu_Click event directly and see if that fixes your issue. I assume you are also modifying the correct actor VRPawn blueprint or the LibretroVRPawn.h? You shouldn’t have to modify the LibretroVRInteractionInterface.h. That only exists to forward inputs to held actors as I understand it.

…There is no core tick/event so just do it in tick. That is an interesting idea though.

[AltoRetrato]

Thanks! I read the code and Blueprints again, and ended up using VRGameMode to try to implement my logic with the Enhanced Input System in the VRPawn. Now the correct events are being called.

But when I try to set the digital input of the core, it doesn't seem to work (e.g., the game doesn't respond to a Joypad Start).

In the Blueprint with the LibretroCoreInstance component, I'm saving a ref. to the core instance in the Game Instance:

And in the VRPawn, I have the event that get the VR controller button press (and release, etc.):

The Set Core Input Digital macro gets the core instance ref. from the Game Instance, and if it is valid, set the digital input:

I added some debug printing and all the values seem to be correct:

LogBlueprintUserMessages: [VRPawn_C_0] [OK] Core Valid. Port: 0, Activated: true, Input: 3, Controller ID: 1,281, Controller Description: Maracas
LogBlueprintUserMessages: [VRPawn_C_0] [OK] Core Valid. Port: 0, Activated: true, Input: 9, Controller ID: 1,281, Controller Description: Maracas

So it seems I'm sending the correct values to the core, the controller (1281 = Maraca) is in the right port (0), yet the game is not responding.

Any idea of what I'm doing wrong now?

[N7Alpha]

Make sure you also set the digital input activated to false. That’s effectively releasing the button which the core/game might only register the input on release. Also see if the core has any settings for the specific game you’re playing. You can drop a breakpoint in core_input_state to see if flycast is actually polling for input and UnrealLibretro is actually giving it the correct inputs. I ended up doing this a lot because the cores/games can be quite temperamental when it comes to handling input. I might just need to tons of debug messages to help with that

[AltoRetrato]

Thanks! Those are all great tips, but in the case of the Flycast / Dreamcast, it should detect any state (pressed and released). I still set the input "on" and "off", but didn't work.

[VRPawn_C_0] [OK] Core Valid. Port: 0, Activated: true, Input: 3, Controller ID: 1,281, Description: Maracas
[VRPawn_C_0] [OK] Core Valid. Port: 0, Activated: false, Input: 3, Controller ID: 1,281, Description: Maracas

And it doesn't have any specific settings - I can run the Flycast as standalone and as a Retroarch core, and in both cases I can map keyboard or controllers to the maracas.

Most importantly, when I disable the lines below in DefaultInput.ini and run the LibretroMap with LibretroGameMode, the game responds when I shoot the LibretroCoreMenu...

;MappableInputConfig=/Game/VRTemplate/Input/PMI_VRTemplate.PMI_VRTemplate
;bUseEnhancedInput=True

So I'm definitely missing something. I'll try looking at everything again to see what I can find.

[AltoRetrato]

First, a bug report: when I use the Set Controller node to change the controller during runtime it always crashes with Unhandled Exception: EXCEPTION_ACCESS_VIOLATION 0x0000090100000001.

Now, back to the controller issues - I think I've finally figured it out.

I was using the 240p Test Suite to get a better sense of what was happening. I highly recommend it for testing emulators.

I noticed that using a Dreamcast controller (RETRO_DEVICE_DEFAULT / RETRO_DEVICE_JOYPAD, from what I understood), everything works as expected in 240p: both digital and analog inputs are registered correctly.

If I use another controller that is not not RETRO_DEVICE_JOYPAD, RETRO_DEVICE_LIGHTGUN, RETRO_DEVICE_ANALOG, RETRO_DEVICE_POINTER, RETRO_DEVICE_MOUSE or RETRO_DEVICE_KEYBOARD (e.g., Ascii Stick, Maracas, Pop'n Music, ...), only Send Input Analog works.

After patching FLibretroContext::core_input_state like this, they started working:

int16_t FLibretroContext::core_input_state(unsigned port, unsigned device, unsigned index, unsigned id) {
    switch (device) {
    case 1025: // Arcade Stick
    case 513:  // Twin Stick
    case 769:  // Saturn Twin-Stick
    case 1281: // Maracas
    case 1537: // Fishing Controller
    case 1793: // Pop'n Music
    case 2049: // Race Controller
    case 2305: // Densha de GO!
    case RETRO_DEVICE_JOYPAD:   return InputState[port][to_integral(ERetroDeviceID::JoypadB)     + id];

I'm not sure if there is a simple way to automatically detect how to read the InputState for each device, but RetroArch seems to do that (yet, at a quick glance at the code, I didn't think it was so simple).

So, is patching FLibretroContext::core_input_state the correct solution for my case, or am I missing something else?

[N7Alpha]

I’ll look into that bug.

If I use another controller that is not not RETRO_DEVICE_JOYPAD, RETRO_DEVICE_LIGHTGUN, RETRO_DEVICE_ANALOG, RETRO_DEVICE_POINTER, RETRO_DEVICE_MOUSE or RETRO_DEVICE_KEYBOARD (e.g., Ascii Stick, Maracas, Pop'n Music, ...), only Send Input Analog works.

I’m not sure what you mean by this. Are you talking about the blueprint functions for setting the input state. They should perform the same task essentially. They only differ in the type they receive.

I’ll add more to my comment later I’m at work right now

[AltoRetrato]

If I use another controller that is not not RETRO_DEVICE_JOYPAD, RETRO_DEVICE_LIGHTGUN, RETRO_DEVICE_ANALOG, RETRO_DEVICE_POINTER, RETRO_DEVICE_MOUSE or RETRO_DEVICE_KEYBOARD (e.g., Ascii Stick, Maracas, Pop'n Music, ...), only Send Input Analog works.

I’m not sure what you mean by this. Are you talking about the blueprint functions for setting the input state. They should perform the same task essentially. They only differ in the type they receive.

I'm glad you asked! I looked at it again and found the issue: it seems core_input_state should be using RETRO_DEVICE_MASK.

In a Libretro Core Instance I can set many different types of Libretro Core Controllers. In the sample screenshot below I'm using "Controller" (RETRO_DEVICE_JOYPAD) and "Light Gun" (RETRO_DEVICE_LIGHTGUN), both common ones, pre-defined in libretro.h, but also two custom controllers that are implemented in the Flycast core: "Maracas" (ID 1281) and "Arcade Stick" (ID 1025):

The Blueprint node Set Input Digital works fine for the default controllers I quoted at the top of this message (all defined in libretro.h), but it doesn't work for "custom" controllers, such as "Maracas" and "Arcade Stick", in this case both "subclassed" from RETRO_DEVICE_JOYPAD. These and more are defined in the Flycast core using the libretro.h macro RETRO_DEVICE_SUBCLASS:

#define RETRO_DEVICE_ASCIISTICK RETRO_DEVICE_SUBCLASS( RETRO_DEVICE_JOYPAD, 3 )
#define RETRO_DEVICE_MARACAS    RETRO_DEVICE_SUBCLASS( RETRO_DEVICE_JOYPAD, 4 )

Values and macro in libretro.h:

#define RETRO_DEVICE_TYPE_SHIFT         8
#define RETRO_DEVICE_MASK               ((1 << RETRO_DEVICE_TYPE_SHIFT) - 1)
#define RETRO_DEVICE_SUBCLASS(base, id) (((id + 1) << RETRO_DEVICE_TYPE_SHIFT) | base)

So, when core_input_state is called with a device of type "Maracas" (ID 1281) or "Arcade Stick" (ID 1025), there is no case for them and it returns 0, when it actually should treat them just like a RETRO_DEVICE_JOYPAD device.

int16_t FLibretroContext::core_input_state(unsigned port, unsigned device, unsigned index, unsigned id) {
    switch (device) {
    case RETRO_DEVICE_JOYPAD:   return InputState[port][to_integral(ERetroDeviceID::JoypadB)     + id];
    case RETRO_DEVICE_LIGHTGUN: return InputState[port][to_integral(ERetroDeviceID::LightgunX)   + id];
    case RETRO_DEVICE_ANALOG:   return InputState[port][to_integral(ERetroDeviceID::AnalogLeftX) + 2 * index + (id % RETRO_DEVICE_ID_JOYPAD_L2)]; // The indexing logic is broken and might OOBs if we're queried for something that isn't an analog trigger or stick
    case RETRO_DEVICE_POINTER:  return InputState[port][to_integral(ERetroDeviceID::PointerX)    + 4 * index + id];
    case RETRO_DEVICE_MOUSE:
    case RETRO_DEVICE_KEYBOARD:
    default:                    return 0;
    }
}

So, to solve this problem, we need to change core_input_state switch to:

    switch (device & RETRO_DEVICE_MASK) {

I tested this and it seems to work just fine.

[N7Alpha]

That is the correct solution I believe. I think the core is supposed to query you if you support applying that mask which is why some cores still worked even though they performed controller subclassing as well. Flycast probably just forgets to do this since people pretty much only use Retroarch as a frontend which is always applying the mask I'm guessing. I will add this in a commit at some point. Thanks for figuring this out. I was afraid I fundamentally misunderstood input handling until I saw it was just the masking issue.

[AltoRetrato]

Awesome! I'll close this issue, then.

And BTW, thanks to your work, we can now play Samba de Amigo in virtual reality with the Oneiric Quest! :D