lucasassislar / nucleuscoop

Starts multiple instances of a game for split-screen multiplayer gaming!
https://www.reddit.com/r/nucleuscoop/comments/clkqp9/all_new_nucleuscoop_supported_games/
MIT License
802 stars 190 forks source link

Virtual Keyboard input - How to use alongside controllers? #197

Open Mister-Curious opened 3 months ago

Mister-Curious commented 3 months ago

So I recently downloaded and installed Nucleus CoOp and am enjoying the possibility of playing Minecraft with my 6 year old at home. After a rough go at setting up MultiMC, I finally got things working, and I now have a better understanding of how I think things work. At this point I think I have identified an issue I would like to see addressed somehow, and if it is in fact possible. To preclude my query, I would like to briefly explain my controller setup. 

I play all my games on my Windows machine using DS4 Controllers alongside a program called DS4Windows which allows me to emulate Xbox (Xinput) which is native to Windows. Besides compatibility, there are other perks of using DS4Windows which include being able to send keystrokes (or combinations of keystrokes) to whichever game I am playing by pressing certain button combinations on the controller. These are essentially virtual keyboard outputs triggered by button presses. 

Its important to know that many games like Minecraft use a plethora of keyboard assignments, so playing on a controller alone can be fairly limiting if there aren't controller assignments available for vital game options. This especially true when mod creators don't implement controller mapping support (Inventory Styles Next is a huge mod that has yet to include controller support). This lack of accessibility means that the only way to access the features of some mods is to use the keyboard (which for many people with disabilities is difficult or impossible). The workaround then is to use a program like DS4Windows to send keystrokes via the controller.  In my case what I have been doing is to simply map a keyboard shortcut in the game settings (in the keyboard bindings), then assigning a button combination for that key in DS4Windows. This basically has DS4Windows send out the virtual keystroke when the button combination is pressed, so the game receives a keystroke.

Now all of this is fine and dandy in single player mode, however when we use Nucleus we are faced with the challenge of having to lock out keyboard input. I am not sure why this has to happen, but my guess is that somehow each instance window of MultiMC needs to be sent one set of unique signals from a controller, but keyboards sort of supersede that in the way that they will send input to whichever window is in focus. This would explain why that when I am sending virtual keystrokes from either controller1 or controller2 (via DS4Windows), that those virtual strokes are sent to whichever window is in focus. 

So here are a few examples of the necessity for accessing keystrokes in Minecraft:

So this all raises a few questions.

1 - Would it be possible to get Nucleus to understand which controller virtual keystrokes are coming from? I am not sure this is possible, as I mentioned earlier it might just be how keyboards are universal in that they send to whatever is in focus. I'd like to imagine that there might be a workaround with this with some kind of virtual keyboards that are assigned separately to each instance window so that the signals aren't getting crossed.

2 - Can we assign two separate virtual keyboards, one to each of the controller slots in Nucleus? I had a little more lick just now with assigning a keyboard to the player1 slot and was surprised that my controller still worked, AND the virtual keystrokes were received from DS4Windows WHILE the Input was locked. This gives me great hope, but I am not sure how this will work with two controllers.

Thank you for taking the time to hear my concerns. I look forward to coming to understand the inner workings of Nucleus a bit better, in hopes that we can make co-op gameplay a little more accessible.