search

redphx / better-xcloud

Userscript to improve Xbox Cloud Gaming (xCloud) and Remote Play experiences on web browsers
https://better-xcloud.github.io
MIT License
819 stars 50 forks source link

[Feature] Keyboard and Mouse Support #98

Closed braxento closed 10 months ago

braxento commented 1 year ago

Keyboard and Mouse support, pretty explanatory.

Not sure if this helps but: https://github.com/guilhermejfreitas/xcloud-with-keyboard-mouse

redphx commented 1 year ago

This feature takes a lot of time so I'm not sure whether I'll implement it or not. If I do, it will be created as a new script.

Haisom commented 11 months ago

@redphx could you explain what actually happened with your try to implement this feature on Release 2.0?

Also, could you update the 2.0 notes to make reference for this Issue? Thank you!

redphx commented 11 months ago

@redphx could you explain what actually happened with your try to implement this feature on Release 2.0?

Also, could you update the 2.0 notes to make reference for this Issue? Thank you!

The MKB feature in 2.0 is not the same as this one.

In version 2.0, I simply activated the MKB feature inside the xCloud website. But since xCloud disabled that feature it became non-functional. I won't try to activate it again as the devs don't want us to use.

redphx commented 10 months ago

I'm working on this feature. Follow this issue if you want to beta-test.

hectorevil commented 10 months ago

Thank you. i want to test too

redphx commented 10 months ago

How to test

  1. Install the test version (changelogs)
  2. Settings > Enable Mouse & Keyboard support
  3. Play any game

How to use

Press F9 to toggle this feature.

Keys:

TODO

Changelog

Feel free to suggest your ideas. Thanks.

image image
hectorevil commented 10 months ago

Tested and works correctly both in cloud gaming and remote streaming. I have also tried to change the controls without any problems.

I know there are some games that support keyboard and mouse on the console natively like Halo Infinite which I have tested via remote reporduction but it detects the controller. I guess it will be one of the limitations when "emulating" this feature.

redphx commented 10 months ago

@hectorevil thanks for testing.

Yes, this is not the same as using native mouse & keyboard. Do you have any suggestions about this feature? I never try other m&kb extensions and I only play using controller so I don't know what to expect.

hectorevil commented 10 months ago

@hectorevil thanks for testing.

Yes, this is not the same as using native mouse & keyboard. Do you have any suggestions about this feature? I never try other m&kb extensions and I only play using controller so I don't know what to expect.

For example on Xbplay allow to play using native mouse & keyboard on Remote and Xcloud. That's why I was asking

More features? Tried another extensions/program like Xbplay or Keyboard & Mouse for Xbox xCloud which allo tow change Mouse Sensitivity or do presets for every game image.

redphx commented 10 months ago

Update Dec 27: xCloud disabled support for all other games, except Flight Simulator.

Dec 23

window.NATIVE_MKB_TITLES = [
    '9PMQDM08SNK9', // MS Flight Simulator

    // Not working anymore
    // '9NP1P1WFS0LB', // Halo Infinite
    // '9PJTHRNVH62H', // Grounded
    // '9P2N57MC619K', // Sea of Thieves
    // '9NBR2VXT87SJ', // Psychonauts 2
    // '9N5JRWWGMS1R', // ARK
    // '9P4KMR76PLLQ', // Gears 5
    // '9NN3HCKW5TPC', // Gears Tactics

    // Bugged
    // '9NG07QJNK38J', // Among Us
    // '9N2Z748SPMTM', // AoE 2
    // '9P731Z4BBCT3', // Atomic Heart
];

There could be more but I don't have time to test them all. Check the full list of Xbox games that support MKB (some of these games might be bugged or not working).

@Haisom

hectorevil commented 10 months ago

tried a few games like Halo Infnite or Fortnite and works well. Also tried High on Life but need some settings like mouse sensitivity

redphx commented 10 months ago

New UI changes:

ricardo404 commented 10 months ago

Is this testing feature still active? I tried here and even selecting the option to activate mouse and keyboard, nothing appears in the Better Xcloud menus, already inside the game (MSFS).

AngusDLX commented 10 months ago

I tested on MSFS and the simulator recognized the keyboard and mouse natively.

Incredible. The mouse just needs a few adjustments related to sensitivity and it will be perfect. You are on the right track.

The keyboard and mouse plugin for Xcloud made by David Idol/Modern Kit is a great interface reference.

I have good news too: The feature also accepts voice commands. Using Voice attack I was able to execute voice commands that activate the keys on the keyboard and mouse. For complex games this helps a lot!

AngusDLX commented 10 months ago

Is this testing feature still active? I tried here and even selecting the option to activate mouse and keyboard, nothing appears in the Better Xcloud menus, already inside the game (MSFS).

I also had difficulty getting it to work. What I did was completely uninstall Better Xcloud from Tampermonkey, restart the browser, install the beta offered here and enable the keyboard and mouse function. After that I managed to make it work.

ricardo404 commented 10 months ago

Esse recurso de teste ainda está ativo? Tentei aqui e mesmo selecionando a opção de ativar mouse e teclado, nada aparece nos menus Better Xcloud, já dentro do jogo (MSFS).

Também tive dificuldade para fazê-lo funcionar. O que eu fiz foi desinstalar completamente o Better Xcloud do Tampermonkey, reiniciar o navegador, instalar o beta oferecido aqui e ativar a função de teclado e mouse. Depois disso consegui fazer dar certo.

Hey Angus, I think I solved it, I forgot to read the part about (only works in full screen). This will solve our 737 double click problem, if you know what I mean. At least I tested it and it worked.

AngusDLX commented 10 months ago

Hey Angus, I think I solved it, I forgot to read the part about (only works in full screen). This will solve our 737 double click problem, if you know what I mean. At least I tested it and it worked.

Yes it solves the problem. The hardest part now is using the keyboard and mouse to fly the aircraft. Are you using a control in conjunction with them? It's working fine?

I tried to configure the mouse to turn a "Yoke" and control the plane but the axes are only accepted up to the limit of the screen.

Even for future purposes, it is possible to use an unsupported peripheral such as (yokes, steering wheels, etc.) using an axis and button conversion software that would translate to the"virtual control". Unfortunately I lost all data about this. But it is possible.

redphx commented 10 months ago

Dec 24:

redphx commented 10 months ago

WIP

image
LucasG021 commented 10 months ago

I tried to change the mouse sensitivity with JS, but nothing takes effect. Are you using any specific API for keyboard and mouse support?

AngusDLX commented 10 months ago

Using the latest version: I didn't find any bugs.

About compatibility with other controls:

*It was possible to use voice commands via voice attack to press the keys

*Virtual control was possible in conjunction with the keyboard and mouse in games that natively support K&M. I believe it works with a physical control too (I don't have one at the moment).

In the flight simulator I used the Vjoy + virtual controller to create a virtual XBOX control and transform the mouse inputs into the right stick inputs and thus control the plane. With a key configured to enable and disable the yoke, everything worked very well.

This possibility opens the door to configuring Joysticks and Steering Wheels in the future even if there is no official support from MS.

LucasG021 commented 10 months ago

I just tested 65 games on xCloud, and all of them worked perfectly well with MKB compatibility. If you allow me, I can send you a list of the games I tested.

Regarding the mouse sensitivity, I haven't been able to solve it with JS yet. I could only manage it by using third-party programs to increase the DPI of my mouse.

oDenial commented 10 months ago

I tested Halo and loved the experience. Do you have plans to add a native keyboard and mouse to Fortnite?

lnahuel16 commented 10 months ago

I tested Halo and loved the experience. Do you have plans to add a native keyboard and mouse to Fortnite?

adds a new line of code: 'BT5P2X999VH2', // Fornite below Gear Tactics, but as @LucasG021 mentions in the previous messages, it has a sensitivity bug, the only way to fix it for now is to increase the dpi of your mouse.

oDenial commented 10 months ago

Can you tell me in more detail, please? I don't know anything about it

oDenial commented 10 months ago

I tested Halo and loved the experience. Do you have plans to add a native keyboard and mouse to Fortnite?

adds a new line of code: 'BT5P2X999VH2', // Fornite below Gear Tactics, but as @LucasG021 mentions in the previous messages, it has a sensitivity bug, the only way to fix it for now is to increase the dpi of your mouse.

I did it, thanks

redphx commented 10 months ago

I tested Halo and loved the experience. Do you have plans to add a native keyboard and mouse to Fortnite?

It's up to the game or xCloud, not me. All I did was enabling this feature.

oDenial commented 10 months ago

Testei o Halo e adorei a experiência. Você tem planos para adicionar um teclado e mouse nativo ao Fortnite?

Cabe ao jogo ou ao xCloud, não a mim. Tudo o que fiz foi ativar esse recurso.

oh yes, thanks for that

LucasG021 commented 10 months ago

The keyboard and mouse feature stopped working for me. Can you tell me if they disabled this feature?

oDenial commented 10 months ago

@redphx My native keyboard and mouse stopped working, do you know what could have happened?

oDenial commented 10 months ago

The keyboard and mouse feature stopped working for me. Can you tell me if they disabled this feature?

mine too

JUCIAN190 commented 10 months ago

my keyboard and mouse is not working in other games is only working in flight simulator

TibiaCrft commented 10 months ago

my keyboard and mouse is not working in other games is only working in flight simulator

same.

redphx commented 10 months ago

It means xCloud disabled the native MKB feature on server-side for real this time.

AngusDLX commented 10 months ago

I can still use the Keyboard and Mouse. I decided to do a test flight on MSFS after receiving notifications here but in my case I can still use it lol

Maybe it will only be possible to use it in Flight Simulator and I hope it stays that way because it's 100% more fun to use the keyboard and mouse in this game (and also because I spent 2 days configuring the controls non-stop lol).

And an update on compatibility with other software: It is possible to use it with REWASD. With REWASD you can transform the mouse into a yoke and switch between the yoke and your normal mouse to press the buttons. It was the best solution I found related to virtualized controls.

This feature + voice attack to give voice commands is heaven

oDenial commented 10 months ago

Isso significa que o xCloud desativou o recurso MKB nativo para real desta vez.

In other words, this was only available for a few days? now it will be impossible to use MKB with better xcloud

Mobin735 commented 10 months ago

@redphx keyboard and mouse support didn't work for any reason, and when would it be fixed?

oDenial commented 10 months ago

@redphx keyboard and mouse support didn't work for any reason, and when would it be fixed?

"It means xCloud disabled the native MKB feature on server-side for real this time."

oDenial commented 10 months ago

@redphx Do you know if you will be able to turn this feature back on?

redphx commented 10 months ago

@oDenial no

redphx commented 10 months ago
oDenial commented 10 months ago

Update Dec 27: xCloud disabled support for all other games, except Flight Simulator.

Dec 23

  • Enable native MKB support for some games. ⚠️ Only works in fullscreen mode.
window.NATIVE_MKB_TITLES = [
    '9PMQDM08SNK9', // MS Flight Simulator

    // Not working anymore
    // '9NP1P1WFS0LB', // Halo Infinite
    // '9PJTHRNVH62H', // Grounded
    // '9P2N57MC619K', // Sea of Thieves
    // '9NBR2VXT87SJ', // Psychonauts 2
    // '9N5JRWWGMS1R', // ARK
    // '9P4KMR76PLLQ', // Gears 5
    // '9NN3HCKW5TPC', // Gears Tactics

    // Bugged
    // '9NG07QJNK38J', // Among Us
    // '9N2Z748SPMTM', // AoE 2
    // '9P731Z4BBCT3', // Atomic Heart
];

There could be more but I don't have time to test them all. Check the full list of Xbox games that support MKB (some of these games might be bugged or not working).

@Haisom

Gears 5 also works with the native keyboard and mouse

redphx commented 10 months ago

Almost there...
The remapping feature is not working yet, but you can play with MKB using the default mapping.

image
delicio002 commented 10 months ago

Do you intend to add a feature to be able to control the games camera without the cursor appearing?, as if it were a native way?

AngusDLX commented 10 months ago

Microsoft Flight Simulator has stopped recognizing the keyboard and mouse natively.

R.I.P.

AngusDLX commented 10 months ago

Do you intend to add a feature to be able to control the games camera without the cursor appearing?, as if it were a native way?

A good solution for this would be a key that inverts the mouse to control the left stick.

I did exactly that with REWASD. The mouse always controlled the right stick but if I pressed and held the alt key, the mouse controlled the left stick so I could move the camera. Designating a key to do this inversion would be a good idea.

It's very good for complex games like MSFS, No Man's Sky, car games, etc.

delicio002 commented 10 months ago

quero muito que o dayz de certo com o teclado e mouse

redphx commented 10 months ago

Do you intend to add a feature to be able to control the games camera without the cursor appearing?, as if it were a native way?

@AngusDLX @delicio002 there will be an option for that, but it will come later

redphx commented 10 months ago

The remapping feature is ready to be tested!
I'll hide the cursor later.

image
L4nzinhu commented 10 months ago

[Bug] Mouse having to move to the edges of the screen when you need to rotate the camera Platform

Device: Laptop OS: Windows Browser: Chrome Browser Version:119 Better xCloud Version: 2.1.2

Bug descrition: It is only possible to rotate the camera a little, it is limited to the edges of the screen, this becomes even more problematic when taking the camera to look at the ground.

Solution idea: lock the mouse in the center of the screen and make it invisible, but in a way that whenever we rotate the mouse it will not leave the center but the camera will be rotated.

VIDEO:

https://github.com/redphx/better-xcloud/assets/155100711/0464badd-0071-4467-8eae-78fa97fc1379