search

rbreaves / kinto

Mac-style shortcut keys for Linux & Windows.
http://kinto.sh
GNU General Public License v2.0
4.24k stars 212 forks source link

Feature Requests/Roadmap #32

Open rbreaves opened 4 years ago

rbreaves commented 4 years ago

So this is not really an issue ticket, just something I plan to keep open with feature requests I have been asked about. People are welcome to add more in the comments however.

High Priorities

Medium

Low

Not a Feature but important.

~A table or spreadsheet needs to be created with shortcuts listed and xkb keycodes and labels.~ Now that the underlying code and base has been created the only thing left to do is to enumerate what all needs to be linked together for as close as 100% coverage possible based on the default shortcuts available. 3rd party apps and additional configuration can be mentioned, but will likely remain outside the scope of Kinto or will simply be inactive until designated 3rd party and optional apps are installed.

* I would really like to add an Accented Popup Window but that one will probably take the most effort and might even be a separate project in of itself, because it will have to actively monitor keyboard input, and I do not want that to be part of Kinto in general. If keyboard input is to be actively monitored then it will be separate. There are methods for detecting when a key is held down, so that with duration and disabling key repeat should be sufficient to trigger a new x11 window with other virtual key options in theory.

** ~On the keyboard caret issue it is needed to fully support word-wise in browsers while still being able to map the arrow keys to other functions as well. This conflict does not exist on macOS, it is aware of input elements vs the general program.~

** ~Scenario: Firefox - Cmd + Left = Back button or Cmd + Left = Beginning of text line, but right now it is one or the other until I can detect the text input. If anyone has any ideas on how I can do this in x11 then let me know and I will implement it.~

~***Note: xkeysnail is not currently installed with or used by Kinto, but I believe it would simplify the process of adding support for one off applications that may not align well with the default system level shortcuts that occur for the GUI based apps and terminal apps in general.~

rbreaves commented 4 years ago

Making notes

To address the status of the caret for wordwise to work and to avoid conflicting with the browser or app shortcuts it looks like hooking into ibus may be required. More details in the following link, signals will have to be hooked/listened in on I believe. https://ibus.github.io/docs/ibus-1.5/IBusInputContext.html

Thanks to @ sarnold from the ubuntu irc channel pointing me in that direction.

More info here on how to get the location or status of the input caret. https://gitlab.gnome.org/GNOME/gnome-shell/issues/494

Ideas on how to work with the specific areas of IBUS that I'd want in C. https://github.com/kovidgoyal/kitty/blob/master/glfw/ibus_glfw.c

https://github.com/sgh/dbus-examples https://unix.stackexchange.com/questions/527283/watching-sd-bus-properties http://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html

Also interesting to make note of for a potential mini virtual keyboard. Eekboard would sorta be how it would have to be done. https://github.com/ueno/eekboard/blob/master/src/client.c

More references to xlib/x11 event types http://www.rahul.net/kenton/xproto/xevents_errors.html https://tronche.com/gui/x/xlib/events/processing-overview.html

rbreaves commented 4 years ago

And this is how you detect whether or not the caret is ready for input via ibus and the terminal.. granted hooking into it directly would be better, but in lieu of that this can be done.

caret ready for input = Panel/Extension/Emoji

dbus-monitor

IBUSADD=$(cat `ls ~/.config/ibus/bus -1rt | tail -n1` | awk -F'IBUS_ADDRESS=' '{print $2}' | xargs)
dbus-monitor --address $IBUSADD "member='SetCursorLocation'" | \
grep --line-buffered -v int32 | \
grep --line-buffered -o -P '(?<=path=/org/freedesktop/IBus/).*(?=[\;] interface)'

output

InputContext_322
engine/simple/25
Panel
Panel/Extension/Emoji
engine/simple/25
InputContext_323
engine/simple/25
InputContext_322
engine/simple/25
engine/simple/25
Panel
Panel/Extension/Emoji
engine/simple/25
engine/simple/25
engine/simple/25
InputContext_323
InputContext_322
engine/simple/25
Panel
Panel/Extension/Emoji
InputContext_332
engine/simple/25
Panel
Panel/Extension/Emoji
rbreaves commented 4 years ago

A more reliable command for detecting input focus on a text field.. essentially 1 means a text field has the focus and it doesn't spit out useless crap while they type, which setcursorlocation does (hence the concept I suppose).

IBUSADD=$(cat `ls ~/.config/ibus/bus -1rt | tail -n1` | awk -F'IBUS_ADDRESS=' '{print $2}' | xargs); \
dbus-monitor --address $IBUSADD "path='/org/freedesktop/IBus/Panel',interface='org.freedesktop.IBus.Panel',member='FocusOut'" 2> /dev/null | \
grep --line-buffered -o -P '(?<=object path \"/org/freedesktop/IBus/InputContext_).*(?=[\"])'

Can also monitor inputcontext via this

IBUSADD=$(cat `ls ~/.config/ibus/bus -1rt | tail -n1` | awk -F'IBUS_ADDRESS=' '{print $2}' | xargs);\
dbus-monitor --address $IBUSADD "interface='org.freedesktop.IBus.InputContext'"
rbreaves commented 4 years ago

So the caret check has more or less been completed, but not 100% just yet as I still have to decide how I really want to deploy the 3rd app category, browsers, which will contain the new run_onInput command option. Not entirely sure that any other app besides browsers will ever need this either, but if I am to make this mac like then it needed to be there.

The code that fixes it is here, also included in an earlier commit is the caret_check.sh script. Any solution in C would have been rather complex.. plus I'd have had to figure out how to do some sort of multi-threading to run XNextEvent and the caret check concurrently. At least this way a script handles writing the state to a /tmp location, and kintox11 reads the state and a new function will timeout XNextEvent if you are in an app category that has a run_onInput command to be ran at all.

https://github.com/rbreaves/kinto/commit/086ab8bb351b724bfa346ea169620ae7dd101c0f

rbreaves commented 4 years ago

Looks like there's a small delay between selecting an input box in a web browser vs other elements as it has to reload the entire keyboard layout each time and that takes a little bit of time. I am going to fix that though by using a combination of xbindkeys and xte to automate the resolution. It is much easier to work with for one thing than xkb, but more importantly when one off fixes need to occur it can happen faster than reloading an entire keyboard layout from an xkb file. I feel like there's just enough of a lag for me to not want to use xkb for input focus changes.

xkb is fast in its own right, but I think it'd be better to keep it primarily focused on global type changes and not be doing one off fixes per an app via xkb, even though that is entirely possible and plausible for apps that don't need a keymap changing based on what widget or element has the focus.

rbreaves commented 4 years ago

For anyone keeping track with this project, I think you can safely checkout out this commit on the dev branch if you want to try out the new keymaps for when text fields have focus in a browser. I have not taken the time to add anything more than forwards and back, as those 2 in particular conflict with word-wise.

56391169ffe42fcefbc852bf681523dc781bc3a1

I also may not be the one taking the time to flesh out ALL of the shortcut keys for individual programs, while kinto is about creating parity with macs keyboard wise - I am aiming for the most coverage with the least effort. I have created a platform of sorts with Kinto to get us there with less need to create completely new shortcuts for almost everything or trying to change modifier keys with 3rd party tools.

At this point though xdotool, xautomation and xbindkeys are being brought in to take care of the one off things, like the browsers and I have demonstrated in the latest commits who to go about configuring the json and and keymaps to handle apps like Firefox and Chrome. (Also I think xdotool is better than xautomation xte.. because you can actually clear the modifier and continually hold it down for additional actions to take place.)

rbreaves commented 4 years ago

So I need a better caret detecting script than this.. this one only works under Ubuntu/Gnome. I need something that will work with KDE & XFCE and ibus installed after the fact.

Once this is resolved I will update dev and merge to master with the new wordwise and browser keymaps (at least the start of the keymaps as this will fix conflicts that would typically be unsolvable with most keyswap solutions.).

This was resolved probably on Feb 18th, not sure, but IBus does work fine under KDE and XFCE and Kinto does configure it. There may be some issues with Manjaro and Gnome w/ IBus however, most likely a missing dependency/package of some kind. It has been well documented in the readme and in another open thread.

IBUSADD=$(cat ~/.config/ibus/bus/`ls ~/.config/ibus/bus -1rt | tail -n1` | awk -F'IBUS_ADDRESS=' '{print $2}' | xargs)
dbus-monitor --address $IBUSADD "path='/org/freedesktop/IBus/Panel',interface='org.freedesktop.IBus.Panel',member='FocusOut'"
# Actually ibus-mozc ibus-anthy installs japanese support - 
# may not need to do that just to run ibus-setup.. 
# unless it brings in missing dependencies for ibus-setup?
sudo apt-get install ibus-mozc ibus-anthy
ibus-setup
rbreaves commented 4 years ago

Very rough rough draft that shows how to remap using udev/hwdb instead of xkb. This should help with Wayland and apps that ignore xkb remaps.

It will not help in remapping a single key to 2 modifier keys at once like xkb though, which is needed for seamless terminal copy and paste. setxkbmap/xkbcomp is still the ideal method to use under x11, but remote desktop type apps, eg teamviewer, anydesk and others will need this method over xkb. Wayland is expected to need this as well.

https://gist.github.com/rbreaves/9a0a2bb8b0d6c2142acbbfa78c382262

jonchun commented 4 years ago

Feature Request: I'd like the option to provide flags for the one-time initialization tweaks on install so that I can integrate it as part of my new script. That's right... I did something completely unnecessary and overkill... See my updated repo. I know that autodetecting keyboard layout is non-trivial so that can remain, but I think people who haven't done any of this stuff before are going to be blindsided by the initial tweak options so I'd like to just provide them with the sane defaults.

Edit: wow... you already did this before with xosx...

rbreaves commented 4 years ago

Haha.. yea you found my xosx project, I had forgotten the exact name of it. But yea that project was a bit ambitious for the time, and specific to Xubuntu as I was trying to replicate the best of galliumOS and OSX at the time. After that project I decided, some months later, to just refocus on the keyboard stuff because I was never really happy with what I had done.

Also the initial Kinto app pre v1.0 did attempt to auto-detect the keyboard type but I just didn't want to maintain it as I know it needed more work and pre 1.0 it was using xprop in bash & used Super in the cmd location during terminal usage.. it wasn't horrible, but it was not as stable or fast as Kinto rewritten in C has been. Xosx also had been using Super during terminal usage. You also didn't have any json config files to work with or modify pre 1.0 release.

I probably will take another look at autodetecting at some point. I still think stability, adding udev/hwdb & a gui tool may be higher on my list at the moment. I keep thinking the tool is stable, but then your testing under KDE slapped me around a bit lol, but it's all good - I'm glad to have really resolved the badwindow issues.

rbreaves commented 4 years ago

Reference material on how to add caret detection under Windows by using Autohotkey. https://autohotkey.com/board/topic/128933-getting-caret-position-in-text/

rbreaves commented 4 years ago

Possible way of detecting terminals in an app having focus. Aka vscode, sublime, etc.

https://unix.stackexchange.com/questions/480052/how-do-i-detect-whether-my-terminal-has-focus-in-the-gui-from-a-shell-script

Could also re-implement the caret input check to update the new xkeysnail Kinto.py config file. It’s being watched for updates.. so patching or modding xkeysnail should not be needed.

wmertens commented 4 years ago

I'm interested in how this would work with Wayland - is there anything fundamental blocking it?

The way I understand this project, is that it does low-level changes influenced by high-level state. So perhaps it would be better to hook into gnome/kde instead of X/Wayland?

rbreaves commented 4 years ago

It will be more DE dependent, making some additional dev work required to support each DE.

I do have a gist for figuring how to detect the current app on gnome. https://gist.github.com/rbreaves/257c3edfa301786e66e964d7ac036269

I have a plasmiod on Plasma 5 forked & modified for triggering scripts on current app name changes.

So some key things are figured out, but setxkbmap & xkbcomp are not available, so DE solutions needed I suppose. At least for the xkb version of Kinto.

https://ask.fedoraproject.org/t/looking-for-alternative-for-setxkbmap-in-wayland/3380

The Kinto version in dev supports udev via xkeysnail so I think Wayland support for it would be an easier path.

rbreaves commented 4 years ago

When/if I get around to Wayland support before someone else it will likely come as a patch to xkeysnail assuming it’s primarily just missing wm_class names that it gets from x11/xlib.

teutat3s commented 4 years ago

I'm really looking forward for wayland support (using sway with wlroots)! Thank you for working on this!

rbreaves commented 4 years ago

Unrelated to most things but interesting to note somewhere, concerning the kitty terminal. https://github.com/kovidgoyal/kitty/issues/885

pshirshov commented 4 years ago

I'm looking forward for wayland support as well! Thanks for your efforts!

rbreaves commented 4 years ago

Method for auto-applying keyboard keymaps

Linux - Apple keyboards http://plind.dk/udev-happiness.html Need to ensure I can detect only keyboards in general as well.

Windows

Powershell - Apple detection Get-PnpDevice -Class 'Keyboard' -Status 'OK' -InstanceId 'VID_05AC'

AHK can more or less detect usb devices coming and going, so Kinto can make use of that. https://autohotkey.com/board/topic/8231-usb-device-connecteddisconnected-notification/page-2

AHKHID library could be used as well, although there's no clear licensing for it.

rbreaves commented 4 years ago

Auto detect and reset for Windows is completed in Alpha. Need to look into enabling multi-keyboard support for both Apple and Windows based keyboards while in Windows.

Interception or Interception AHK wrapper may provide some answers

https://github.com/evilC/AutoHotInterception https://www.autohotkey.com/boards/viewtopic.php?t=7545

dylan-k commented 3 years ago

I have some autohotkey stuff that helps me use Mac media keys. Would it be useful to share?

rbreaves commented 3 years ago

Yea, please do, I may try & mention the year or show a preview of the media keys since Apple had been changing it once in awhile.

rbreaves commented 3 years ago

Kinto tray and GUI app is done and is sitting in Dev, will be merged into master soon after a bit more testing of the relocation of OS level shortcut keys into xkeysnail as the old OS shortcut method has been removed and so has support for x11/c based implementation of Kinto.

EquusRaphanus commented 3 years ago

Hey, got a feature request for you! The installer should prompt you with configuration options - whether you want it for terminals only or system-wide. Having already rebound many shortcuts in my system's settings and wanting to use this to switch the keys in terminal, I thought this was broken until I edited kinto.py to get rid of all of the non-Terminal rebinds.

rbreaves commented 3 years ago

So you're not wanting to use the Cmd key position for Ctrl at all when using normal GUI apps? @EquusRaphanus I mean that could certainly be added as a feature in the Tweaks menu and if someone made that modification to the system tray to support it I would certainly look at supporting that configuration. And yes the Kinto.py file is fairly easy to change so that no keymaps apply at all for normal GUI apps if that is what you prefer.

Kinto is designed to give a mac-style shortcut key experience out of the box as fast & easily as possible, so asking questions during the install that deviate from that by adding or taking away from it is something I only want to reserve for a Tweaks style menu, which the Linux version has today. (Windows will need to follow that example).

The Windows installer does prompt users during the install, but at some point it will be more like the Linux installer in that it will only ask the user to press 1 or 2 keys during the install - all other options or features will have to be set or enabled by the system tray or Kinto app after that. I really don't like the idea of having a lot of configurations for users to go through during the setup as they could be numerous and quickly grow, also it encourages the quick addition of features w/o any quick or apparent way of turning them off or toggling them. By only accepting those type of changes as a Tweaks menu option after the install it keeps Kinto simple & features that go against any well known or established UX out of the equation.

I have been asked, w/ great passion, to include all sorts of scenarios at this point as though some configurations ought to be obvious even when no known OS even has their preference set as a default 😂. I am sometimes surprised by some people's expectations, I am happy to include just about anything, but it will be in the Tweaks menu & kept out of view during the install. If there's one thing I know for certain it is that people get nervous & easily turn away when you start asking them how they want to configure things, no matter how smart, intuitive or well reasoned those prompts may be. There's something that is sometimes implied when you do installer config options too and that is "this is the only time you have to make this choice" so it avoids that scenario as well. I'd rather add in a blurb during the setup that if you want to change your settings, or a specific setting then to please checkout the Tweaks menu after the install.

EquusRaphanus commented 3 years ago

@rbreaves, I'm on KDE Plasma. I had already remapped several GUI keyboard shortcuts to be more Mac-like (using altwin:ctrl_alt_win in my xorg.conf.d for keyboards). Now that I've stumbled upon this, I wanted to integrate Kinto to make Terminals behave as well. So I removed the xorg.conf.d settings, but without reverting all my other Plasma System Settings based shortcuts, several things stopped working. What made things more confusing was that when I tried to rebind some of those shortcuts with Kinto on, the default kinto keybindings were making that impossible until I stopped the xkeysnail service (because some of the multi-key bindings were causing the DE to produce weird results when trying to read keypresses - for example it thought what I assumed was Cmd+Tab to Alt+F1 until I disabled the non-Terminal shortcuts.

In any case, thanks for the great program and I'm looking forward to see how this works for me!

rbreaves commented 3 years ago

@EquusRaphanus Cmd-Tab should remap to Alt-Tab as would be expected. Not sure why you'd say Alt-F1, it is Cmd-Space that remaps to that, an application start menu activation as it tends to work like spotlight.

There was a time when Cmd-Tab would remap to a custom combo on an F key but I removed that several versions ago so that fewer system level shortcut changes would be made.. to none really. Makes disabling, suspending or removing Kinto a lot cleaner as well since the OS pretty well stays as it was.

I have liked KDE in the past but of the DE's I develop this for it tends to be tougher to work with simply because some of its changes just can't be quickly tested without a lot of logging off and back on. Gnome, Budgie or XFCE don't tend to be this way, either the change is instant or restarting part of the DE is sufficient. As far as DE's that I think work best with Kinto I'd put it in this order Budgie, Gnome, XFCE and KDE, but as far as I know they all work as I'd expect them to.

I would like to setup some CI/Pipelines though to continuously test the remappings for me with every update. I don't make major changes often but it is at a point where I'd like assurance that the remappings do not ever regress with future updates.

beefchimi commented 3 years ago

Just curious if there is any progress on:

exclude games option (full screen apps?)

I have to keep remembering to "disable" Kinto whenever I launch a game 🙃 maybe there is an easy way to tell Kinto to disable itself if Steam if running?

rbreaves commented 3 years ago

I mean xprop --spy (linux) can always be used to tell you you wm_class if using linux & then adding that to “remotes” array would exclude that specific game. Windows AHK has Window spy but something w/ remotes more or less, just written out differently.

Steam running & steam running in the background I imagine will look the same. Only thing useful is to hook into something unique to a game running full screen I think.. & even then you’d need to work out a script to check however many seconds for full screen & then it can either kill or start Kinto.

I don’t game much at all so not much of a priority.

51n7 commented 2 years ago

A solution to the game issue might be to set a global shortcut to disable/enable kinto, that way you can be in full screen still and disable the keymap if/when you want. I wrote an AutoHotkey snippet to do the same thing by double tapping right control but this is to quickly disable AHK so other users can still use the Windows machine without being confused.

miclgael commented 4 days ago 
exclude games option (full screen apps?)

Thanks for making Kinto! Is there any update to the game issue?