Make it easy to customize the keymapper with a second config

[joshgoebel]

Is your feature request related to a problem? Please describe.

I'd like to make Kinto easier to customize without someone losing their changes to the global kinto.ky every time a user upgrades... ie the system kinto.py should be CANON... but users should be able to easily drop changes on top of that in their own kinto_custom.py or some such.

Describe the solution you'd like

A second config file that holds custom user mods... and we'd also need to make it easy to "alter" keymaps, not just add new ones, ie:

alter_keymap("General GUI", {
  # custom changes here
})

Describe alternatives you've considered

Nope. Just wanted to start this discussion.

Additional context

Nope.

[RedBearAK]

@joshgoebel

Previous conversation from last year on this:

https://github.com/rbreaves/kinto/issues/480

At the very least the installer script should make a backup of the user's existing config file, if there is one.

The most simplistic solution is just importing defined shortcuts from a secondary file, and doing the import near the top of the default Kinto config, so that they override anything coming later. Just have to make sure that when there is a syntax error the user can still know what line in which file is causing the error, so that it can be fixed. If the main Kinto config could still load successfully even if there was a syntax error in the external custom file, that would be a nice bonus.

[joshgoebel]

The most simplistic solution is just importing defined shortcuts from a secondary file,

That doesn't really work for altering existing mappings... I think any user config needs to be layered on "top" of the existing config (with the ability to overwrite, etc). Just an import or cat a b isn't going to cut it IMHO.

update: Ok funnily enough it could work (in some cases) since the order or override is kind of "backwards" but having duplicate conditional keymaps is only going to make the mapper slower and make it harder to debug when you have issues...

[joshgoebel]

Actually I think just being able to drop files into the keymapper folder could be nice... so you download "softwareZ.keymap" and just drop it into a folder then when the keymapper runs that file is loaded and parsed... so supporting new software is as easy as cp...

We aren't there yet, but it's possible to the keymap to get too large... it's not optimal to have 200 keymaps that have to be scanned at every single keypress when a user only users 5 of those packages, etc.

[RedBearAK]

@joshgoebel

update: Ok funnily enough it could work (in some cases) since the order or override is kind of "backwards" but having duplicate conditional keymaps is only going to make the mapper slower and make it harder to debug when you have issues...

How much would that really be a problem though? There are already a lot of situations where a single app has multiple layers of keymaps applying to it, if I understand correctly. Most of the time customized shortcuts will be applying to specific apps, to override things from the "general" block. So it would be just like plopping a new keymap in near the beginning of the file.

We aren't there yet, but it's possible to the keymap to get too large... it's not optimal to have 200 keymaps that have to be scanned at every single keypress when a user only users 5 of those packages, etc.

Is it possible to load a new keymap on the fly, if you encounter a matching WM_CLASS pattern? I can absolutely foresee a database of a thousand different keymaps for specific apps. If they could be pulled into action only when needed...

[joshgoebel]

There are already a lot of situations where a single app has multiple layers of keymaps applying to it, i

Are there? Why? This seems suboptimal and harder to debug when you have the Firefox 1, Firefox 2, Firefox 3 and Firefox custom keymaps... It seems best if there was a single Firefox keymap... again, there is also a performance cost of every new keymap, having to run the conditionals over and over.

I can absolutely foresee a database of a thousand different keymaps for specific apps.

Your making one of my points for me though. 😎 If there was 1000 keymaps I would certainly NOT want them in one massive file... There are multiple good reasons to start wanting to split things up at some point.

Is it possible to load a new keymap on the fly, if you encounter a matching WM_CLASS pattern?

Sure, there are options we could consider if things got burdensome.

[joshgoebel]

Is it possible to load a new keymap on the fly, if you encounter a matching WM_CLASS pattern?

Your forgetting that keymaps aren't only conditional on WM_CLASS though they can be condtiional on just about anything... but yes that trick could work for WM_CLASS conditional keymaps. (which likely solves the "large app library" problem)

[joshgoebel]

There is an ownership/organization question too... I'm more likely to contribute a FULL Sublime Text keymap if I became known as the Sublime Text keymap maintainer. Until then I'll probably just contribute one offs... but if I had my own little area to work in I might be tempted to pull up the mac and linux keymaps side by side and actually make a FULL keymap that covers everything, once and for all.

I'm hesistant to do that in a file full of 30 other configs.

[RedBearAK]

@joshgoebel

If there was 1000 keymaps I would certainly NOT want them in one massive file.

Yes, I was agreeing with that. Having them as external "module" files would probably make sense. But you'd need to at least load into memory the part that tells you what conditions to watch for (like WM_CLASS or WM_NAME) to trigger loading the full keymap on the fly.

This seems suboptimal and harder to debug when you have the Firefox 1, Firefox 2, Firefox 3 and Firefox custom keymaps

I was just talking about how a single app will have "General GUI" and other keymaps all active at the same time. Don't know why it would be much of a problem to have "Default Firefox" and "Custom Firefox Overrides" keymaps simultenously applying.

In my Finder Mods I used a general block for shortcuts common to all (or most), and then override blocks for specific tweaks that each file manager app would need to work correctly. It reduced the number of lines necessary to support them all by quite a bit.

[joshgoebel]

Don't know why it would be much of a problem to have "Default Firefox" and "Custom Firefox Overrides" keymaps simultenously applying.

That's potentially twice as many keymaps... and why, what is the advantage of two maps vs one?

Just because we already have a lot of tribbles doesn't mean that we shouldn't care about more.... https://en.wikipedia.org/wiki/Tribble

---

In my Finder Mods I used a general block for shortcuts common ...It reduced the number of lines necessary to support them all by quite a bit.

Having sep files or modules doesn't prevent things like this... it would just need to be thought thru.

[RedBearAK]

@joshgoebel

ownership

Well, that's a bit egoistic, but also understandable. But it also reminds me that there could potentially be multiple keymaps for the same app, written by different people, that end users could choose between. For apps like Sublime Text or VSCode people have different preferences, even while using macOS.

That's potentially twice as many keymaps... and why, what is the advantage of two maps vs one?

Still not getting why the number of keymaps is an issue. They come in layers already, starting with the general block that applies to everything unless specific shortcuts are overridden. That's just how the Kinto config is laid out. There's a general web browser block, and then specific shortcuts for Chrome(s) and Firefox. Is there something wrong with that?

Having sep files or modules doesn't prevent things like this... it would just need to be thought thru.

I don't think I was saying it would. Just that if I hadn't done it that way there would be a lot of redundant lines for each file browser app that would have to be fixed individually if I made a mistake.

[joshgoebel]

Still not getting why the number of keymaps is an issue.

Simple O(n)... 200 maps is slower and more bloated than 100 because it duplicates work. And you haven't really given any reasons why double maps is better...

I'm interested in first talking about what is optimal, then as a final step circling back to what is possible. I think that's how you have better outcomes IMHO.

[RedBearAK]

And you haven't really given any reasons why double maps is better.

Consolidation of code, like I said?

You're not giving me any reason, other than a theoretical asymptotic limit, why it's a problem to have (1) general shortcuts, then (2) general browser shortcuts, then (3) Chrome-specific shortcuts all applying to a Chrome window. I don't understand how you think things should be done differently. Should every application with a keymap contain all of the shortcuts from the "General GUI" block redundantly, making the config file ten, twenty times longer?

I really don't know where you're coming from on this. The layers of keymaps has never caused me an issue with Kinto, as far as I know.

I feel like we must be talking about two completely different things.

And the "Custom Firefox Overrides" would be coming from an external config. Obviously unless the user wanted to override some of the defaults, there wouldn't be a second keymap. But there are already multiple keymaps that apply to every Firefox window, as described.

[joshgoebel]

Well, that's a bit egoistic, but also understandable.

I use the word in a broad sense. I see people pour tons of effort - sometimes their whole lives into projects they have some degree of ownership (even if it's a small piece) - this is really what OSS is built on largely... but then when you're just one random committer out of 1000, no real ownership, no responsibility, no idea if their patch will even be merged or not...

I think some ownership greatly encourages involvement...

[joshgoebel]

Consolidation of code, like I said?

I'm not sure I follow what you mean there...

why it's a problem to have (1) general shortcuts, then (2) general browser shortcuts, then (3) Chrome-specific shortcuts all applying to a Chrome window.

That was never my argument. My argument is that there is no need for also:

(4) Chrome (5) Chrome (more) (6) Chrome (custom)

All the Chrome specific stuff could just go in keymap("Chrome"), period - behind a SINGLE conditional check. Because the cost isn't theoretical... every time a key is pressed a ton of CPU is spent to figure out what keymaps to use... it's honestly ridiculous and only possible because of how insanely fast computers are, but that doesn't mean it's good or shouldn't be improved.

The layers of keymaps has never caused me an issue with Kinto

This argument is why software today is so slow. https://tonsky.me/blog/disenchantment/ Just because you personally haven't seen a problem doesn't mean that "more is fine" or that there isn't a real cost to more more more.

[joshgoebel]

Often the easiest way to make something faster is to do less work... if someone wants to extend/customize Chrome's keymap we should let them just do that - not force the keymapper to check for Chrome TWICE every time a key is pressed.

[joshgoebel]

The layers of keymaps has never caused me an issue with Kinto

Also people forget about power cost... even if "it's plenty fast" doing less work always means less energy... doing half the work means your battery lasts twice as long - even if you never notice a speed issue... there is always a cost. There was a time period (maybe still is) where Safari runs circles around Chrome in power consumption... Chrome is a plenty fast browser on good hardware, but Safari is faster - and if you don't notice it in speed you notice it in battery.

And even if you're plugged into the wall there is the electricity bill, the environment, etc... :)

[joshgoebel]

Just one example: (thankss for reminding me) 🙂

https://github.com/joshgoebel/keyszer/issues/25

This [asking X for the focused app] seems terribly expensive and slow yet in practical usage (on my fast computer) it seems "fine"... but it would still be much better if we could fix the underlying issue... keymapping is something that should be blazing fast not just "it seems ok to me".

Keymapping IMHO is something that should be "kernel level" fast. Obviously we'll never get there with Python, but we can get MUCH closer and I intent to.

[RedBearAK]

I think some ownership greatly encourages involvement...

That, or at least acknowledgement/attribution. I'd generally agree.

This [asking X for the focused app] seems terribly expensive and slow yet in practical usage (on my fast computer) it seems "fine"... but it would still be much better if we could fix the underlying issue... keymapping is something that should be blazing fast not just "it seems ok to me".

I'm very much in favor of theoretically making the keymapper as efficient as possible. But I still don't know how you would want to do the keymap layers differently, or why having just one more custom override for a specific app will really be an issue in most cases. Even with the keymapper in its current highly inefficient state.

As far as I know I've never been able to tell by looking at CPU usage that Kinto is even running.

Keymapping IMHO is something that should be "kernel level" fast. Obviously we'll never get there with Python, but we can get MUCH closer and I intend to.

Sounds fantastic to me. Just fix that Cmd+click first. ;-P

The argument to spend a lot of effort on the performance would probably go a lot further if you could actually point to a measurable effect or delay in real world usage. But theoretically a great thing to work toward.

[joshgoebel]

Sounds fantastic to me. Just fix that Cmd+click first. ;-P

Oh I was meaning to tell you to shorten the delay and see if that helps... I need to fix it first though... I wonder if say 0.2 seconds might be fast enough... depends how fast your brain works and how fast you click after hitting the key...

go a lot further if you could actually point to a measurable effect or delay in real world usage.

Have you ever used/owned a slow computer? Wouldn't it have been nice if it was 10x faster? I have had that experience, and I haven't forgotten it... people use much slower computers than my daily driver, Raspberry Pi, etc...

As far as I know I've never been able to tell by looking at CPU usage that Kinto is even running.

That only means it's low CPU not efficient... most modern software is terribly inefficient while also low CPU.

[joshgoebel]

or at least acknowledgement/attribution.

I think it's a spiderweb... those things are closely related to ownership IMHO.

[RedBearAK]

Raspberry Pi

I've actually had a couple of Pis for a year or so. Running Ubuntu 22.04 with Kinto on an 8GB model (working much better than earlier versions with the new ZRAM support), and have run Twister OS (Raspbian) with Mac theme and Kinto on a 2GB model. Kinto did not seem to ever be a noticeable drag.

Also, for a while I was playing with a single-core 32-bit P4 laptop with 1GB of RAM, running Linux Mint and Kinto. The remapper never seemed to cause a performance issue that didn't already exist without it.

But, like I said, I'll be looking forward to anything you can possibly do to make it even better.

That only means it's low CPU not efficient

Yes, it means it's efficient enough that most people don't care. You'll need to make a chart of test results showing how much more efficient your branch is after you fix all the things you think are slowing it down.

[joshgoebel]

You'll need to make a chart of test results showing how much more efficient your branch is after you fix all the things you think are slowing it down.

Oh I'd love to but that's not the highest priority... sadly many things will likely get fixed before we can measure just by common sense. If someone wanted to contribute some performance tests though I'd certainly be happy to have them.

It's hard to measure the full stack though since that involves an entire X stack (or at least it's outside my knowledge base)... but one can add metrics to the app and watch it live (which I've done)... and you can mass-benchmark a lot of the internal portions (resolving modmaps/keymaps, looking up keys, etc)...

I have a feeling there is a lot of low hanging fruit.

[joshgoebel]

Sounds fantastic to me. Just fix that Cmd+click first. ;-P

f8165f5

And try tuning the delay (manually) from 1s to lower and see if that helps... tranform line 200 or so... call_later.