Support for layout definitions in ZMK/Zephyr modules?

[lesshonor]

As I'm guessing you're aware, ZMK is shying away from maintaining boards and shields in its main repository, and to that end has provided a canonical way to externalize these definitions into modules. This allows experienced stakeholders to maintain "source of truth" modules which end users can simply source—instead of end users having to fork, duplicating code, and assuming a maintenance burden that is likely not understood until the build breaks.

Much like board and shield definitions, it would be ideal if Keymap Editor could refer back to one of these canonical modules or some other repository with a dedicated maintainer rather than end users having to wrangle their own copies. I understand that you maintain a repo with layout definitions; while I laud that effort, it also strikes me as you assuming the exact type of data management burden that ZMK is taking great pains to decentralize.

It's probably not reasonable to parse the west manifest, but...

• Maybe some way for the user to say "my layout definition is over at this URL" when it comes up as "not found".
• Maybe these dedicated layout definition repos could be pulled in as submodules (...this is probably just as complex as manifest-wrangling)

I understand that this would be in no way a small undertaking, but wanted to at least gauge the temperature of the room post-Zephyr module addition.

Your time and consideration are very much appreciated.

[nickcoutsos]

Hi!

Thanks so much for starting this discussion, I appreciate you putting so much consideration into the potential addition. I'm aware of ZMK's support for Zephyr modules but I haven't looked into it in much detail yet. My high level understanding is that in addition to the boards and/or shields a module provides they would also provide the layout JSON data?

I definitely see the benefit, but one thing that immediately jumps to my mind is the additional overhead of blindly searching additional repositories that may or may not include this metadata. That sounds minor, but the trick is that if I'm not looking for something specific (e.g., for corne I already know to look for config/boards/shields/corne/... and not config/boards/<arch?>/<keyboard name>/...) so it has to be a more generic. Even with GitHub's GraphQL API I can avoid recursively traversing the repo but it still takes time, and I don't know that actually cloning repositories will scale (I don't know how long this will be the case but right now I'm getting by on my cloud provider's free tier).

Caching helps a little but it's conceivable that someone creating such a module will be making changes quickly and testing so I have to consider that possibility. Even just watching my logs I've seen users load up the app and basically refresh triggering these external calls over and over again. All that to say I'm trying to be mindful of resource usage by preparing layout data upfront when possible and still allowing users to "sideload" custom versions.

I do actually like the idea of looking at west.yml to potentially understand some things a little better (right now I'm waffling on whether or not to try and replace layer label properties with display-name, and one way to be safer about that in the short term might be using that config to see that they're building against ZMK main already) even if I'm not necessarily willing to fetch external resources from it.

Even just allowing the user to paste in the URL to metadata is pretty reasonable, CORS issues aside. I've been toying with the idea of having a "setup wizard" component to the app to help users who get as far as creating an empty GitHub repo and then lose steam, and that would be useful for extending to the out-of-tree keyboards.

I hope that gives you some idea of where my head is at. Sorry if it's a little hard to follow, I'm quickly typing this up during my lunch break, but I'm certainly interested in continuing the discussion and exploring more ideas.

[lesshonor]

My high level understanding is that in addition to the boards and/or shields a module provides they would also provide the layout JSON data?

This is exactly what I first imagined when I began organizing a module with some number of keyboards in it: https://github.com/lesshonor/mechwild-zmk-keyboards ..Instead of putting $name.json into the corresponding user config repo next to the board or shield's keymap, it would live next to the board or shield in the module repo.

But I must stress: it's very early days, I am unfamiliar with your codebase, and my personal experience running public-facing websites predates the existence of nodejs. (What's a "CORS"? /s) Some other method could easily be more sensible from a scalability perspective, and make effectively the same amount of organizational sense.

Even just watching my logs I've seen users load up the app and basically refresh triggering these external calls over and over again.

Ahaha—if you saw a significant jump in usage over the weekend, that may have been me preparing layout definitions for the shields in the repo linked above.

[nickcoutsos]

Hey, just want to give you a heads up that I'm not (deliberately) ignoring this. I did have a response drafted yesterday that I generally like the idea but would have to explore options to not get killed by the extra API requests, but this comes after receiving some unfortunate news at work so I'm afraid there won't be any movement on this anytime soon.

If you'd like to keep the discussion going here or share updates with your experiments or conventions for working with Zephyr modules I'm happy to keep reading and offer my insight where I can.

[lesshonor]

You've already responded more times and more quickly than I ever expected, so don't stress over it. I too am busy slaying various dragons (...work and otherwise) and while I intend to get the aforementioned module + config repo up and running sooner than later, that is in no way reliant on this proposed/potential enhancement. If I do anything or come up with anything, I'll drop a note. ✌🏾