per-key led feature

[Retic1337]

creating this to start work on the per key led lighting.

[joelspadin]

I think the first thing to do is to get a list of use cases for per key lighting. Here are a few I can think of:

1. Set different solid colors per key (e.g. mods one color and everything else a second color).
2. Dim/disable lighting for keys that have no function in the current layer.
3. Status LEDs (caps lock, num lock)
4. Static animations (e.g. color wave across keyboard)
5. Dynamic animations (e.g. ripples from pressed keys)

Other considerations:

It would be nice to have one system that manages all LEDs and then have mappings on top of that for per key LEDs, underglow LEDs, and standalone status LEDs. Having separate systems for each of those would likely lead to inconsistencies and make it difficult to do things like creating an animation that syncs between key and underglow LEDs.

Just like with the existing kscan and composite kscan drivers, we should support multiple LED drivers (e.g. direct GPIO, daisy chained ws2818, Lumissil matrix driver) and combinations of them controlling different subsets of LEDs.

Storing colors as red, green, and blue bytes is probably the format that works best across all LED drivers. We can still convert to/from HSV as needed for things like adjusting the underglow color. To support single color LEDs with the same system, the driver can simply read the red channel and adjust a PWM or use an on/off threshold.

We need to somehow support wireless splits. Central could send per LED color changes to the peripheral, but that probably won't perform well with fast animations. The peripheral likely needs to run the same LED control code as the central and just get high level commands like hue/brightness changes from central.

[Retic1337]

for #1 and #2 we will need to import the keymap into the driver to determine colors.

3 will require reading status of caps, num, etc..

4 i think would have to be a map of the leds position x and y relative to their placement on the pcb. so esc on a tkl would be 0,255 and right arrow would be 255,0.

5 would be easy once #4 is done.

Currently have a background color working with per key color changing based upon keypress. Where are we going to want to be creating led maps and defining colors etc, board.keymap? or in the device-tree?

[joelspadin]

for #1 and #2 we will need to import the keymap into the driver to determine colors.

3 will require reading status of caps, num, etc..

I think we should structure things so that the low-level LED drivers don't have any knowledge of the keymaps. Instead, something built on top of that (not sure if it would also be considered a driver or just application code) should be deciding what color each LED should be and sending commands to the LED driver(s). It is this higher level code that would need to know about the keymap and lock statuses.

One idea I had for this was to have a base LED map/animation that defines the color of each LED and then stack "transforms" on top of that which can optionally modify the colors before they are sent to the LED driver(s). For example, a status indicator could be a transform that overrides the color of one specific key based on the lock status. Dimming keys based on layer could be another transform that reads the keymap and overrides LEDs accordingly.

Pseudocode for how this might work:

# User-configurable part
def base_animation(colors):
    for n in range(len(colors)):
        colors[n] = LED_MAP[n]

def caps_transform(colors):
    if is_caps_on:
        colors[CAPS_INDEX] = (0, 255, 0)

def layer_highlight_transform(colors):
    for n in range(len(colors))
        if not current_layer_has_function(n):
            colors[n] = (0, 0, 0)

TRANSFORMS = [layer_highlight_transform, caps_transform]

# This part happens the same way no matter how things are configured
colors = [(0, 0, 0)] * LED_COUNT

def update_colors():
    base_animation(colors)

    for transform in TRANSFORMS:
        transform(colors)

    send_to_driver(colors)

Both the base animation and transforms would also need a way to trigger a color update (e.g. periodic for an animation, on layer change for layer-based dimming, etc.).

Where are we going to want to be creating led maps and defining colors etc, board.keymap? or in the device-tree?

The organization that makes the most sense to me is to define all the driver and hardware configuration in the board DTS/shield overlay and then put the LED color maps and any other user configuration (e.g. enable/disable dimming keys based on layer, etc.) in the keymap.

[KingCoinless]

No notes on how to write this yet, but I do have a recommendation for a specific kind of reactive per-key effect. On my Kemove 64/66 keyboard, there is a lighting mode which, when pressing any key, reacts on that key in a random color and disappears. Fn + Up and Down Arrow cause the brightness levels to raise or lower. Fn + Left and Right Arrow cause the effect's dissipation rate to speed up or slow down. It might be interesting to offer something like this in ZMK, or perhaps even a lighting mode which reacts around a specific color (for example, blue, or tints of blue). The color (or tints of) could be adjusted through the 'RGB_HUE' and 'RGB_HUD' key commands like normal. We could give the light effects a dissipation rate range of maybe 200ms - 2s, with nine steps of 200ms in between (200 -> 400 - > 600, etc). It's a very pretty effect, and one that I think would be great to include here.

[ice9js]

I've been thinking about the kind of data structures that make sense to support this.
Specifically how @joelspadin already pointed out, it'd be nice to have the animation 'engine' support different LED drivers and combinations of them. I'd even expand on those guidelines:

• One IC should be able to support multiple logic groups of the LEDs as well. Like, per-key-rgb AND underglow to give an extreme example. With the new ISSI chips, one or at most two would certainly be enough to support all of your lighting needs and so it'd be cool to take advantage instead of needing to use and support additional hardware.
• Make it possible to have different LEDs follow different animations even if the behavior is the same. One might have a different effect on the bottom and on the top, or define multiple light strips each responsible for its own thing. Effectively, fine grained control.
• Eventually all of this should be reconfigurable on the fly without having to flash the board.
• _It might be interesting to expose an API for other parts of the firmware to control the animation/parameters for a given area. Like, using your underglow LEDs as a charging indicator or supporting power on/off animations.

Let's assume we'll be using Zephyr's led_strip API as a common starting point.
A single GPIO-driven LED can be represented as a strip with length of 1, a driver would be simple enough. Similarly, LED matrices can be transposed into a 1d array and vice versa.
Thus, we will need a pixels[] array:

This would be a 1D array containing the values for all LEDs on the board.
In the example above, LEDs 0...m-1 are driven by one IC while m...n are driven by another. Organizing it this way allows to pass the appropriate pointers straight to led_strip.update_rgb() saving memory and processing overhead.

That was the easy part. Now, let's add a second pixel_meta[0;n] array in parallel. This array would contain all the necessary settings for each pixel. For the purpose of illustration, I'm going to portray it as an array of structs. However it could just be multiple arrays containing individual props if getting the right device tree values into the right places proves tricky.

The meta would look something like this: (pseudocode)

meta[0...n] {
    animation:
        Pointer to the animation function

    key:
        the key identifier or NULL if it's not a key LED

    position:
        x: [0;255]
        y: [0;255]

    flags: ?
}

Let's take it step by step:

• Animations would be functions. Each could potentially live in its own file and maintain its own state necessary to keep it going depending on what kind of animation it might be - not interfering with any other ones that might be going on. It would be called with the pixel meta and expected to return the pixel values for the new frame.
• Key marks what key the LED belongs to. Which can be used for overriding the default animation. In which case we might need another alt_animation property or something?
• Let's assume your keyboard PCB is the reference plane, in which case these would be the relative x and y coordinates. Keyboards aren't squares though so potentially, somewhere, an x/y ratio needs to exist as well to be references by the animations if necessary.
• Any additional flags we might need I guess? Could be useful for dedicated indicator LEDs or to account for any edge cases.

So this is more or less my angle on it, I'd be curious to hear what others think before I start making inroads towards an actual implementation.

One catch I can already think of would be that in this state, it'd not allow for a dynamic animation from one key to override the animation on another key unless it was already assigned to the same animation.
Or we could force all LEDs through all active _key pixels' animations and merge the result. Flags would probably be a good candidate to define the merge behavior on a per-pixel basis. In case, say, you have some indicator LEDs in the middle and don't want them to be affected by that.

[joelspadin]

I was thinking that rather than having each key hold a pointer to an animation function, you could implement something like an animation driver API and then have each animation be an instance of a driver that implements it. Then you could set up the animations like layers and enable/disable each one individually. Every time you want to update the LEDs, you'd loop through the enabled animations and just call one update function on each, giving it a pixel buffer (and maybe also a list of which pixels it's assigned to update?).

Then you could easily implement things like an "animation" that just takes the output from the previous animation and zeroes out every key that doesn't have a function on the current layer, or one that overrides just the caps key based on the indicator state. As a bonus, using the driver API means each instance of the driver maintains its own state, so you could make much more complex animations like one that listens to key events and lights up the keys you press. I think that would eliminate any need for flags or an alt_animation property. Instead of flagging a key as being an indicator, you'd just put a devicetree property on the indicator animation to tell it that key 20 is caps lock.

Animations should also be able to tell the animation system when they need to update. Say you have a solid color animation as the base, an animation that changes based on the layer, and one for indicator LEDs, then you only need to refresh when the layer or indicator state changes. The animation drivers could listen for ZMK events and then call "request animation update" function which queues up a work item to do an animation update.

For animations that smoothly animate instead of just updating on specific events, you wouldn't want to have multiple animations asking the animation system to update on their own timers. If you had two animations requesting updates at different frequencies/phases, you'd update far more often than you need to. To handle this, you could have a global periodic timer, and have the animations' update functions return a Boolean. If any animation returns true, then the global timer will schedule another update. If not, the timer will stop and wait for an animation to manually trigger an update. (Would also let us do other optimizations like throttling the update rate when on battery if that ends up being a big power user.)

[The-Briel-Deal]

Update on this, it looks like there's a implementation here diff here, this should cover 3 out of the 5 use cases covered above. (atleast for the the glove 80)

I'm going to dive into this tomorrow and see if I can generalize the implementation and make it more general so it can cover other use cases.

My goal is to just make an easy to use interface so its easy to change lighting in a keymap for a first pass.

[steynh]

I think in general it would be really useful if we could specify an RGB grid for each layer. That way it becomes possible to visually highlight groups of keys (such as a numpad block) in different layers. And then it also becomes very obvious which layer is active. Both of these would be so helpful when just getting used to a new layout.

Animations are cool to have, but they seem quite complex to implement. Whereas just a simple grid per layer seems relatively simple to implement and actually offers a lot of practical benefits.

[darknao]

Update on this, it looks like there's a implementation here diff here, this should cover 3 out of the 5 use cases covered above. (atleast for the the glove 80)

I'm going to dive into this tomorrow and see if I can generalize the implementation and make it more general so it can cover other use cases.

My goal is to just make an easy to use interface so its easy to change lighting in a keymap for a first pass.

You might also be interested at https://github.com/moergo-sc/zmk/pull/30 which is based on the implementation you mentioned but taken a bit further. It's still specific to glove80, but I think it needs a bit less work to make it available to more boards.

[joelspadin]

I still think we should provide a generic driver API for setting RGB LED colors with optional animations. We would have the ability to implement "animations" that don't constantly animate, for example with an animation driver that stores a bitmap per layer and requests a new frame any time the layer changes.

My current thoughts on how this might work (a simplified version of my comment from a few years ago):

• The animation driver API would consist of the following functions:
  • "Enable" and "disable" functions to tell a driver that it is/isn't being used.
  • A "render" function which is provided a pixel buffer and returns a value indicating whether the animation should continue.
• Animation control code would have a reference to a "root" animation driver.
  • Upon enabling lighting, it would call this driver's render function.
  • As long as the render function continues to return true, the animation control code will keep rendering on a timer.
  • If the render function returns false, the animation control code will stop rendering.
  • Animation drivers can call a "request frame" function to restart rendering.
• Animations could be composed by having one driver's render function call a different driver's render function. This would allow for creating just about any behavior you want from a very simple API. For example (python-ish pseudocode):

# caps lock animation
class CapsLockAnimation(Animation):
    child: Animation
    leds: list[int]
    color: Color

    def render(self, bitmap):
        # Draw the child animation
        continue_anim = child.render(bitmap)

        # Then overwrite some pixels if caps lock is on
        if zmk_caps_lock_active():
            for index in self.leds:
                bitmap[index] = self.color

        return continue_anim

def on_caps_lock_changed():
    # If the animation is stopped, we need to restart it any time caps lock changes
    zmk_request_animation_frame()

# layer animation
class LayerAnimation(Animation):
    # Set an animation for each layer
    children: list[Animation]

    def render(self, bitmap):
        # Pass through to the animation for the highest active layer
        layers = zmk_keymap_layer_state()
        return self.children[highest_bit(layers)].render(bitmap)

def on_layer_state_changed():
    # If the animation is stopped, we need to restart it any time the layer state changes
    zmk_request_animation_frame()

# solid color animation
class SolidColorAnimation(Animation):
    color: Color 

    def render(self, bitmap):
        # Set every pixel to the same color
        for i in range(len(bitmap)):
            bitmap[i] = self.color

        # Colors aren't changing over time, so stop animating
        return False

# color cycle animation
class ColorCycleAnimation(Animation):
    speed: float = 0.1
    hue: int = 0
    last_timestamp: int = 0

    def enable(self):
        self.last_timestamp = now()

    def render(self, bitmap):
        # Rotate the hue according to the time elapsed since the last frame
        time_elapsed = now() - self.last_timestamp
        self.last_timestamp += time_elapsed

        self.hue = (self.hue + time_elapsed * self.speed) % 360
        color = hsv_to_rgb(hue, 100, 100)

        # Set every pixel to the same color
        for i in range(len(bitmap)):
            bitmap[i] = color

        # Animation runs forever
        return True

# Keyboard-specific setup
layer0 = SolidColorAnimation(color=WHITE)
layer1 = SolidColorAnimation(color=RED)
layer2 = ColorCycleAnimation()

layers = LayerAnimation(children=[layer0, layer1, layer2])

root = CapsLockAnimation(leds=[1], color=YELLOW, child=layers)

(For brevity, I've omitted several details, like how an animation's enable function should call its child's enable function, or how the layer switching animation should enable only the child animation for the active layer and disable the others.)

[AnthonyNguyen168]

I wonder what is making this feature takes so much time. I am a developer and really need this feature for my new keyboard design. Since AFAIK, QMK does not supporting both RGB-per-key and Bluetooth on one PCB. So if ZMK support it, our PCB can support both these features on a single PCB.

[joelspadin]

None of the core maintainers are actively working on this because other features are higher priority. That said, there's nothing stopping anyone from implementing this if they feel strongly enough that it should exist.

[The-Briel-Deal]

Sorry about not updating this. Looked into this a bit and ended up getting sidetracked. Will hopefully try my hand at this again this weekend.

[dbodyas]

I'm building a split keyboard and now I'm on stage of wiring leds. What option should I choose or it doesn't matter?

[joelspadin]

If we implement things properly, the order should not matter. For simple things like solid colors, the order doesn't matter at all, or for things like a caps lock indicator, you should be able to choose the specific LED indices that are correct for your keyboard.

For more complex animations, we'd need a way to associate LEDs with physical positions and/or specific keys. The work Pete is doing in #2283 could be useful for this. For example, an animation that does an RGB wave horizontally across the board could simply rotate the hue of each LED according to its X coordinate if we have that information.

[gpmontt]

I think the project from Nick Coutsos would be a great place to tweak whatever color you want in the appropriate key. We need to a new way to define how many led are in the strip in a better way like

&led_strip {
    chain-length = <NUMBER_OF_LED>;
};

whatever.conf

NUMBER_OF_LED=<xx>