search

KMKfw / kmk_firmware

Clackety Keyboards Powered by Python
https://kmkfw.zulipchat.com
Other
1.42k stars 480 forks source link

Add support for displays #163

Closed dzervas closed 2 years ago

dzervas commented 4 years ago

Mainly I'm talking about I2C/SPI OLED displays but more can come later on. It's a big undertaking and these decisions have to be taken:

knox1138 commented 3 years ago

If someone can show me how to get which layer is active I can work on some of these.

kdb424 commented 3 years ago 
from kmk.extensions import Extension
from kmk.kmktime import ticks_diff, ticks_ms

class WPM(Extension):
    def __init__(self, debug=False):
        self.enable = True
        self.history = [0]
        self.timeout = 1000
        self.checkpoint = ticks_ms()
        self.debug = debug
        self.counter = 0
        self.old_wpm = 0
        self.wpm = 0

    def on_runtime_enable(self, keyboard):
        keyboard.wpm = 0
        return

    def on_runtime_disable(self, keyboard):
        keyboard.wpm = None
        return

    def during_bootup(self, keyboard):
        keyboard.wpm = 0
        return

    def before_matrix_scan(self, keyboard):
        return

    def after_matrix_scan(self, keyboard, matrix_update):
        if self.enable:
            new_history = self._add_character(
                matrix_update,
                keyboard.secondary_matrix_update)
            if new_history is not None:
                self.history.append(new_history)
            if len(self.history) > 10:
                self.history.pop(0)
            self.wpm = self.calculate_wpm()

            if self.wpm != 0 and self.wpm != self.old_wpm:
                keyboard.wpm = self.wpm
                if self.debug:
                    print(f'WPM: {self.wpm}')
                self.old_wpm = self.wpm
        return

    def before_hid_send(self, keyboard):
        return

    def after_hid_send(self, keyboard):
        return

    def on_powersave_enable(self, keyboard):
        self.enable = False
        return

    def on_powersave_disable(self, keyboard):
        self.enable = True
        keyboard.wpm = self.wpm
        return

    def _add_character(self, matrix_update, second_update):
        if ticks_diff(ticks_ms(), self.checkpoint) > self.timeout:
            self.checkpoint = ticks_ms()
            ret = self.counter
            self.counter = 0
            return ret
        if matrix_update or second_update:
            self.counter += 1

    def calculate_wpm(self):
        # No, there is no real math here, though it seems to work out. I'm
        # not sure why this reports about the same as QMK
        return int(sum(self.history[:-1]) * .6)

This code is incomplete, but here is an example of what I started with for WPM. I used Circuitpython's OLED libraries in order to output the text. This work is all based on this branch. https://github.com/KMKfw/kmk_firmware/tree/topic-merge-keyboard-and-state-rebased

I believe that layer status is exposed in that branch to extensions in an immutable state as to protect the core as well.

pullenrc commented 3 years ago

@knox1138 For testing you can get the current active layer by calling self._state.active_layers[0] after the while loop starts in Keyboard.Go()

knox1138 commented 3 years ago

That's actually what i was going to try tonight. Thank you.

On Fri, Jun 11, 2021, 11:12 AM pullenrc @.***> wrote:

@knox1138 https://github.com/knox1138 For testing you can get the current active layer by calling self._state.active_layers[0] after the while loop starts in Keyboard.Go()

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-859650908, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4HPSAOFWOYWZJ5FHPTTSIRV5ANCNFSM4RMCPMKA .

knox1138 commented 3 years ago

Apparently InternalState.active_layers was the correct thing to call

On Fri, Jun 11, 2021, 11:12 AM pullenrc @.***> wrote:

@knox1138 https://github.com/knox1138 For testing you can get the current active layer by calling self._state.active_layers[0] after the while loop starts in Keyboard.Go()

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-859650908, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4HPSAOFWOYWZJ5FHPTTSIRV5ANCNFSM4RMCPMKA .

klardotsh commented 3 years ago

This conversation reminds me that topic-merge-keyboard-and-state-rebased really needs merged. @kdb424 should we just do an IRL hackathon some week soon to iron out the split KB bugs? I think that's all that's blocking merge, and getting that in would really help clean up efforts like this

kdb424 commented 3 years ago

I can work on the wired side, but I don't even own controllers for the bluetooth side anymore. We'll have to backport a few changes as well.

klardotsh commented 3 years ago

I have Feather nRF52840s somewhere in the pile of crap in my kitchen, I think I have at least one NiceNano somewhere (maybe in one of the Irises?), and I know I have at least one MakerDiary MDK, so we can somehow figure this out. Wired side I'm good to hack on whatever, I have plenty of M4 hardware laying around

knox1138 commented 3 years ago

I will say looking at gigahawks example of a merged keyboard and state is alot easier to follow. Not that I have any better idea what I'm doing, but I get a better idea of how it all works together. I've spent more time this weekend than I want to admit trying to figure it all out. Also, at this point, anyone who professionally codes and isn't an extreme drug addict or chainsmoker deserves a medal.

pullenrc commented 3 years ago

I'm gonna get on that merged state business tonight, looks interesting. No drugs, but heavy, liberal use of scotch doesn't hurt. I have started some oled integration, with mode badges and key logging, I'll pop that into the rebased state and see if its worth a pull request.

knox1138 commented 3 years ago

I'd love to see how you pulled the data for the oled. For some reason I have a much easier time with programming displays than figuring out the right syntax to get tye data I want.

On Mon, Jun 14, 2021, 9:46 AM pullenrc @.***> wrote:

I'm gonna get on that merged state business tonight, looks interesting. No drugs, but heavy, liberal use of scotch doesn't hurt. I have started some oled integration, with mode badges and key logging, I'll pop that into the rebased state and see if its worth a pull request.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-860697728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4GO53QYHKGJ534NK2TTSYB3XANCNFSM4RMCPMKA .

pullenrc commented 3 years ago

I had to create some place holders, and some new classes for the oleds. The mode information for the master branch is accessible in the class InternalState, via a call to self._state.active_layers[0], (the first index of the list, as its prepended to when the mode is changed). It's self._state instead of InternalState because of the instantiation on line ~385 of kmk_keyboard.py. Check out my fork for more details. https://github.com/pullenrc/kmk_firmware

knox1138 commented 3 years ago

Ok. I'll try some things cause it seems like that's where I should be calling it from. I just had no luck with it. It's more than likely I was doing something incorrectly. On Mon, Jun 14, 2021, 11:20 AM pullenrc @.***> wrote:

I had to create some place holders, and some new classes for the oleds. The mode information for the master branch is accessible in the class InternalState, via a call to self._state.active_layers[0], (the first index of the list, as its prepended to when the mode is changed). It's self._state instead of InternalState because of the instantiation on line ~385 of kmk_keyboard.py.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-860770554, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4BYMPWPX32JKQCQKLDTSYM2TANCNFSM4RMCPMKA .

knox1138 commented 3 years ago

What does instatiation mean? Cause you have nothing on line 385 of your keyboard.py. Either way, after double-checking calling InternalState.active_layers[0] works for me, and self._state.active_layers[0] doesn't.

On Mon, Jun 14, 2021, 11:20 AM pullenrc @.***> wrote:

I had to create some place holders, and some new classes for the oleds. The mode information for the master branch is accessible in the class InternalState, via a call to self._state.active_layers[0], (the first index of the list, as its prepended to when the mode is changed). It's self._state instead of InternalState because of the instantiation on line ~385 of kmk_keyboard.py.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-860770554, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4BYMPWPX32JKQCQKLDTSYM2TANCNFSM4RMCPMKA .

pullenrc commented 3 years ago

My bad, the line number is 201 from the master branch, and 319 in my fork. Also, I assumed your comments were based on the master branch, as that was what I was working out of. Instantiation is creating an instance from a class. InternalState is the class, self._state is the instance.

knox1138 commented 3 years ago

Ok, I'll check all that out after work when I get home. For context, I have about a month of self-taught experience with python as my first programming language, so while I can do things like get an oled to display what layer is active I rarely understand why what I do works and if there is a "proper" way to do it.

On Tue, Jun 15, 2021, 8:46 AM pullenrc @.***> wrote:

My bad, the line number is 201 from the master branch, and 319 in my fork. Also, I assumed your comments were based on the master branch, as that was what I was working out of. Instantiation is creating an instance from a class. InternalState is the class, self._state is the instance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-861467851, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4DDNWEJH5UK55X7GQTTS5DSZANCNFSM4RMCPMKA .

pullenrc commented 3 years ago

I hear ya, I am self taught as well, and by no means a pro. Just been banging my face on the keyboard longer.

knox1138 commented 3 years ago

" just banging my face on the keyboard longer"... Lol! I needed that, and understand that too well.

On Tue, Jun 15, 2021, 10:15 AM pullenrc @.***> wrote:

I hear ya, I am self taught as well, and by no means a pro. Just been banging my face on the keyboard longer.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KMKfw/kmk_firmware/issues/163#issuecomment-861536252, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUDDR4A4M3DCPSTGCAIRZF3TS5N6PANCNFSM4RMCPMKA .

Tonasz commented 2 years ago

What is the status of it? I guess everybody is rolling their own implementation on forks or OLED is a bit forgotten as most picked KMK for BLE, for which OLED just drains battery.

Anyway, I started implementing my version of OLED, a bit different than current long time ago touched PR and not using displayio (as I currently have to stick with bitbangio for i2c). It may resume a discussion here or not. https://github.com/Tonasz/kmk_firmware/commit/9ab73315241ff4197613e782a52d657cf5a60529

That's only a draft, but my idea was to have main class which handles initialization and scene change (currently I change them using key codes) + scene classes. Each scene class receives a keyboards sandbox (like extensions) and display handle to draw on it whatever it wants. I assumed only 1306 oled for now, but there could be additional level of abstraction, so scenes would receive display-independent object for drawing.

Sample scenes which come to my mind:

giovanniborella commented 2 years ago

@Tonasz having a scene-like structure would be great for building menus, maybe an overlay for a toolbar-type of thing. Any thoughts on support for animations? what FPS are we looking at?

Tonasz commented 2 years ago

My above code (from previous comment) was based on framebuf implementation, but when I started adding stuff and do some simple measures, I noticed that it won't be probably fast enough for animations. For example text was rendered so long, that when I enabled layer info label, I was loosing key presses. It was better for pixel drawing, but I don't have answer for your question, especially as I never really planned any animations on OLED (maybe expect of few frames of bongo cat).

Intro

I transformed above linked code to DisplayIO driver version and I'm more happy with it. I don't feel sure with that code (if I were, I would just start PR) but I'd like to come back to my above idea, expand it and provide another example.

https://github.com/Tonasz/kmk_firmware/blob/display/kmk/extensions/display.py https://github.com/Tonasz/kmk_firmware/blob/display/docs/display.md

I still feel that people generally handcraft their OLED drivers, but I think we have to provide bare-bone working example. My current concept resolve around simple display handler and example scenes.

Scenes

Each scene is closed thing doing one thing. In above sample I got StatusScene (which display current layer and rgb mode, but things like caps lock, WPM etc are good fit), KeypressesScene (not practical, but fancy way of showing which button was pressed) and BitmapLogoScene (just static image). We would need to set up on very basic and common stuff and not bloat repository which complicated scenes which could be hard to maintain. If somebody would like to share his cool OLED code, which shows status using images or scene which shows CPU/RAM usage of PC, he can just share his repository code with encapsulated scene. But on the other hand, person using common screen could just hook it up and import example scenes working out of box, before committing to further fiddling.

Implementation-wise, when I moved to displayio I found out that each scene can be just a Group, what simplify stuff switching scenes or displaying multiple scenes on each other (overlay toolbar mentioned above). It looks like diplayio also take care of what to refresh and what not, so it would be a lot easier to implement stuff, but I'm not sure how good it handles low level stuff what could make difference with animations. Not yet implemented, but each scene could define it's own auto-checked requirements like width, height or color palette of screen.

Display handler

Simple class which handles display initialization and scenes switching. Currently, it is kinda tailored for SSD1306, but I see displayio is perfect abstraction layer if we would like to more generic support. Most screens have official or community implementation of displays, which afaik are simple wrappers for initialization or commands like sleep. Displayio handles I2C or SPI communication. We could decide if we want user to just pass such object (like adafruit_displayio_ssd1306.SSD1306 or CircuitPython_GC9A01.GC9A01 which extends displayio.Display) or have predefined collection enum (the latter is less generic but can simplify providing palettes, screen sizes and so on).

I appreciate feedback about what do you think about whole concept, current prototype or vision of having OLED display at all. I'm also aware that at least @xs5871 got some display stuff with WPM implemented, so I'm curious his implementation. Especially as there already was one PR, which were discarded.

xs5871 commented 2 years ago

I'm also aware that at least @xs5871 got some display stuff with WPM implemented, so I'm curious his implementation.

Sorry for the confusion. No display, but I've got a WPM extension on the way. As always, it's down to the pareto principle standing in my way.

ldsands commented 2 years ago

I don't know how helpful this is, but I was able to try out @Tonasz's implementation of display on my split kb (Sofle). Overall, I think it worked well, thank you for your work on it. I have two notes about my experience with it.

edit: The issue with layers not updating was an error on my part. I removed that from the comment.

kdb424 commented 2 years ago

Closing as there is at least one OLED module, and I've written several custom ones as tests alone. I don't believe that abstraction on this to be hyper generalized will lead anywhere good as it is already fairly ram intensive as is. If someone steps up to prove me wrong, please reference this PR, but for now, I think it's in a good place.